I've seen a bunch of this years ago. It was caused by stress in the barrel steel. I though I had read that that problem didn't exist any more as all factory barrels are stress relieved. I shoot three shots fairly fast then let the barrel cool while I walk down range and check the target. Barrels gettin hot on me, and sometimes they do, haven't effected grouping in years. And I sure would not rule out the tight barrel channel. I float all my barrels till I can slide a piece of printer paper doubled over down it all the way to the action. I have read where bedding the barrel tight all the way works well also, I've never tried it. Bedding under the chamber then floating from there works and I did it a lot in years past. But it's a little tricky as the point where the bedding material stops has to but really straight across the stock. If not, your gonna see the barrel bouncing off it and it won't shoot well at all. I used to mask off the barrel to the front of the chamber with two layers of tape, it made the barrel fairly tight under it but it also made a square dam to hold back bedding compound. Then remove the tape when cured and the float is there.
One other thing to check is that the action is well seated in the stock and the screws are adequately tight. If not you'll get a two group situation caused by the action moving in the stock when fired, doesn't have to move much. If you do have a bedding problem, and you might not, you'll never fix the problem till you fix the bedding. First place I'd look.