I had a .300 Win Mag and my personal experience with mine was that the warmer the barrel got, the worst the groups became.  I always went to the range, sighted it in, then cleaned it real good.  After I cleaned it, I would shoot two to foul the barrel, then check to ensure the sight's were still 'on target'.  After that, I cased the rifle, and went hunting.  I never had a issue hitting my target while hunting using this method, but it's the only rifle I've ever owned that acted this way.  For whatever reason, after I put about 12 - 15 rnds. through the barrel, it wouldn't shoot accurately any more until I cleaned it. Another factor to consider is that you're shooting factory loads, and the warm barrel could be affecting the powder.  Since factory loading's don't have the same stringent measures we use as hand loaders, that may also be affecting the accuracy of your rifle.  Just my two cents, I hope this helps you out some how, I know how frustrating it can be when a rifle acts the way you've described.

When ever a barrel starts to heat up and some even a little bit your point of impact will start to change.  The way that I sight in a rifle is to fire 3 rounds and then let the barrel cool completely then fire 3 more and let it cool.  When you start to change the scope settings when the barrel is hot you are just chasing a greased pig. 

One thing, have you ever noticed that the small caliber varmit rifles usually have a real heavy barrel.  They do this so that it takes longer for it to heat up, but it will also take it longer to cool off once you get it hot. 

I have a couple of rifles that the groups open up quickly after as little as four or five shots. A sporter weight .220 Swift being the worst. But in every case the groups just get larger and larger as the heat builds but not a real total shift of where the group prints like your situation. Sounds like a different issue of some sort to me.

I agree with Critter though and only fire 3 to 5 shots and let the barrell cool down all the way. I have no rifles that I could fire 12 rounds without causing major accuracy problems. If it's that hot things will be moving around some.

That's why I always try to bring several rifle so I have one to shoot while the others are cooling off as it does take some time.

Thanks for all the input guys. Critter, your explanation sounds the most logical for my experience. It's a Browning XBolt with a factory rubberized camo synthetic stock. From the outside the barrel looks to be floating nicely but I can't see what happening under the barrel. Since the groups weren't opening up, they shifted right to left, I'd have to go with your thoughts or I'm still thinking in my own weird way that since the wind was right to left and cooled the right side of the barrel quicker than the left the right side would contract faster slightly curving the barrel to the right. Dunno, I'm no scientist. But the shot groups were all pretty much within one half inch of a vertical line which tells me my breathing is what was affecting shot placement in that regard, the groups just moved 2 MOA across the paper each time.

the idea that one side of the barrel is cooling faster than the other based on the wind assumes thatwind flow is laminar. in fact, the wind is turbulent, meaning that there are minor currents mixed with larger ones. the wind is tumbling across the land, rather than flowing accross it.

i think it would be more plausible to say that when your gun heats up, the metal changes more on one side of the barrel because the bore is not perfect down the center of the barrel. when my 243 heats up, it pulls left. i think, because there is more metal on the right side of the barrel, so when it heats, it expands more than the other side with less metal.

i have thought about this for many hours and this was the best i can come up with.

I don't think that it is the more metal on the barrel but rather more wood on the stock. 

Do the piece of paper trick on your cold barrel.  Then after shooting it a few times do it again on the hot barrel and see where it tightens up.  As the metal gets hot it will expand and come in contact or almost come in contact with the wood or fiberglass of your stock.  Once it touches or gets close it will start to affect accuracy as the barrel vibrates when you shoot it and it touches the stock. 

Good idea with the paper. Next time I'm out and getting things heated up I'll try that.

Thanks again for the thoughts!

I have the same experience as most of you do. After shooting till the barrel gets warm the groups open up. After I let it cool down the groups tighten back up.