Yeah, I find that odd too. I'm relatively young and far, far from a seasoned hunter or shooter. However, I can't think of a good reason that a rifle would become inaccurate. Just doesn't make sense to me. Maybe you could fire 20 rounds in two minutes and cause your barrel to warp? But beyond abuse like this, why would a rifle permanently lose it's ability to group shots?
As for the manufacturing problems (burrs in the barrel), I think that's a crap shoot. Every manufacturer is going to produce a lemon once in a while and it's always possible that it will be the one you end up with. Weatherby Vanguard does have the prefired grouping, so that would seem to be pretty good assurance that the rifle shoots straight.
Otherwise, I still think that you're worrying too much about this (my opinion, obviously). Grab a Remington, Browning, Savage, Winchester, etc. and they'll all shoot pretty darn good out of the box.