For a varmit rifle, I'd take the 22-250. The 25-06 works great but I see it more as a hunting cartridge. You could make whatever you want into a varmit rifle, I used a 338 win mag years ago. It was a hoot but a day with it sored ya up some. As you know I use a 243 now and I like it.
.22-250 or the .25-06. opinions on which is better? Would be for an all around varmit rifle.
Of the 2 you mentioned the .22/250 would be best - the key words are "all around". The .25/06 is a great cross over cartridge when deer are on the menu. Ditto on what ChesterGolf said about the .243, one my all time favorites.
If you're lucky enough to be on private land and can leave your stand up during the off season you can tie an old piece of cloth or something of the sort to your stand. I like to use an old camo bandana that I have. That way the deer get used to the movement and eventually don't bother even looking up when there is slight movement. This may just give you the time to draw your bow or raise your gun without being detected.