Do I need a huge antenna on my roof, or a Big Tower in my backyard?
That said, getting a good antenna up is important. It's more important than having a lot of power, especially for VHF and UHF. But it doesn't necessarily have to be overly large, nor on a big tower.
The universe of antennas (should that be "antennae?") is a large one, and they come in pretty much all shapes and sizes. Odds are that there's one that will fit your needs, regardless of your operational requirements and constraints.
One of the biggest antenna challenges comes for those who live in apartments or in any place subject to covenants about antenna construction. (The good news on that last point is that the Amateur Radio Parity Act of 2017, if passed, will prohibit HOAs and the like from preventing hams from putting up antennas, if that prohibition makes communication impossible). There are, however, a lot of clever "stealth" antennas that can work even in very restrictive environments.
VHF and UHF are line-of-sight bands, so you need, theoretically, a good clear path free of high obstructions between you and the repeater or the individual ham you want to communicate with. The antenna itself can be of modest size; a popular 2m/440 J-Pole design has as its longest element a piece a little under 5' in length.
Once you have picked an antenna, you'll often need to experiment with its exact placement, but the upside of that is that you may well be able to find a good spot that isn't on top of your roof (or equivalent hight on a tower).
"Tower" can also take different forms. Some hams have had good results with simple aluminum flagpoles as the supports for lightweight antennas (though if you go that route, it's a good idea for the flagpole to be braced against something sturdy partway up, rather than free-standing. But it's also worth pointing out that the wind load on a 2m/440 J-Pole is less than that of the flag for which the pole was designed.
If you want to put up a rotatable HF beam antenna, then you do need a tower or at least a stout roof structure on which to mount the rotator and antenna. Most beginning hams don't put up this kind of antenna, so it's less a requirement than something to aspire to!