It takes forever compared to driving in most cases, catching the transit often involves a walk, and a lot of public transit is kind of grungy. This doesn't apply or matter as much in some cases (e.g. the NY subway) but in general it's not great, at least not in the USA.
Does transit not always involve a walk, even a small one? It's not like everyone lives next to a subway station, and most destinations aren't right next to a subway station either.