I think it comes from experiences of coming in to codebases and seeing that people are using the ORM for everything, even when it's obviously suboptimal and raw SQL would be better.

Compromise, best tool for the job, nuance, etc - all tend to get tossed out when people make tech decisions. Someone using an ORM may not even be aware that raw SQL is possible.