Rotation for speed/efficiency/compression reasons (indeed with PNG's horizontal line filters it can have a compression reason too) should have been a flag part of the compressed image data format and for use by the encoder/decoder only (which does have caveats for renderers to handle partial decoding though... but the point is to have the behavior rigorously specified and encoded in the image format itself and handled by exactly one known place namely the decoder), not part of metadata
It's basically a shame that the exif metadata contains things that affect the rendering
Most modern camera modules have built in hardware codecs like mjpeg, region of interest selection, and frame mirror/flip options.
This is particularly important on smartphones and battery operated devices. However, most smartphone devices simply save the photo the same way regardless of orientation, and simply add a display-rotated flag to the metadata.
It can be super annoying sometimes, as one can't really disable the feature on many devices. =3
the main reason is probably that the chip is already outputting the image in a lossy format, and if you reorder the pixels you must reencode the image which means degrading the image, so it's much better to just change the exif orientation.
JPEG can be rotated losslessly. `jpegtran` can do it, for example (and comes with a script called `exifautotran` to automatically normalise the orientation of a bunch of JPEG files at once).
Burst mode in cameras means the sensor is readout is buffered in RAM while the encoding and writing to persistent storage catches up. Rotating the buffer would be part of the latter and not affect burst speed - and is an insignificant cost anyway.
If a smartphone camera is doing it, then bad camera app!