The gain, as far as local AI goes for Google, is that, at Google scale, the CPU/GPU time to run even a small model like Gemma will add up across Gmail's millions of users. If clients have the hardware for it (which Pixel 9's have) it means Gmail's servers aren't burning CPU/GPU time on it.
As far as how Gmail's existing offline mode works, I don't know.
To make the discussion easier, we can look at the state of GMail before widespread AI was a thing.
(I have to use some weaselwording here, because GMail had decent spam detection since basically forever, and whether you call that AI or not depends on where we have shifted the goalposts at the moment.)
Heh that brings to mind that old chestnut, something like "We call it AI until it works, then we call it machine learning." Google's running the servers tho, so the assumption is the servers would be running the spam filter against it before the user. In my mind, anyway. So it's fair to point out that Google also has the compute resources to not have to care. To the hypothetical though, could Google make Gmail a local-first? I say yes. Will they? I doubt it.
As far as how Gmail's existing offline mode works, I don't know.