I feel the same way about synthetic data. Seems intuitively wrong that you can get new insights / unlock new abilities from generated data that you could not from the original data.
The new information comes from our choice in how to generate that data. We're not just blindly making synthetic data, we come up with clever way to generate synthetic data that is hopefully high quality and can improve our models (and if it doesn't, we don't use it).
