You don't need them, but they do make things easier for a lot of people. With Android, at least, you can do plenty from the command line if that's your jam.
My problem with them is not so much that they are GUI tools. It's more that they are bloated, low-quality and a bit unpredictable. IMO and IME, of course. They do get the job done, and people can get used to them if they use daily. But if using them is not your daily job (and Stockholm Syndrome hasn't set in), they make for a terrible developer experience. They take a lot of time to setup and there's often various problems with versioning, for example. All IMO and IME, of course.
I used to work in a Cordova/PhoneGap/Ionic/[whatever the name is today] app I had to make those bi-monthly excursions to the codebase that would always take a couple days because of Android Studio or Xcode. Setting the tooling in a new computer or teaching this to a new developer would require a lot of fiddling with version for half a day or more until it worked properly.
Sure, if you work on it everyday it doesn't suck, but working with multiple apps or working with different things was always a terrible experience.
I only used Android Studio when it was in beta so can't say much about that. But XCode is honestly quite good, not perfect by any means, but especially with Swift and SwiftUI it has some really good features (to wit: live previews).
Provisioning and testing purchases is always a mess, but that's mostly because the code world meets politics there.
But that sounds like a function of the complexity involved with mobile development and those tools being a do it everything tool for their development (with Android especially, all the different os versions you can target, the different form factors). What would you like for casual use?
The problem is not the form factors or the depth of the tooling. I'm not even going into different form factors.
The major issue here is how the default experience is unpredictable and messy, and restoring/kickstarting a proper development environment takes hours/days instead of minutes, and it breaks quite often if you're
What I would like is predictable builds and isolation. Packaged, versioned and rollback-able development dependencies, like we have with backend/frontend web tools.
Things like Cordova's CLI and Unity's Build Server help halfway with this, but there's more work to be done even with those tools.
What I'm suggesting is that some company could step up and handles this, plus things like CIs, deployment bureaucracy, instant demos of web apps on phones, etc, like someone mentioned above that Expo does for React Native.
