I've got a very nice mirrorless camera and glass and eventually came to the same conclusion: It's just not worth the hassle even for the improvement in image quality.
However, I have found that it's absolutely worth it to upgrade to a better microphone. Just about anything is better than the mic built into most computers and better voice quality will give you more presence and make it significantly more enjoyable for others to listen to you. Wearing headphones also helps so that the computer isn't forced to do echo cancelation on the signal.
A decent dynamic mic handles background noise well by design, as opposed to using software to attempt to process the dirty signal coming from a tiny ECM. I can comfortably take a call on my setup while people are talking next to me and with the TV on. If you want to go further there are ENG mics which can handle even noisier environments - think an interview in the middle of Times Square.
A mixer or interface also allows you to monitor the microphone signal in your headphones or speakers. It keeps your tone and volume more consistent and you'll immediately notice if you accidentally back away from the mic (as opposed to the other side complaining they can't hear).
I got a highish quality webcam (the Dell one with no mic, I use airpods) for this reason. It looks MUCH better than a built in webcam, not up to the level of a mirrorless camera, but not that you can really tell on a compressed stream.
Every other day I have someone ask me about my mirrorless setup (I frequently have calls with new people), it is night and day to my expensive waste of a webcam.
However, I have found that it's absolutely worth it to upgrade to a better microphone. Just about anything is better than the mic built into most computers and better voice quality will give you more presence and make it significantly more enjoyable for others to listen to you. Wearing headphones also helps so that the computer isn't forced to do echo cancelation on the signal.