People are not resources. They aren't being "taken", unless you're talking about literal kidnapping and forced labor here (which, I believe, last time happened during Operation Paperclip, if we're talking about US). They come to US, because they believe it is a better place for them. If they wanted to "help elevate their own countries", they'd stay there. Coercing them to do so (including indirectly by depriving them of the opportunity to make that choice in the first place) is unethical.

Speaking as an immigrant myself, I don't believe my country wants me. It might want my skills and abilities, sure, but it seems to strongly dislike my liberal political views, and they can't have one without the other. So they get neither.