Turing's original paper is worth a read here. He already pre-empted a lot of the objections back then. You are right that Turing's original test is much harder than just passing a few shallow softball questions.
However, I do count responding well to softball questions as important markers of our progress. Ten years ago, no one was able to pass even this mark.
(And from a commercial point of view, good enough 'understanding' and responses might be good enough for many tasks.
For example, I am a native German speaker, but I found that Google Translate often already does a better job of translating English to German than I could do, unless I spent unreasonable amounts of time on the task.
At the moment the most efficient way for me to produce a translation is to let Google Translate do the work, and then proof-read to spot cases were the machine didn't have enough context to resolve an ambiguity the right way.)
You might like to try out GPT-3 (seems access is widely available now) for translation some time, it's remarkably good, better than myself (well my German is pretty rusty) and I think much better than Google Translate...
(Basically just write a prompt like "translate the following to German/Chinese/Whatever:")
See https://academic.oup.com/mind/article/LIX/236/433/986238 for the original paper.
However, I do count responding well to softball questions as important markers of our progress. Ten years ago, no one was able to pass even this mark.
(And from a commercial point of view, good enough 'understanding' and responses might be good enough for many tasks.
For example, I am a native German speaker, but I found that Google Translate often already does a better job of translating English to German than I could do, unless I spent unreasonable amounts of time on the task.
At the moment the most efficient way for me to produce a translation is to let Google Translate do the work, and then proof-read to spot cases were the machine didn't have enough context to resolve an ambiguity the right way.)