How sure are we that knowledge in humans is fundamentally different from a language model? How do you generally represent knowledge? Graphs? Patterns with typed place holders? Could those structures be embedded into a language model?
>How sure are we that knowledge in humans is fundamentally different from a language model?
Because humans learn about language in real world contexts, accompanied by multiple sensory streams. LLMs learn about it solely by being exposed to text. Imagine a child kept in a closet who learns about the world solely by reading books, with no indication which are fact and which are fiction, and no exposure to the physical world outside. LLMs cannot reason, only regurgitate probabilistically.
What does grok semantics mean, how would you test this?
I am not trying to argue that language model are close to human level reasoning or whatever, but it is not obvious to me that a language model of some sort is fundamentally unable to to achieve this.
The question of whether language models can "reason" ultimately comes back to the chinese room argument. There aren't any universally accepted resolutions to that argument either, nor whether those limitations (if they exist) are fundamental.
However, the current incarnation of LLMs like ChatGPT fail at straightforward reasoning around things like categorizing sailfish and mathematical proofs. I'm incredibly impressed at how powerful the simplistic methods they use are, but the deficiencies make it clear that it's not abstractly reasoning from definitions in the same way humans are able to.
A fun thing to do: Ask ChatGPT a question, and ask for references. Then try to track down the references. I just asked a simple question about the Thirty Years War, and with the answer it gave me two valid references (including C.V. Wedgewood, yay), and one that was completely made up.
As far as I understand, LLMs have no model of reality to base results on; they're just an enormously complex statistical model of "what is the next word in the series".
> is this all just a game of syntax that _feels_ like semantics from the inside?
The point is that in case of LLMs, there is no mystery here: it is syntactic templates (x) statistics. But do we know that 'syntax' is all there is to our mental reasoning? We don't know this (yet). I have flirted with the notion - heretical for me since I 'believe' in primacy of meaning, not form - and can kinda squint and see meaning possibly being a 'virtual' phenomena and possibly simply a 'narrative' of 'arcs' connecting 'facts' mapped to a syntactic template. That meaning is simply a process. [So, not even "42" ..]
Could be. Does not even have to feel like anything, this could be a independent additional component. This goes into the direction of the Chinese room, how do you figure out if someone understands something? I would say you ask questions and judge by the answers.
