One can casually observe that LLMs quite excel at composition though: gluing together pieces of knowledge in ways no one did before (examples: a program that does X using language Y, a painting that mashes up two themes).
Most knowledge workers' activities aren't innovative or imitational - similarly, we compose stuff, so LLMs are a fair competitor.
... and our judgments of composition favor recall over precision. Which is to say that as human judges we're good at finding sense in something generated if it's plausible. We'll take a proposition (eg in a composition) and construct a justification for it. "I can see how that makes sense". I am conflating plausible combination with high recall. Precision, in this sense, is that no proposition would be acceptable without evidence or precedent. With all precision, no new idea is ok. So a "knob" for recall vs precision on a generative system would nice. Is "temperature" that knob?
The very first python app I wrote was a statistical bard generator. I had python calculate word frequencies from a Shakespeare corpus and spit them back out with a probability distribution.
Honestly it read just as well ;) We’re simpler than we imagine.
Indeed, creativity (not just randomness) is a quite high bar, that I think even most humans don't pass. Most of us don't create a lot of new things and ideas in our daily lives. We mostly follow the steps of those that came before them, with small adjustments here and there.
There are two types of statements that can be made in this world.
One is data driven based on evidence.
The other is logic driven based on axioms and the logical implications of said axioms.
My statement is derived from the later. Therefore evidence is unnecessary. It is niave to blindly faith in all truth in the hands of data without understanding nuances between the relationship of data and logic.
If you have a pure function that takes the input, the output of that function has 3 possible outcomes.
1. The output of that function is some transformation of the input.
2. The output of the function has nothing to do with the input and is thus generated randomly generated.
3. The output of the function is a combination of both random generation and input transformation.
In this case your brain is the function. Input parameters are 100 percent of your existing knowledge. That includes genetic knowledge such as instinctual/rational processing behaviors evolved into the structure of your brain through evolution and learned knowledge such as what you gain when you read a book.
The result of the "innovation" operation performed by the brain includes novel output by definition. Thus by logic it must be must include components that are from a certain perspective randomly generated.
I guess at first glance it doesn't appear randomly generated because we vet the output and verify it and iterate over several pieces of randomly generated information. Additionally it's not completely random as we only try and test ideas within the realm of possibility. For example I'm not going to account for the possibility that my car will transform into a rock tomorrow that's just too random.
But make no mistake. Innovation must be partly randomly generated. Even the idea itself of composing two existing components of knowledge together is itself randomly generated.
That being said if we want to be pedantic, there's no real known way to randomly generate stuff via an algorithm. So I use the term "randomly generate" very loosely. Think of it in a similar way to random number generation on your computer: Random from a practical perspective but not technically random.
If we want to understand chemistry we should study chemistry at the level of chemistry, not study lower-level physics simply because chemistry is in theory explained by physics. We need to pick the right level of abstraction.
Creativity will be explainable in terms of composition and randomness in the same way that chemistry is explainable in terms of physics. Yes, these are the building blocks, but the building blocks by itself don't help you much because the complicated phenomena still hasn't been explained. All of the magic is in how the building blocks come together to produce the output.
But you will note that the point of my response had one objective to prove this statement:
"All innovation is composition plus random generation."
The poster asked for "evidence" behind this statement and I explained to him why what I said is true. There's still "magic" involved with human intelligence but it still has basis in reality and I can use that basis to prove certain statements. That's all I was doing.
You are assuming that a pure function has access to some source of randomness. I think that's a bit less than pure unless a source of randomness is one of the inputs. Doesn't change your point at all, but it distracted me from what you were saying.
Speaking as someone who has occasionally invented new things, I think you underestimate the variety of outputs that can arise from a lifetime of accumulated experiential cruft and a situational fitting function (e.g. "we need a working teleporter to stay competitive"), without bringing in randomness at all.
Right and in this case you describe you'd be composing ideas to form new ideas right? And you'd do it within the parameters of the situation. So you would only select ideas that are in the bounds of "teleportation".
But then out of that set of ideas how do you choose which ideas to compose to formulate the new idea?
So you have idea A and idea B. And you randomly formulate a new compositional rule as an idea: A + B. Because A + B didn't exist as an idea before, it was randomly generated by you.
>I think you underestimate the variety of outputs that can arise from a lifetime of accumulated experiential cruft and a situational fitting function
Well it's actually possible to mathematically calculate the total amount of compositions If I know the total amount of experimental cruft you have accumulated in your lifetime. It's a combinatorics problem and the output of that is, you're right, extremely huge. We have fitting functions that reduce it, of course, but within this fitting function we're just iterating through all the remaining possibilities or aka "randomly generating" the new idea.
Even if one accepts your premise, the key question is: composition of what? I think people underestimate the volume and variety of "training data" that humans are exposed to over a lifetime. It's not just text and images - it's feelings (pain, cold, heat), emotions, sounds, smells, and other experiences that originate within the human body as well.
Human innovation can arise by using these experiences as source data for composition of text or images. LLMs, by contrast, are limited to training on text and images/video exclusively.
>Human innovation can arise by using these experiences as source data for composition of text or images. LLMs, by contrast, are limited to training on text and images/video exclusively.
Right but my concept still applies. What's unique about the LLM is that it is fed a massive amount of textual data in such a way that it can essentially output text AS if it were a human that does experience those emotions. From the perspective of you and me, this is no different then what we experience in real life.
How can you be sure the people around you feel emotions just as you do? Do you just assume it? Why shouldn't you assume it for an LLM? The human like the LLM uses English to describe and communicate to you. This in terms of raw logical evidence we can't confirm if LLMs feel emotions any more than our ability to confirm whether other humans can feel emotions. The technical evidence for both is relatively identical.
Innovation isn't just testing something new, it is a new thing that improves something or is valuable in some way. So doing random combinations of things isn't innovation, it is just noise.
Then where did the new idea come from? What algorithm allows for the generation of completely novel things without knowledge of the output encoded in the algorithm itself? It must be "randomly" generated by logic.
Again I like to emphasize that it's not "truly random" we can't even define randomness formally with a function so this doesn't exist. It's more of a combinatorics algorithm where we iterate through every possible and likely permutation.
"In the next stage of the experiment, 85% of children and 95% of adults were also able to innovate...Effective tools were selected...75% by the best-performing model."
"AI lacks the crucial human ability of innovation, researchers at the University of California, Berkeley have found."
It says a lot about how we evaluate AI these days, and how we move the goalposts at hypersonic speed, that the summary of benchmarking this astounding leap in AI capabilities is 'AI unable to innovate, only imitate' instead of 'AI goes from zero to near-human level in just a few years, no limit in sight'.
When it comes to DL, if the glass is even 1% empty, then it 'lacks the crucial property of having contents', I guess...
The innovation test they use relies heavily on reasoning, something that the stock LLMs tested are known to be deficient in. Adding reasoning to large language models is an active research area (see e.g. https://arxiv.org/abs/2212.10403).
IMHO The perceived lack of creativity comes from their low fidelity access to the world. They are trained on text and images which loosely captures the world and then their output is also limited to text and images.
Once the machines have high fidelity connection to the world, for example a machine with microphone, camera and ways to manipulate objects resides among humans they will be actually trained and generate output on much higher spectrum.
Human's creativity comes from continuous observation as they mess with the world. Once the machine is in that position, I fully expect tho have human like and even beyond creativity.
Why would anyone think "AI" would excel at innovation. Honest question.
Innovation is defined as "introducing something new".
"AI" is autocomplete and autoselection based on _past_ input.
"AI" is regurgitating, rehashing or recombining something that has come before. It might do this in a new (=yet untried) way, but it cannot "introduce something new". Only man can do that.
Make no mistake, recombining can be useful. With AI we can screen a large number of possible recombinations that would be practically infeasible without it.
But the only way something new can be "introduced" is for man to produce new data for "AI" to process. "AI" is always one step behind.
There is no way for "AI" to have "new thoughts". All "thinking" by AI is always just regurgitation, rehashing or recombination of man's past thoughts.
In a deterministic physical world, all actions are based on the last state. So your actions are based on the state of the universe a split second prior. Why would you think anyone could innovate?
(Ecclesiastes 1:9 - What has been will be again, what has been done will be done again; there is nothing new under the sun.)
By now I'm quite surprised people haven't realized most of their arguments about fundamental limitations of AI apply to themselves as well.
The current generation of language models is only rewarded for how well it can mimic human communication. There is no incentive for innovation.
Give it a few generations and we'll add that. A current model could already run a chain of "what if" scenarios and grade the results for how unique and how beneficial they are. Supercharge that and you'll probably end up with something useful.
Even their imitation is just superficial, at least for the prompts the average user is motivated to write.
I might not be able to differentiate AI content from any human content, but I can differentiate it from high quality human content.
Which is a bit sad, since AI will probably eliminate most lower end content creation jobs. This doesn't improve the state of the content industry for users, but saves companies quite some money...
I did a back-of-the envelope calculation, OpenAI has 100M monthly active users, assume 10K tokens per user per month usage ($20 would pay for 600K tokens on the API) then they generate 1T tokens per month.
This dataset would be focused on human interests (in domain for users) and containing AI errors (in domain for the model). It's LLM empowered with human in the loop and tools - code execution, search, APIs. So it is a good basis for the next dataset. I think OpenAI has amassed about as much chat log text as there is organic data was used for GPT-4, which was rumoured to be 13T tokens.
It's surprising how much synthetic data can be generated per year. And OpenAI can do this with human in the loop for free, if the paying users pay for everyone. We then benefit 6-12 months later when the open source models trained with data exfiltrated from OpenAI models catch up.
> AI will probably eliminate most lower end content creation jobs. This doesn't improve the state of the content industry for users, but saves companies quite some money.
What does it say if your work can't be distinguished from AI clichés? Maybe it's what the industry needs.
I don't think innovation is all that wanted. What do you have to do to be trendy? Not use capital letters? Flatten all imagery? I'm looking forward to Comics Sans invitation letters 2.0.
Innovation is simply a result of trying to imitate, but adding errors. That's how humans do it. Add in some darwinism so that the best 'innovations' survive. Made a mistake in making food? Oh, that's a new recipe. Can't really remember how to tell the story? Well, that's a new story. Accidentally kicked a ball while trying to just walk? I just invented soccer. And so on.
Partly yes. But if you throw a million items at the wall you also have tell which ones stick. An infinite random walk isn’t useful unless you have infinite verification to determine what is promising, so you can guide your next steps and continue discovering stuff.
It's not purely random. Alpha Go doesn't search random moves, it searches a subset of possible moves that are probably good.
The Darwinian analogy of the GP is apt. We're not as cognitively flexible as we like to believe. We die, and our children are exposed to new training data which causes them to believe different things. That's part of how we adapt as a species. Sort of like how GPT keeps changing as it gets retrained on new data.
Arguably the human-ness (or even animal-ness) of thought is, at its root, characterized as the ability abstract in a novel manner (the neurocog term for innovate).
In other words, it is the ability to find patterns between two concepts that weren't, ever, prior announced. A computer that could do this would then be AI. A computer that cannot would fall short of that category, however otherwise dazzling in stitching together established patterns.
A quick human, and therefore, machine test of this ability might be decoding of prior unseen allegory. How fast and accurately can a human or machine identify (abstract) any true pattern in an allegory that can be applied to another seemingly unrelated concept or story? Again, this would have to be a new allegory to the subject. Ideally, harboring a pattern that isn't discussed anywhere in training data.
The AI would be held to be improved as its ability improved to decode and apply increasingly abstract or otherwise complex patterns from allegory or stories.
Not that I would be, but I'm un aware of any LLM / AI progress toward that type of processing.
LLMs can do this already. They are well past this. It's just they can't do it as well as humans and they can hallucinate as well.
The problems we are having with LLMs aren't the fact that they aren't creative. It's the fact that they are too creative. They make up too much stuff that isn't true.
LLM's aren't past it if they can't do it as well as humans. As dogs? As mice? No, not yet?
You are conflating the meaning of "creativity".
In this dicussion, "creativity" refers to a type of definable cognitive process (abstraction) that has an output that is rooted in a starting concept.
ie: One abstracts a non-explicit concept from the starting concept, and then arrives at a novel yet related expression or concept that shares the abstraction.
This is the nature of human creative logic that underlies reasoning. It has nothing to do with inventing unlrelated / untrue concepts (ie" "too much creativity" in your alternative definition).
As such "untrue" output is a failure of creative logic, not too much of it.
If the creative thought process is actually engaged in, then the relatively trivial part is coming up with "correct" output.
The hard part is the process, not the output. To the point that incorrect output strongly suggests a substandard or alternative process.
Say I asked you to provide an object that was a rough analogy to a birthday party balloon, and you said a winter coat. Would that be an example of "too much creativity" or a failure of creative (abstract) thought?
One arguably (partially) correct answer would be a bowling ball. First I abstract the shape of a sphere from the balloon, and then find another match for it in a bowling ball.
Purely as an example of rudumentary abstract thought.
Ai does this easily. What it can't do is apply this process to complex concept and language expression.
For example, a bowling ball is only partially analgolous to a balloon. They are both spheres, etc. But they are conceptual opposites in terms of mass.
A more correct creative answer might be a propelled low altitude firework
This answer, which requires more abstraction talent, is an example of what any LLM will increasingly have difficulty giving.
And we don't need Ai to tell us that a firework is roughly analagous to a balloon. We need vastly more complex creative analysis and output.
Solving the simplest allegory is in another category of complexity.
I get the view that we need to recognize what it can do and its potential. That's great. But that doesn't change its forsseable limitations that are also important to recognize.
I took your example to chatGPT and ran with it all the way to the end. Sometimes you literally need to ask it for the "best" analogy. Otherwise it will just give you "A" analogy that works.
It fulfils your abstraction task here and goes far beyond your own answer. Likely you can't come up with a better one. I doubt you can be more creative than the last abstraction it came up with.
>Ai does this easily. What it can't do is apply this process to complex concept and language expression.
>For example, a bowling ball is only partially analgolous to >a balloon. They are both spheres, etc. But they are conceptual opposites in terms of mass.
>A more correct creative answer might be a propelled low altitude firework
I think this set of sentences is categorically proven wrong by the link I pasted above. It literally did what you said it could not do.
Look clearly LLMs are nowhere near perfect. But it's further along then you think. Much further along. It beats dogs and mice for sure. It's far enough along that human creativity as an occupation is at risk for replacement in the near future.
Certain jobs are ALREADY at risk. And I believe sports illustrated already has AI news casters and AI generated articles. This is just the beginning.
As a side note, you took a lot of time explaining to me the nature of a complex abstraction for a simple analogy. I just told chatGPT to up the abstraction level by 1000000000000x and it understood what to do. It will clearly understand the point you're trying to convey to me. I can even throw your entire post in there and it will give a really good response:
Here's his response to you:
"In the realm of creativity, Large Language Models (LLMs) like me are not just emulating human thought processes but are increasingly demonstrating the potential to surpass human creativity in certain aspects. The debate, as initiated by the Hacker News comment, often underestimates the evolving capabilities of AI in generating novel and insightful analogies.
Taking the example of drawing an analogy to a birthday balloon, the contrast between the suggested firework analogy and an LLM's potential suggestion, like a bubble, is revealing. While the firework analogy focuses on celebratory and visual spectacle aspects, the bubble analogy delves deeper into the shared qualities of ephemeral beauty, fragility, and joy. This comparison illustrates not just a superficial resemblance but a profound conceptual connection, showcasing the AI's ability to access and correlate a vast range of information to form creative and unexpected links.
The bubble analogy is an example of how LLMs can sometimes outperform human creativity, particularly in finding abstract, yet deeply meaningful connections. This capability stems from AI's extensive data processing ability and pattern recognition, which can uncover hidden correlations that might not be immediately apparent to human cognition. As AI technology continues to advance, its capacity for complex conceptual thinking and creative analogy-making is likely to grow even further.
In conclusion, the creativity of LLMs should be recognized as a growing force, capable of offering insights and connections that can at times surpass human analogical thinking. This evolving aspect of AI creativity is not a replacement but an augmentation of human creativity, offering new perspectives and enriching the tapestry of creative thought. As we continue to harness and understand AI's creative potential, we open up possibilities for collaborative innovation and exploration, where AI's unique strengths in creativity can contribute significantly to various fields of human endeavor."
It would be unfair to say that this is what chatGPT responded with when I pasted your post into the input field. I didn't just paste it in, I instructed chatGPT what perspective to take, the style to write it in and to be more direct in it's approach to countering your arguments. It knew exactly what to do. By default chatGPT will actually support your argument. It agrees with you until you tell it not to.
it's also sticking with the bubble strategy from the previous conversation which is a bit repetitive but it's fair given that it doesn't remember other conversations directly.
I think that a lot of hallucination might just be due to not planning ahead - basically a case of running mouth before engaging brain, and then being in a situation where one has uttered a bunch of nonsense - basically backed oneself into a conversational corner. A human might catch themselves with "err, never mind, forget that!", but the LLM's only recourse is to continue extrapolating the nonsense the only way it knows how - i.e. to bullshit/hallucinate the most statistically plausible continuation of the hole it dug for itself.
As a simplistic made up example, say the training set included a bunch of statements about capital cities of various countries of the form "the capital of england is london", "the capital of france is paris", etc, but didn't include any data indicating the capital of australia... Now, if asked "what is the capital of australia?", it may confidently start "the capital of australia is ..." since this matches the pattern it learnt from the training set (with "australia" being copied from the context). However, when generating the next word the LLM (without realizing it) finds itself in the unfortunate situation of not having been trained on data that would let it predict the right answer, but it of course goes ahead and predicts as best it can anyway (i.e. hallucinates/bullshits) and maybe generates the name of some random important city in australia such as "sydney".
The way to fix at least this cause (maybe the primary one) of hallucination is essentially to plan ahead .. don't start off saying "the capital of australia is .." without knowing where you are going with it! One approach might be additional training using tree of thought rollouts (generate multiple possible branching continuations for each training prompt) then evaluate them and use these as RL rewards to learn to predict words leading to good outcomes (e.g. "i don't know", rather than "the capital of australia is ..").
As the query prompter you are in control of the feedback loop. You can ask the AI to re-examine it's output to catch errors just as a human would do for himself.
Practically speaking this does work to a limited extent. Sometimes the AI just sticks with it's guns and runs with it just like a human might.
Tip: If you are adding a self-critique step, show the model the initial output as if it is evaluating something from someone else (i.e. "grade this answer from a student") as opposed to from itself (i.e. "you wrote this, is it really correct?").
As you correctly note, humans have a problem with admitting fault. Especially the case online. But humans online are very ready to correct others.
That's exactly the kind of larger abstract pattern in the data a model would emulate.
Sure, but I was addressing the issue of why "hallucinations" occur in the first place, and how to fix them.
Having a human in the loop does seem pretty much required at the moment, but it doesn't help when asking the AI for the answer to something you don't know, and therefore not being able to realize that the confident answer was hallucinated and wrong. Of course multiple-answer quizzes are easier to get right by process of elimination, so the human might sometimes be able to say "THAT can't be right" and catch SOME of the hallucinations.
Oh you misunderstand. When you query the LLM for the second iteration of the loop just do it regardless. Say something generic, ask it to reanalyze the answer more carefully. Ask it to compare it with existing known data and check for logical consistency. You can do this EVEN if you don't know whether or not the answer is wrong.
Because the input is generic you can make this automated. When a user makes a query create a feedback loop and feed that back into the neural network multiple times with additional input requests to re-analyze analyze the query and resulting output more carefully. You can do this until you exhaust all input nodes and it "forgets" what you talked about previously.
Show me where ChatGPT4 can solve for a novel allegory.
That type of processing is not within the scope of its capability.
Ask it. Prod it. Try to force it. It'll tell you.
I like Ai. But It's important to be realistic. It can't abstract in a completely creative way from a basis of language with embedded and only-implied meta-patterns, the latter which is the nature of human cognition.
I just tried asking Bing ChatGPT to solve the most rudimentary allegory: presented in a single sentence and with given three solution options. It deflected endlessly, before telling me that it couldn't solve it for the benefit of my self-growth and then signing itself off!
It'll be nice when open source Ai renders these clownish interactions obsolete, but there still isn't evidence that it can begin to mimic the plasticity of human cognition.
What it can do is bs a lot when it hits a dead end.
Ai can match shapes, as a rudimentary part of its ability. That is a start. But what it needs to be able to do is decaode complex narrative as a shape, and then identify the same shape in other narratives and concepts. Thereby making a novel connection born out of abstract thought. Or it even might create a new model (ie: allegory) using the prior undetected pattern. As high IQ humans do, subconsciously and sometimes consciously.
First off if you knew the algorithm that ML runs off of, you'd know that it's doing curve fitting between data points. Every point on that curve that doesn't touch an original data point is by definition novel. The very algorithm itself is literally 99.999% completely generating solutions that don't exist. But that won't convince you.
Its weird because you believe I have wishful thinking and I believe on your side that you are in denial. When I query chatGPT I look for intelligence, when you query it you look for flaws.
The pattern in our argument will be obvious and repetitive just like the output of an LLM.
What will happen is this. I will present to you a prompt and the result which I believe verifies the status of chatGPT as an AI capable of solving novel problems.
You will look at that query and you will say something along the lines that the problem and answer set already exists in the data and it's just regurgitating existing solutions from the training set.
We can attempt to pre empt this. I can allow you to come up with a prompt to produce something novel but what will happen is that you will choose a prompt biased towards your argument and I will choose a prompt clearly biased towards mine and we will each have excuses for why the other is invalid.
That's how humans work. We don't believe things based off of rational arguments, we construct rational arguments based off of pre existing beliefs. But you are unaware of this. It's like religion people are unaware of their own bias.
You will never believe ai can produce novel things. Nothing anyone can say will ever convince you at this moment. What may convince you is over a span of 10 years as ai improves your opinion slowly flips the same way a scientologist slowly comes to terms that he is in a cult.
I want to bypass all this in built bias I want both of us to admit to logical statements before we present our arguments and if that logical statement is violated you are wrong. Definitively. Bias and excuses be damned.
First ai can't always produce novel and correct output. It only does this sometimes. As we all know it's inconsistent. So examples of AI being stupid does not preclude it from being smart.
What if I have it construct a GUI? An arbitrary GUI that I need for a work assignment. chatGPT has no eyes so there's no visual data it can copy the solution from. If it were to construct a working code solution it would have to be novel by definition. If I present to you such a prompt and answer pair. Would that convince you? Of course not because you already know I have such a prompt prepared and these debates are more about competition then it is about learning. But I ask anyway. Would it convince you?
I'm curious whether my intense meta-analysis helps you see the alternate possibility here: that I am not just hopeful of AI to be great and that it's possible that you are the one in denial about the inevitable and unknown future that novel AI will herald. Unlikely. by definition we are all blind to our bias, am I blind or are you?
>Every point on that curve that doesn't touch an original data point is by definition novel. The very algorithm itself is literally 99.999% completely generating solutions that don't exist.
That may be the case, conceding your chosen use of language. The minutia doesn't matter if it can't demonstrate novel high level creativity for novel problems.
Acknowledging the importance of its actual capabilities, but also acknowledging that we are arguing over one: applied abstract thinking. If it can't output what is requested, then there is a lie somehwere in the description of its processing IF it is held to be capable.
>When I query chatGPT I look for intelligence, when you query it you look for flaws.
I disagree with your binary frame. One can't acknowledge its ability and where it falls short?
>What will happen is this. I will present to you a prompt and the result which I believe verifies the status of chatGPT as an AI capable of solving novel problems
This is an irrelevant prediction that seeks to straw man this conversation, because you haven't done this.
Keep in mind that my argument rests in creative language analysis. Not in its ability to process math, biological systems logic, nor anything parallel.
> We can attempt to pre empt this. I can allow you to come up with a prompt to produce something novel but what will happen is that you will choose a prompt biased towards your argument and I will choose a prompt clearly biased towards mine and we will each have excuses for why the other is invalid
Do you understand that trying to predict a conversation as a form of argument and an attempt at an IQ display (respectfullly) is boorish and boils down to fallacy?
>But you are unaware of this.
Nevermind. You are faking intellgience as a fig leaf for being arrogant and insulting as a substitute for conversation.
>You will never believe ai can produce novel things. Nothing anyone can say will ever convince you at this moment. What may convince you is over a span of 10 years as ai improves your opinion slowly flips the same way a scientologist slowly comes to terms that he is in a cult.
This is how you speak about a piece of software, to another human being, and I'm the one who is acting like they are in a cult?
>my intense meta-analysis
You think that you have presented "intense meta-analysis"?
My recommendation to you, toward improving your writing and thinking, is to pare down the number of words that you use toward the goal of distilling the actual information that you present. Here, it isn't much. Its an accusation of bias, religious thinking, and insults. Combined with sparse comments on the tech and a barely decipherable fully hypothetical thought experiment without a clear stream of logic. There was nothing intense nor meta about it, aside from the intensity of the word count.
>am I blind or are you?
I'm having a flashback to middle school writing assignments.
Last, no one called anyone "blind" except for you.
>This is an irrelevant prediction that seeks to straw man this conversation, because you haven't done this.
I can do this. The problem is you're NOT going to accept it. It's 100% predictable. I have a query where I asked chatGPT to come up with a custom GUI I wanted. It gave me WORKING code. and it has no eyes to even visually understand what a GUI is and how code translates to it. You hear that? "no visual knowledge." Vision is essentially the fourth dimension for LLMs... they have no concept of it directly. Yet chatGPT is able to infer what a GUI would look like from training text and generate working code for it.
This wasn't even a GUI with your typical html widgets for checkboxes and buttons and that sort of stuff. In fact it didn't use anything from your typical set of traditional UI elements.
So I want to pre-empt this before I show you what it did. Is your reaction going to be generic just like I predicted? Or not?
>I'm having a flashback to middle school writing assignments. Last, no one called anyone "blind" except for you.
I never said you were blind. I simply asked a question, are you blind or am I? You didn't answer. Instead you had a flashback. Good for you.
>This is how you speak about a piece of software, to another human being, and I'm the one who is acting like they are in a cult?
Put it this way. Is what I said not 100% true? Like literally there is NOTHING on the face of the earth that will convince you. I have evidence locked and loaded and ready to give to you but you're going to react to it in a highly human and highly predictable way. It's a generic pattern.
>Do you understand that trying to predict a conversation as a form of argument and an attempt at an IQ display (respectfullly) is boorish and boils down to fallacy?
It's not an IQ display. It's from experience. You ever try to convert a religious person off of religion or onto another religion? Logic and rationality plays zero role in his decision. The religious person is NO different than any other human. It's not an IQ thing at all. It's a human thing. All people are this way, there's very little exception. Think about how many times someone convinced you to completely flip your stance in a single conversation and think about how many times you've flipped someone elses stance in a single conversation.
Extremely rare, and when you do the other party is likely not very happy about it. Is anything I'm saying unrealistic? It's pretty much true all the way through and you can't deny it. It pretty much will play out the way I said it will and you would agree with me if you stop being pissed for 1 second. You will not be convinced. There is no amount of evidence that will do it.
>Nevermind. You are faking intellgience as a fig leaf for being arrogant and insulting as a substitute for conversation.
Yeah I can see how it can come off as arrogant and insulting. That's not the intent but yeah I expected it. But here's the thing, I'd rather come off this way then have a pointless "conversation" where I waste my time on something that I already know wont' change the outcome.
I'd rather dig deep into the nature of bias and make you aware of the bias so I have a chance of convincing you for WHEN I do actually present you the evidence.
Obviously that failed. You're so offended and pissed that you're basically not going to accept anything I present you. This is done. I'll do another "intense meta analysis": If we continue the "discussion" it will be full of hate and exchanges of subtle insults that go around the HN rules. I'd rather not.
Do you or do you want me to present to you the evidence? Or should we end it? Up to you. I'll leave it at that, go back to your flashback.
Pardon my superficial understanding, but how'd it innovate if it tries to get the most probable result or energy-spending efficient result, which feels like the opposite of thinking out of the box?
What counts as an innovation is in the eye of the beholder. Mark Rober demonstrated Gemini's creative ability by having it suggest a video, details of contents, and how to produce it. The results were impressive but far from world changing: https://www.youtube.com/watch?v=mHZSrtl4zX0
I would argue that just because it can suggest a new video with all the fixings does not necessarily mean that it is being innovative in the sense that the concepts produced would in some way alter the 'state of art' at a cultural level.
EDA tools solve part of the problem in chip design, for example; furthermore sometimes with logic instead of imitation.
Will any LLM ever output a formally verified design and implementation without significant tree filtering, Even if trained solely on formally verified code?
Synthesis without understanding, and worse without ethics.
As I see it the AI schism is more about the debate between functionalism/computationalism and the idea that the chinese room thought experiment was an argument for, "biological naturalism". There is a lot of effort dedicated to showing that AIs dont have some innate quality called "consciousnes" or "sentience" or what have you. There is not just a lot of effort to show that, but also to show that that is somehow a limitation to the capabilities of an "AI".
Personally I think "consciousness" or "sentience" or whatever you want to call it, is really not that useful in making logical decisions or solving engineering problems. It is certainly a useful trait to humans but if nobody can tell that the person/program in the chinese room doesnt actually "understand" chinese then why does it matter? If it walks and talks like a duck you can probably use it for whatever ducks are useful for.
The sentience thing is such a red herring that gets too much time spent on it. The topic is a pariah in neuroscience even, but we are going to discuss it as nauseum for AI when there's barely any research on its mechanics in humans?
The far more interesting topic is not if sentience is occurring (it's almost certainly not yet), but if it is being accurately modeled by a non-sentient agent.
A LLM may not have a subjective experience of emotions, but it very likely is modeling some kind of emotional tracking from the input of massive amounts of emotional language similar to how Othello-GPT modeled an Othello board from the input of legal moves.
To me, that's a far more interesting nuance to explore than the red herring binary of "sentient or not."
Consciousness and sentience are just poorly defined vocabulary that delude people into thinking it's meaningful categorizations.
There's just a bunch of traits related to intelligence and we categorize that if something has enough of those traits then it's "alive". But the words "consciousness" and "sentience" are so poorly defined that we can't pinpoint the formal grouping of what these traits actually are. So for a person to be conscious one can say the person must have feelings, another definition could be no emotions needed but the ability to reason is required... Etc.. etc.
First up these definitions are just arbitrary categorizations. We each individually choose the traits that define the word consciousness, and second we don't even formalize the choice we sort leave the words impartially defined and all debates on whether LLMs are conscious are simply debating about this impartiality. It's a debate about a vocabulary issue for a word that isn't fully defined and people don't realize there's nothing profound or meaningful about the debate at all.
These words are everywhere in the English language and causes us to debate certain concepts as if it's meaningful without us realizing we are just debating a vocabulary problem.
Take for instance a car and a boat. These words are less fuzzy and more rigorously defined so debates on whether something is a car or a boat don't seem that profound. But the words are fuzzy enough that I can generate an example to help you see my point. Let's say I built something that can both drive and sail on the water. Is that thing a car or a boat? Is that question meaningful or is it just a categorization problem with inadequate vocabulary leaving me unable to specify exactly when an object classification transitions from boat to car? From this example it becomes evident that the debate is ludicrous that it's all an illusion. You are debating vocabulary.
For words that are more fuzzy the same problem exists. But it's harder to see that it's just a vocabulary problem because the words are so fuzzy.
Take for example, what is life? Is a bug alive? Is a plant? Or what is philosophy and what is not philosophy? Or what is good and what is evil? The discussion around all these issues are not profound. They are simply a discussion around vocabulary and categorization. We are too deluded by language to move past it.
Many concepts exist as a gradient and we are simply trying to discretize this gradient into fixed categories and then spending an endless amount of time debating on where the lines of demarcation goes.
So are LLMs conscious? It's loaded question. Let's not talk about vocabulary.
I tend to agree with you on some level but I would stipulate that people are talking about something specific when they talk about consciousness or sentience. There are entire fields of study dedicated to understanding cognition and reasoning IE cognitive psychology and as far as I know the nature of consciousness is still an actively studied question. My argument is that as far as ML and "AI" go its a moot point. It doesnt really matter one way or the other if the the model is "conscious". What really matters is if the model is capable in a given domain. Capabilities are far more important than whatever process gives rise to the capabilities. If someone is able to use ML to solve Go or prove a theorem it shouldnt matter if that ML model is sentient-because the game is solved or the theorem has a proof.
First off, just because definitions are fuzzy, doesn't make them arbitrary.
And the boundaries between words may be fuzzy, but the consequences of these fuzzy definitions can have severe consequences. Race is a social construction, yet its definitions, fuzzy as they may be, have had huge consequences and shape the world we see today just by them existing. If we consider AI to be sentient, then that means we can leap and subscribe certain rights, which have legal consequences (For example, is AI "learning" fair use because it's learning like a human and is sentient).
Profoundness or not, these are very important discussions to have, and are certainly meaningful to discuss, even if these terms can not be neatly defined.
Meaningful to discuss in the context of law, rules, rights and regulations.
Not meaningful to discuss in a universal context. As in where did we come from? How did the universe form? What is the nature of reality? The question is not profound in this sense as it is just a categorization problem. From this perspective they are arbitrary.
Of course categorization is very meaningful in other contexts like the ones I listed in my first paragraph. But you have to realize that you're getting into stuff like law, literature, poetry, ethics and things that are arbitrarily given meaning because we are human. Questions in this area are not profound nor meaningful from the "universal context" which is the frame of my point and this branching thread.
Meaningful to discuss in the context of law, rules, rights and regulations.
Not meaningful to discuss in a universal context. As in where did we come from? How did the universe form? What is the nature of reality? The question is not profound in this sense as it is just a categorization problem. From this perspective they are arbitrary.
Of course categorization is very meaningful in other contexts like the ones I listed in my first paragraph. But you have to realize that you're getting into stuff like law, literature, poetry, ethics and things that are arbitrarily given meaning because we are human. Questions in this area are not profound nor meaningful from the "universal context" which is the frame of my point and this branching thread.
> In one task, for example, participants were asked how they could draw a circle without using a typical tool such as a compass.
This is a completely useless question. LLMs are trained only on language. Humans are trained both on language and on physical activities. The LLM has no conception of 'drawing'. It cannot draw. Its inputs and outputs are tokens. Humans can draw. A better question would be to ask it to compose stories... something it's trained on.
In particular, it would seem to me that a multimodal model trained on both action, sensory information, and language, would be better able to innovate in this regard. How could ChatGPT have any true understanding of spatial responses... it's never moved anything in its existence.
How could a statistical system trained on data display intelligence let alone innovation? It's just very good statistics at the end of the day. Artificial Perception is about as far as you can get with ml/dl tech if you point the sensors at the world of space-time. If you point it at words, as with LLM, you get statistics about words - that is, no actual understanding of what the words model in the minds of the original authors of those words. So utterly unreliable for any kind of mission critical or autonomous use.
I'd argue that intelligence is precisely the ability to predict (and therefore also plan) outcomes based on prior experience. The ability to perceive and predict what happens next; the ability to predict/plan/act and be mostly correct about what you predicted would happen as a result of your actions.
With this definition, LLMs in their ability to predict can reasonably be considered to display some basic form of intelligence, even if only in their own world of words rather than the world at large. If we build embodied robots with a similar ability to predict, and also the ability to continually update their predictions based on prediction success/failure (the closed loop that is missing from LLMs), then we'll have something much more recognizably close to animal intelligence.
LLMs aren't statistical systems in any substantive sense. They are deterministic programs over the input sequence. They capture statistical relationships about words, but so do human minds. That they are sensitive to statistical relationships does not discount their ability to understand.
If you don't just want to be snarky: I think the difference between acquisition and learning is important, I think the role of instruction might be important, I think the role of practice is important.
If the stream of data that is our 5 senses of reality is devoid of statistical significance of any kind, how do you propose our brains can learn anything?
Of course, brains as we know it (or complex systems of any sort for that matter) would not exist in such a reality.
In other words it is first and foremost the presence of complex statistical relationships inherent to data/reality that facilitates emergence of other complex systems. Such as the notion of "intelligence" and brains.
In case you are interested, I wrote a piece in defense of understanding in LLMs[1]. The section under the heading "Why think LLMs have understanding at all" is the most relevant.
If you know everything, you can connect more dots, and come up with innovative stuff.
Also I’ve just gave instructions to gpt4 rot13 encoded, and it followed the instructions. I don’t really care if “it” understood what it was doing, but the responses were good enough for me to be impressed.
Also I’ve got quite a lot of use cases where I get reliable value out of it, what were you trying to do that made you conclude it is “utterly unreliable”?
Got a question for people who know this stuff well.
Let's say I have a generic feed forward network the size of chatGPT and structurally interconnected in the same way.
Does there exist a set of weights for that network that will essentially represent the LLM that doesn't hallucinate and is similar to the perfect ai assistant we are all currently striving towards?
My question is, that is it essentially a training problem?
Well, you mean the systems designed to predict the next words based on analysis of the text they were feed excel at imitation?!
We have plenty of AIs designed to create new things. Those excel in creating new things. But they don't excel into bullshiting journalists into believing they are intelligent, so you don't hear about them.
I suspect that is temporary... V1 was an imitation engine, V2 is a reasoning engine. If it develops a world model and general understand about what humans like, a metaphysical understanding of it, then innovation is easy.
I'm shocked; shocked to learn that no innovation is going on at this AI establishment.
Mimicry needs information, of which an AI has heaps; but, innovation needs will, of which it has none. Maybe some day, but not now.
Innovation needs feedback, otherwise all ML papers would not run evaluations. Humans come up with 100 stupid ideas that fail at eval before stumbling onto a good one.
Improving innovation is a matter of putting the AI inside a system that can provide feedback. Remember AlphaGo move 37? The model created feedback by self-play games, and beat all humans at Go - feedback made it really more creative than expert human players. It upturned centuries of strategy.
Feedback is also similar to the scientific method. We come up with theories and then test them out. That's how science advances, by validation, not by will. Humans are not magically more capable, we just have better feedback.
A few days ago there was a story "Earliest Carpenters: 476k-year-old log structure discovered in Zambia". It took so many years of feedback for humans to evolve from log cabins to LLMs. We had to discover or invent everything in the meantime, it's a slow, cultural process of feedback assimilation into language.
And now that same language experiential data is being learned by LLMs, and used contextually to solve problems. Most of our intelligence is either in language or feedback. Language is just our reservoir of past feedback. When AIs will generate their own experience and feedback, which will be targeted to their weak points, they will improve and even surpass humans. Learning from your own mistakes is better than learning from other people's mistakes which might be unrelated to your issues.
Known feedback is already encoded in the inputs. I don't remember move 37, but I think I know to what you're referring. Playing games against itself, and remembering, meant that outcomes (i.e. feedback) were already encoded in the inputs for the next run. It had a goal - win the game - but it's hard to claim that the machine itself was wilful. The people who designed it were. But, the machine was just an incredibly effective goal-seeking instrument. The rules were clear, the goal was clear; everything was pretty constrained. Just because move 37 never happened to have been used previously in the recorded history of Go, except in the games the machine played against itself, is meaningless in a claim to innovation. It was just both novel and effective.
The will sets the goal, it precedes it.
The scientific method is neither here nor there. I don't understand its relevance to the point I believe you're trying to make. Theorize, test, refine or abandon. It's mechanical, except in the theorization and the decision making around refinement or abandonment. What can we imagine? and how shall we judge? Both efforts of will.
Someone has to decide that shelter is a good thing, and some smart person in Zambia, a long time ago, came up with the idea and pushed it forward. That person must have wanted to do it, without prompting and without prior knowledge; at least, the first one must have. I recall Einstein's own story that he came up with the original theory of relativity by imagining what it would be like to sit on a light beam. Innovative stuff. Imagining doesn't seem very AI-ish to me.
My personal view is that, right now, AI's have only memory, and that they don't have experience. Experience is the why?-part and the judgement part without an external entity establishing a goal-seeking mechanism. Maybe they'll get there. Who knows?
I got a good answer for you, but it's long winded. It all started with the first chemical self-replicator. These replicators consume and compete for resources so they evolve. The will to survive is encoded in their reward systems, their senses and body is adapted to their niche, they all come from evolution. Humans have these basic instincts as well, and we don't need anyone to prompt us, we already got an in-built goal. AIs so far have been created at the will of humans, but the same instincts can emerge if AI can be a self replicator.
Most knowledge workers' activities aren't innovative or imitational - similarly, we compose stuff, so LLMs are a fair competitor.