> Humans also use neural nets to reason about concepts. We have a lot of neurons, but so does GPT-4.
I feel this is an abuse of the language. Biological neurons and ANN neurons aren’t the same or even all that similar. Brains don’t do backprop for example. Only forward passes. There’s a zoo of neurotransmitters which change the behavior of individual neurons or regions in the brain. Unused neurons in the brain can be repurposed for other things (for example if your arm is amputated).
>Biological neurons and ANN neurons aren’t the same or even all that similar.
They're not the same but they're definitely similar.
>Brains don’t do backprop for example.
We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse, so we stuck with backpropagation. We've made more complicated neurons that better resemble biological neurons, but it is faster and works better if you just add extra simple neurons, so we do that instead. Spiking neural networks have connection patterns more similar to what you see in the brain, but they learn slower and are tougher to work with than regular layered neural networks, so we use layered neural networks instead.
The only reason modern NNs aren't closer to their biological counterparts is because they genuinely suffer for it.
The secret of bird flight was wings. Not feathers. Not flapping.
> The only reason modern NNs aren't closer to their biological counterparts is because they genuinely suffer for it.
And yet, there's no ANN that's as good at interacting with the real world as the simplest worms we've studied, despite having many times more neurons than those worms have cells.
We are clearly still missing some key pieces of the puzzle for intelligence, so claiming that the difference between ANNs and biological neurons is irrelevant is quite premature. We are far away from having an airfoil moment in AI research.
Simple organisms are not necessarily easier to imitate than more complex ones. Our vehicles can drive faster than the fastest animals, but no car can absorb nutrients across a cell membrane. The simpler a system is, the more an imitation has to stick to the specifics to the system's particular implementation. Larger, more complex systems can usually perform more abstract tasks, which allows more diverse approaches.
I assert that most robotics AI, including self driving cars, are at least as good at interacting with the real world as the simplest worms we've studied. Do different things, but at least as good.
My point isn’t about whether ANNs work, it’s that they’re so fundamentally different (both locally at the level of the neuron and globally at the level of the brain) that calling them both “neurons” is pretty imprecise.
Silicon simulations of brains may “suffer” from being faithful but this also discounts the advantages that brains have. As I mentioned for example, brains can repurpose neurons for other tasks. Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples.
Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to. If I ask you to study a textbook on cats you won’t suddenly forget the difference between cats and dogs.
>Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples.
There is not a single brain on earth that is the blank slate a typical ANN is. "Brains generalize from one example" is pretty dubious. Millions of years of evolution matter.
>As I mentioned for example, brains can repurpose neurons for other tasks.
Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation ?
>Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to.
> There is not a single brain on earth that is the blank slate a typical ANN is.
Of course structure matters, but biological neurons have far more degrees of freedom than those in ANNs. The fact that we even need to keep differentiating between the two is an indication that classifying both as “neurons” is not accurate.
> Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation?
It’s a difference in capabilities of the things themselves. A biological neuron organically seeks out new connections. Sure we could program that into an ANN somehow but the fact that nodes in an ANN don’t have this capability out of the box is a fundamental difference.
> CF may well be a simple matter of scale
For a moment, a big enough network might be able to mirror an entire brain with the lottery ticket hypothesis. But if it takes two or ten or a thousand ANN neurons to simulate the degrees of freedom of a biological neuron, are they really the same?
> Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples.
Since the other comment already went for the evolution and structure angle, I'll go for the other part. What single example? What test have you seen done on the brain capacity of few weeks old fetuses? Our brains start learning patterns in the world before we are even born. How much input does a baby receive every single second from it's eyes and ears and every other sense?
Even when you are "analyzing" a new object for the first time, you receive a continuous stream of sensory input of it. Our brain even requires that to work, if you put a single frame different in a fast enough display, most times you won't even notice the extra frame and your brain will just ignore it.
I feel this is an abuse of the language. Biological neurons and ANN neurons aren’t the same or even all that similar. Brains don’t do backprop for example. Only forward passes. There’s a zoo of neurotransmitters which change the behavior of individual neurons or regions in the brain. Unused neurons in the brain can be repurposed for other things (for example if your arm is amputated).