> [A]NNs do not model the brain, and have basically nothing to do with it.
ANNs don't yet model the structure of the brain but it seems plausible that they could do in the future as the result of some "convergent evolution".
ANNs have a fair model of individual neurons. Artificial and biological neurons do roughly the same thing when evaluated, but they are connected and trained very differently.
For me it's too much of a coincidence that the two most generally intelligent systems (ANNs and BNNs) are both "linear networks of activation functions".
We have not managed to build general intelligence from any other formalism, and neither has nature.
Viewing ANNs as a poor model of BNNs may be looking at the question backwards. You could say that BNNs are trying desperately hard to model the pure mathematics of ANNs within the confines of biochemistry. The fact that a biological neuron is not exactly a ReLU may say more about the limitations of biology rather than the limitations of ReLUs.
> For me it's too much of a coincidence that the two most generally intelligent systems (ANNs and BNNs) are both "linear networks of activation functions".
I am unconvinced that "linear network" appropriately defines BNNs. Can you clarify this?
> Artificial and biological neurons do roughly the same thing when evaluated
They most certainly do not. The idea that biological neurons are a kind of programmable on/off switch is completely wrong. There is plenty of computation happening inside a single neuron.
The fact that we once thought only the interconnect of neurons is important, and not the individual neurons themselves, is actually pretty strange. We know full well that every single cell is capable of computation, as it must react to its internal and external environment to be able to function. Neurons are more specialized for computation than other cells, and of course the biological neural network is a still a huge part of animal intelligence, but the simple model where we ignored what was going in inside each neuron should have been seen as extremely unlikely to be the full picture from the start.
ANNs don't yet model the structure of the brain but it seems plausible that they could do in the future as the result of some "convergent evolution".
ANNs have a fair model of individual neurons. Artificial and biological neurons do roughly the same thing when evaluated, but they are connected and trained very differently.
For me it's too much of a coincidence that the two most generally intelligent systems (ANNs and BNNs) are both "linear networks of activation functions".
We have not managed to build general intelligence from any other formalism, and neither has nature.
Viewing ANNs as a poor model of BNNs may be looking at the question backwards. You could say that BNNs are trying desperately hard to model the pure mathematics of ANNs within the confines of biochemistry. The fact that a biological neuron is not exactly a ReLU may say more about the limitations of biology rather than the limitations of ReLUs.