Neuron connections

[Anonymous No.4713577]

So the brain has 100 billion neurons, and the most complex artificial neural net (http://en.wikipedia.org/wiki/Artificial_neural_network)) to date will have 1 billion (http://www.artificialbrains.com/spinnaker).).

It's easy to say that the artificial neural network (ANN) has 1% of the complexity of the brain, but it's not that simple. What defines the complexity of a system is not merely the number of units in the system, but the number of meaningful interactions that can happen between the units.

For instance, if each unit in a system can interact with each other:
3 units = 3 possible interactions
5 units = 10 possible interactions
10 units = 45 possible interactions
100 units = 4950 possible interactions
1,000,000 units = 499,999,950,000 possible interactions

It's clear to see that for a linear increase of unit number, the increase in interaction number is nonlinear, and thus far more important at large numbers.
10 is 10% of 100, but 45 is only 0.9%, and the significance decreases as size of the system (number of units) increase.

I want to apply this logic to brains and ANNs. Biological neurons have between 5,000 and 200,000 inputs from other neurons. Assuming that the ANN cited above functions on a similar number of inputs, I want to:
1) quantify the complexity of the brain in terms of interactions
2) do the same for an equivalent system of only 1 billion units
3) work out difference in complexity between the two systems

Unfortunately, my math skills end at the previous example =(
I am hopeful that someone here can help me or point me in the right direction!

[Anonymous No.4713592]

>>4713577
Doesn't the number of connections between any one neuron and its neighboring neurons pretty much sum up the complexity of the system?

[Anonymous No.4713609]

'pretty much', yes.

In fact I can roughly answer my own question by saying that the most advanced ANN to date will be (much?) less than 1% as complex as a human brain.

Still, I'm too curious for my own good, and I want to see if I can get some numbers.

[Anonymous No.4713614]

>>4713609
Well, obviously the number of neurons doesn't completely quantify the complexity of a neural network, but the number of connections between neurons across the entire system would, wouldn't it? How many neural connections are there in the human brain and how many neural connections are in the ANN?

[Anonymous No.4713630]

Let's formulate like this:
If the brain has 100 billion neurons, and each neuron is connected to 5000 other neurons, how many neural connections exist in the brain?

[Anonymous No.4713644]

>>4713630
Exactly. There are hundreds of trillions of neural connections in the brain.

I wonder if having the same number of connections but a different number of neurons would affect the end result. Would an ANN with more neurons but fewer connections be just as capable as the human brain? What about fewer neurons but more connections per neuron?

[Anonymous No.4713654]

>3 units = 3 possible interactions
>5 units = 10 possible interactions
>10 units = 45 possible interactions
>100 units = 4950 possible interactions
>1,000,000 units = 499,999,950,000 possible interactions

What's the math behind this? I can't make sense of it.

[Anonymous No.4713666]

keep in mind, you only use a model of neurons/synapses that will be "simulated" on silicon.
its not a 100%-imitation of the real ones, so they behave differently and the more complex or lasting your simulation is, the more it will diverge from its role model.
you cannot treat artificial and real neurons as even.

[Anonymous No.4713670]

>>4713654
Number of possible links between neurons. There are 3 unique connections that can be made between 3 neurons, six connections for four neurons, ten for five...

I can't come up with a formula.

[Anonymous No.4713675]

>>4713644
Right, but do you know how the hundreds of trillions value is calculated? Answering this could answer other questions, such as the ones you ask.

>>4713654
Where N is the number of units, the number of possible interactions is defined by (N(N-1))/2

[Anonymous No.4713681]

>>4713670
That explanation is a good example, but too simple for neurons. In the example, each unit can link to each other unit. Neurons are limited to between 5000 and 200000 connections each.

[Anonymous No.4713707]

>>4713670
(continued)
Got it.

I came up with the <span class="math">\sum_{j=1}^{n}n-j[/spoiler]. With the help of WolframAlpha, that became <span class="math">\frac{1}{2}\left ( n+1 \right )[/spoiler].

[Anonymous No.4713713]

I like this thread and I hope more people will contribute.

[Anonymous No.4713716]

>>4713675
>Right, but do you know how the hundreds of trillions value is calculated? Answering this could answer other questions, such as the ones you ask.
Yes, it's just a simple matter of multiplying the number of neurons by the average number of connections per neuron.

[Anonymous No.4713721]

>>4713681
This is further influenced by different kind of neuron signaling. Spike trains to left and right and inhibitory and excitatory and different neurotransmitters and whatever else adds a fair share of complexity to an already awfully complex system.

Henry Markrams solution of not doing any abstraction shortcuts but only adding MOAR FLOPS is probably the most sensible intial approach, once we have a high resolution model we can then start to abstract and see which abstractions break the model and which works.

[Anonymous No.4713742]

I've got an unrelated question.

The human body generally replaces ever single within it every 11 years. The constituent parts that makes up an individual are gone, but it is meaningful to still refer to that individual as if e was the same person. This is true of every living thing. Every individual living organism cycles through matter but it is still meaningful to refer to an individual organism as if it were the same organism it was when it was made out of different parts. Governmental bodies cycle through generations of employees, politicians, and buildings but still retain their structure.

Might there be a name for something that retains its structure or form despite a replacement of its constituent parts? I feel like there should be.

[Anonymous No.4713747]

>>4713742
Molecules move in and out, but not all cells die and are replaced. In particular lungs and brains do not regenerate.

[Anonymous No.4713751]

I've got an unrelated question.

The human body generally replaces ever single atom within it every 11 years. The constituent parts that makes up an individual are gone, but it is meaningful to still refer to that individual as if e was the same person. This is true of every living thing. Every individual living organism cycles through matter but it is still meaningful to refer to an individual organism as if it were the same organism it was when it was made out of different parts. Governmental bodies cycle through generations of employees, politicians, and buildings but still retain their structure.

Might there be a name for something that retains its structure or form despite a replacement of its constituent parts? I feel like there should be.

[Anonymous No.4713848]

>>4713747
Those cells cycle through atoms like every other part of the body, and yet they are the same cells and the body is the same body. There should be a word for structures that retain their form despite replacing their constituent parts.

[Anonymous No.4713954]

>>4713848
This kind of individuality is a functional or extrinsic notion of individuality.
http://plato.stanford.edu/entries/intrinsic-extrinsic/
http://plato.stanford.edu/entries/identity/
http://plato.stanford.edu/entries/identity-indiscernible/

[Anonymous No.4714001]

>>4713848
The problem is that people think identity is transitive over time. That is, if a=b now, then a=b in the past; or, if a=b now, then a=b in the future.

It is not perplexing that a theoretical perfect clone and the original would be indistinguishable at a past point while being distinguishable going forward (say because they end up having different memories by having had different experiences).

When we abstract away time we can have "pure" transitivity. For instance, if a particle like an electron has no "memory" then all electrons will necessarily be indistinguishable.

If this is true, and we are only composed of elementary particles, then whence memory?

[Anonymous No.4714017]

>>4713751
Organic seems like a fitting description.

[Anonymous No.4714088]

>>4714001
Information is a matter of structure.

[Anonymous No.4714196]

There are also issues to take into consideration about graded signals vs absolute signals.

[Anonymous No.4714203]

>>4714088
Then reductionist science (that is, physics) will never be able to explain consciousness.

[Anonymous No.4714255]

>>4714203
Only if reductionist science can never explain a car's movement.

[Anonymous No.4714660]

>>4714017
"Organic" means "containing carbon".

[Anonymous No.4714668]

>>4714255
>>4714203
>>4714196
>>4714088
>>4714017
>>4714001
>>4713954
Are you two high?

[Anonymous No.4714675]

>>4714001
To sum up how memory works:
Repetitive/traumatic experience is recorded in our brains by the formation of neural networks to process the input.

After the aforementioned imprinting, the system is maintained by the activation of genes which produce prions (proteins that never shut off) which "repair" the networks.

The genes can go silent over time, or be silenced (amnesia) but depending on the amount of time that passes, the memories may still be recovered if they have not yet been "pruned".

Look up Eric Kandel if you want to learn more, he won a Nobel for his work on memory

[Anonymous No.4714680]

>>4714675
>the system is maintained by the activation of genes which produce prions

>prions

Oh look, it's this shit again: someone clueless presenting ignorance as fact.

Just. No.

[Anonymous No.4714716]

>>4714680
http://en.wikipedia.org/wiki/Prion#PrP_and_long-term_memory

lrn2winmoarplx

[Anonymous No.4714722]

By the way, the actual resource I learned about the relationship between prions and memory is In Search of Memory by Eric Kandel winner of the Nobel Prize.

Learn to educate yourself before you try to educate others.

[Anonymous No.4714749]

>>4714675

You're just saying things that sound right to someone not versed in the field, but which are absolutely wrong. Gene silencing does not result in amnesia. Terms like "pruning" and "imprinting" are used in entirely the wrong context here. While prion proteins have been implicated in long-term memory, you're explaining it in entirely the wrong way.

There's also no proof that prions "repair" neural networks. It's all very hypothetical.

[Anonymous No.4714761]

>>4714722
I'm a medical student, your explanation is extremely butchered. (proteins that don't shut off? seriously what the fuck are you trying to say?)

Prions are misfolded proteins and potential infective agents, use PrP or prion protein if you want to differentiate and not be misunderstood by everyone.

Just because you read a single source doesn't make you an instant expert, if you're not familiar with terminology or mechanics of the rest of the field you'll always come off as sounding ignorant or odd. It doesn't matter if it's biology/medicine or physics/engineering.

[Anonymous No.4714784]

>>4714761
Point taken. I suppose I don't get /sci/ enough credit, my statement was casual at best, I was just pointing out that there is a solid theory of memory and I provided further resources, it isn't my duty to educate 4chan; however I will watch my wording in the future.

[Anonymous No.4714797]

>>4714784
>>4714761

why /sci/ is the best board