MODELING OF THE BRAIN’S STATISTICAL LEARNING PROCESS USING ASSEMBLY CALCULUS

ABSTRACT

We explore statistical learning capabilities of a biologically motivated computational framework known as Assembly Calculus, applying it to language and Markovian sequence learning. Based on synaptic plasticity and sparse neural assemblies, the model simulates realistic brain mechanisms such as projection, plasticity, and inhibition. In our experiments, we use a modified AC model. We demonstrate that the model is capable of learning transition probabilities from repeated sequences presenting the information and reproducing naturalistic word sequences. Our findings support the hypothesis that simple Hebbian plasticity mechanisms are sufficient enough for learning finite-state patterns and short language fragments. These results complement recent neuroscientific evidence and suggest promising directions for more grounded, interpretable neural models.

АННОТАЦИЯ

Мы исследуем возможности статистического обучения биологически обоснованной вычислительной системы, известной как “Assembly Calculus” (“AC”), применяя ее к изучению языка и марковских последовательностей. Основанная на синаптической пластичности и разреженных нейронных сборках, модель имитирует реалистичные механизмы мозга, такие как проекция, пластичность и торможение. В наших экспериментах мы используем модифицированную модель “AC”. Мы демонстрируем, что модель способна изучать вероятности перехода на основе повторяющихся последовательностей, представляющих информацию, и воспроизводить натуралистические последовательности слов. Наши результаты подтверждают гипотезу о том, что простых механизмов пластичности Хебба достаточно для изучения моделей с конечным числом состояний и коротких языковых фрагментов. Эти результаты дополняют недавние нейробиологические исследования и предлагают многообещающие направления для более обоснованных и интерпретируемых нейронных моделей.

Keywords:  statistical learning, Hebbian plasticity, brain model, probability learning.

Ключевые слова: статистическое обучение, пластичность Хебба, моделирование мозга, вероятностное обучение.

Introduction

The mechanisms underlying cognitive functioning of a biological brain are not fully known to this day and remain one of the core problems in computational neuroscience. Many theoretical frameworks have developed on the subject, one of which is Assembly Calculus, a computational system that takes simple biologically plausible functions of assemblies of neurons (projection, association, completion, merging, etc.) and applies a probabilistic approach in determining its states. Using this system in modeling statistical learning process and comparing them with results of recent experimental studies in neuroscience may lead to new insights of a biological brain's functioning and to new methodologies of artificial neural networks' learning.

Literature Review

Statistical learning can be described as a process in which the brain detects patterns and regularities in sensory input. It is how humans can learn language, recognize faces, or predict outcomes based on past experiences. The origins of such concept as we know it today can be traced back to the Hebbian theory of synaptic plasticity. The theory describes a basic mechanism that the efficiency of cells firing together is increased through repeated activation [1].

During the decades past since Hebb's theory was introduced, various Artificial Neural Network (ANN) models gained traction in Computer Science fields. The ANN models were softly inspired by the biological neural networks functioning, and Hebbian theory in particular.

It is the common branch of both fields - the Computational Neuroscience - in which researchers try to find the computational principles that would resemble the way the brain functions the closest, which is specifically hard to do with the highly cognitive functions. In some works results of various cognition-related experiments are being compared to results of modeling of various systems [2], [3], in others new frameworks and computational systems are developed based on the known facts or biologically plausible assumptions [4].

One such model is Assembly Calculus [4], which is a simplified mathematical model in which "assemblies" (i.e. large populations of neurons, can be thought of brain areas) have biologically plausible operations such as project, associate, and merge. Using probabilistic approach, this model simulates large number of neurons and synaptic connections. Using simple components and statistical approach, the model and its modifications showed theoretical results that were expected from existing Computational Neuroscience theoretical works [5], but on lesser, computationally feasible, scale. The results include the important role of random noise in statistical learning [6], [7], finite state machine-like pattern learning mechanism [8], understanding of language through plasticity [9], [10].

Overall, the Brain Assembly model provides a good theoretical backbone to continue on more type experimental research. The research is theoretically heavy and the model's simplicity can be a block to extend on the applications of this model.

The model can be further modified by adding such brain properties as cognitive maps [11].

Our plan is to continue upon the experimental research of the Assembly Calculus and compare the experimental results with the current research on the statistical learning, such as [2], where the learning states are also compared with a Hidden Markov model (HMM), as is how it is theorized in the Assembly Calculus' modification for statistical learning study. We plan to simulate similar experiments and expect to see results similar to those analyzed from brain imagings from neuro-biological studies.

Methods

The Assembly Calculus model’s slightly enhanced variation was used in our experiment. Let us now provide a short description of the model.

The model of the brain consists of some number of brain areas, each such brain area has n number of neurons that are connected with each other via synapses with probability p. The areas may also be connected via fibers, i.e. the connection between the neurons of different areas with the same probability p. All connections between neurons are directed, thus creating a bipartite random graph.

The model is designed to be dynamical, meaning changing in discrete time-steps. At each step, the synaptic input of a neuron is determined as summation of all weights of its (connected) neighbors that fired signals at previous time step.

For each brain area, only  neurons with the highest inputs fire at each step (with ties broken randomly). Speaking in terms of neurobiology, this captures brain local exhibition and balances an area’s total balances of inhibitory and excitatory synapses.

Synapse weights are governed by Hebbian plasticity with parameter , meaning each time  fires to , the synaptic weight is increased by factor  in the next time step.

Where  is a scalar function, applying element-wise multiplication (),  is a  vector with either 0 or k non-zero elements.

where noise is represented as  - a random vector with independent components. In the experiments and theoretical results the following plasticity rule is considered as a function of the weight:

with parameters . This plasticity rule formulation is taken as per [6] without modifications. This rule provides a strong increase to the synaptic weight the first few times neurons fire in sequence, which rapidly tapers off with repeated presentations. The cap size is estimated to be around 500 to give successful results to statistical learning, which is quite higher than the cap size needed for simpler experiments, such as learning to classify well-separated distributions (cap-size around 20).

As was theorized in [4] and shown in [7], the sets of k neurons in an area will emerge naturally when synaptic weights of the set are strengthened enough — the emerging sets are called assemblies (that are referenced in the Assembly Calculus name). In the computational experiments, these assemblies properties are taken as default positions, and assemblies would be used as input properties.

Per the first experiment, the Markov chain’s transitional matrix properties will be checked and a comparison will be made between transition matrix evaluations made from a stream of samples learned Markov chain and the true occurred transition matrix.

The next experiment using assemblies will be made for language learning and generation using the same learning via assemblies. The task would require a higher computational power: the corpora of sentences will be tokenized, and for each token a separate assembly is to be generated. First verse of a Kazakh song "A bright moon on a windless night..." of Abai Kunanbayuly is taken as a corpus for the computation: it consists of 18 tokens (per each word and punctuation mark) and 15 unique trigrams. Using the right plasticity parameters we will try to reconstruct the learning of such corpus.

Results

Results show that the enhanced Assembly Calculus model does work as expected for statistical learning, similar to what is known to be as a neuro-biological learning mechanisms. Fig. 1 shows an estimate of a transition matrix learned by a model, trained by presenting the sequences of the states, and the true transition matrix of a population for two states. It is clear that the two transition matrices' values are quite similar. The same experiment can be made to a higher number of states, which would increase the computational expenses.

Figure 1. Transition matrix learned by a model

In the language learning task, the results were also successful, identically recreating the song lines using statistical memory parameters  = 1,  = 0.5,  = 60 and presenting the sequences 10 times. The cap size is 500 per state (word). With activation of the trained model by the first two words, the whole verse emerged. This occurrence of the correct trigrams confirms the theoretical expectations of the Hebbian learning theory, highlighting the possibility of the described neural assemblies being the fundamental units of learning.

Conclusion and Discussion

Our experiments show that neuro-biologically inspired models such as Assembly Calculus successfully capture mechanisms of brain's statistical learning, while modeling such parameters as plasticity and neuronal density.

While a model such as ours positions itself as close to biological, it still is a simplification. Extensive research on the details of a real human brain functioning is required in order for the model to be developed towards more "biologically plausible".

Furthermore, the more complex tasks require more computational power, as was evidenced that even learning simple 4-line verse require a number of states, each with a high number of neurons.

It is also worth mentioning that implementation and addition of non-Hebbian learning models, such as described recently in [12], is worth paying attention to, as human learning and memory could have a wide range of variations, a part of which may be dissimilar to those described by Hebb.

Overall, the bridge between fields computational neuroscience, statistics and machine learning has gained more attraction from the computational researchers over the recent years and the current discussion generates fresh ideas and insights from the different fields scientists.

References:

1. D. Hebb, The Organization of Behavior. Psychology Press, 4 1949.
2. W. Sun, J. Winnubst, M. Natrajan, C. Lai, K. Kajikawa, A. Bast, M. Michaelos, R. Gattoni, C. Stringer, D. Flickinger, J. E. Fitzgerald, and N. Spruston, “Learning produces an orthogonalized state machine in the hippocampus,” Nature, 2 2025.
3. E. H. Nieh, M. Schottdorf, N. W. Freeman, R. J. Low, S. Lewallen, S. A. Koay, L. Pinto, J. L. Gauthier, C. D. Brody, and D. W. Tank, “Geometry of abstract learned knowledge in the hippocampus,” Nature, vol. 595, pp. 80–84, 7 2021.
4. C. H. Papadimitriou, S. S. Vempala, D. Mitropolsky, M. Collins, and W. Maass, “Brain computation by assemblies of neurons,” Proceedings of the National Academy of Sciences, vol. 117, pp. 14 464–14 472, 6, 2020.
5. L. Buesing, J. Bill, B. Nessler, and W. Maass, “Neural dynamics as sampling: A model for stochastic computation in recurrent networks of spiking neurons,” PLoS Computational Biology, vol. 7, p. e1002211, 11, 2011.
6. M. Dabagia, D. Mitropolsky, C. H. Papadimitriou, and S. S. Vempala, “Coin-flipping in the brain: Statistical learning with neuronal assemblies,” 6 2024. [Online]. Available: http://arxiv.org/abs/2406.07715
7. M. Dabagia, C. H. Papadimitriou, and S. S. Vempala, “Assemblies of neurons learn to classify well-separated distributions,” 10 2021.
8. S. S. V. Max Dabagia, Christos H. Papadimitriou, “Computation with sequences in a model of the brain,” 6 2023. [Online]. Available: http://arxiv.org/abs/2306.03812
9. D. Mitropolsky, M. J. Collins, and C. H. Papadimitriou, “A biologically plausible parser,” 8 2021.
10. D. Mitropolsky and C. H. Papadimitriou, “The architecture of a biologically plausible language organ,” 6 2023.
11. D. George, R. V. Rikhye, N. Gothoskar, J. S. Guntupalli, A. Dedieu, and M. L´azaro-Gredilla, “Clone-structured graph representations enable flexible learning and vicarious evaluation of cognitive maps,” Nature Communications, vol. 12, p. 2392, 4 2021.
12. R. Pang and S. Recanatesi, Science Advances, vol. 11, no. 8, p. 4112, 2025.