1. Introduction
Artificial intelligence (AI) is experiencing a rapid development phase from weak to strong. Breakthroughs in deep learning algorithms, along with the expansion of data and computing power, have enabled large models to achieve or even surpass human-level performance in various fields. However, these models also exhibit notable drawbacks, such as excessive data and computing power consumption, susceptibility to catastrophic forgetting, and a deficiency in logical reasoning capabilities within black-box models. To truly realize machine intelligence, it is essential to explore new and sustainable approaches that diverge from the current deep neural network paradigms.
1.1. Large models and their inherent drawbacks
Large models, exemplified by ChatGPT, have adopted the technical route map of big data, massive computing power, and strong algorithms. This has significantly advanced fields such as language understanding, intent recognition, and content generation, reaching the pinnacle of contemporary AI. However, large models are plagued by three inherent drawbacks. First, they consume data and computing power excessively, resulting in significant energy consumption. For example, merely training GPT-3 requires 1.287 × 10
6 kW·h of electricity [
1], and ChatGPT consumes 5.6 × 10
5 kW·h of electricity daily during the problem-solving phase [
2]. Second, they are susceptible to catastrophic forgetting. When large models learn new tasks, they often quickly forget previously learned knowledge, leading to a significant decline in performance on earlier tasks [
3]. Finally, they are black-box models with a deficiency in logical reasoning capabilities. For instance, Turing Award winner Yann LeCun believes that the existing autoregressive large models can hardly perform reasoning and planning tasks.
The root causes of the aforementioned limitations can be attributed to the architecture of artificial neural networks, the training mechanisms used for large models, and the models’ data-driven reasoning mechanisms. First, the architecture of artificial neural networks lacks alignment with the underlying mechanisms of problems, resulting in poor adaptability and interpretability. Second, during the training phase, the backpropagation algorithm iteratively updates all parameters based on global errors. This process consumes a large amount of training data and computing power; it can also lead to the forgetting of earlier knowledge. Third, during the reasoning phase, the forward propagation mechanism involves all parameters in the computation, which also consumes a substantial amount of computing power.
1.2. Inspiration from the human brain’s memory mechanisms
Research in brain science and cognitive science has been a consistent driving force throughout the 70-year development of AI. Studies have shown that memory plays a fundamental role in human brain intelligence [
4], [
5], [
6]. More specifically, the impact of memory on the brain’s intellectual activities (e.g., learning, abstraction, association, and reasoning) runs through the three main stages of encoding, storage, and retrieval.
The unique memory mechanisms of the human brain provide important insights for overcoming the inherent flaws of traditional artificial neural networks. First, the brain’s memory-activation mechanism can retrieve a small amount of knowledge from the long-term memory (LTM) into the working memory for reasoning. Although the human brain contains approximately 10
11 neurons and 10
15 synapses, its energy consumption is only 20-23 W, while the energy consumption of equivalently scaled large models is five orders of magnitude higher [
7]. Second, the synaptic plasticity [
8] of the neurons in the human brain is the foundation of learning and memory formation. This plasticity is characterized by local, autonomous, and unsupervised features, which prevent the catastrophic forgetting problem observed in large models, in which all parameters are updated, resulting in new things being learned at the cost of old ones being forgotten. Lastly, the dual-system cooperation mechanism [
9] of brain memory underpins complex reasoning in humans. This mechanism holds the potential to enhance the weak logical reasoning capabilities of black-box models in large-scale AI systems.
1.3. Machine memory intelligence
Inspired by the memory mechanisms of the human brain, we propose the fundamental concept of “machine memory,” which we define as a multilayered, distributed network storage structure formed by encoding external information into a machine-representable and computable format. This structure supports dynamic updates and reorganization, spatiotemporal associations, abstract concrete associations, and fuzzy hash access. Centered on machine memory, we also propose a machine memory intelligence (M
2I) framework (
Fig. 1), which utilizes representation, learning, and reasoning modules, along with brain-inspired mechanisms, to form two interactive loops that enable the machine memory’s associative representation, continual learning, and collaborative reasoning. Using this architecture, M
2I can fundamentally break through the difficulties of brute-force computation, catastrophic forgetting, and deficient logical reasoning inherent in existing large models.
Table 1 provides a summarized comparison of M
2I and large models in the three capabilities of representation, learning, and reasoning.
M2I contains three main modules:
•A representation module. This module is responsible for the structured representation of various types of knowledge or information encoded in the learning module; it then realizes hierarchical classified organization and storage. Inspired by the associative mechanisms of the human brain and the distinct representations of various memory structures, the representation module should incorporate the key features of associative memory. These include the ability to support spatiotemporal associations, abstract-concrete associations, and fuzzy hash access, among others. Concurrently, it offers fundamental operations for heterogeneous knowledge or information post-encoding, such as abstraction, association, updating, and access.
•A learning module. This module is designed to extract new knowledge from external scenarios and datasets, encode this knowledge, and form the basic units that constitute machine memory. Emulating the dual nature of the human brain’s learning process and its memory-replay mechanisms, the learning module integrates both local and global learning strategies. This integration enables continual learning and equips the system with the capacity for autonomous reconstruction and consolidation of machine memory.
•A reasoning module. This module activates and retrieves relevant machine memories iteratively, according to the task to be solved and the current scene, and synergizes different reasoning modes to obtain the solution result. Informed by the dual-process theory and the reward mechanisms of the human brain, the reasoning module is designed to integrate both intuitive and logical reasoning modes, fostering collaborative problem-solving. Moreover, the module is capable of migrating redundant system II processes to system I, thereby facilitating intuitive reasoning and increasing the speed and energy efficiency of the reasoning process.
The interaction between the learning module and the representation module creates a learning loop. Here, the learning module collaborates with both local and global learning mechanisms to encode external scenes and datasets into the basic units of machine memory, which are then represented by the representation module as spatiotemporal associations and abstract-concrete associations of machine memory. Meanwhile, the learning module also needs to read (or retrieve) existing machine memories, collaborating in local learning, global learning, and the playback process, thereby forming a loop.
The interaction between the reasoning module and the representation module creates a reasoning loop. The reasoning module extracts key information from the tasks to be solved and external scenes for encoding, activating the machine memory within the representation module based on this information. It then utilizes the retrieved memories to facilitate intuitive reasoning and logical reasoning. The above process is executed iteratively, forming a loop.
M2I focuses on four directions: ① What neural mechanism supports machine memory? ② How can associative representation of machine memory be achieved? ③ How can continual learning be carried out under low-power conditions? ④ How can the dual-system cooperation of intuition and logic in reasoning be realized? 2 Related work on brain-memory-inspired machine models, 3 The neural mechanisms of machine memory, 3.1 Key issues regarding the neural mechanisms of machine memory, 3.2 Major progress in the neural mechanisms of machine memory, 4 Associative representations in machine memory, 4.1 Key issues related to associative representations in machine memory, 4.2 Major progress related to associative representations in machine memory, 4.3 Discussion, 4.3.1 What human brain memory research inspired associative representation in machine memory?, 4.3.2 How does the M, 5 Continual learning in machine memory systematically address the key issues and significant advancements in each of these four directions. In summary, M2I adopts an entirely new technical methodology, which holds the potential to thoroughly overcome the inherent bottlenecks encountered by current large models.
2. Related work on brain-memory-inspired machine models
In this section, we review the latest advancements in machine models that draw inspiration from the memory functions of the human brain. First of all, spiking neural networks (SNNs) represent the forefront of artificial neural networks, embracing a brain-inspired approach to information processing [
10]. Within the biological brain, neurons transmit information across synapses through electrical pulses, operating in a sparse and asynchronous fashion. SNNs emulate this functionality, with their neurons exchanging information via temporal events known as spikes and adhering to integrate-and-fire (IF) dynamics. Spike-timing-dependent plasticity (STDP) is a locally derived learning rule and has been proposed to train SNNs. As a synaptic plasticity mechanism, STDP fine-tunes synaptic weights in accordance with the precise timing of spikes between neurons. This temporal modulation allows neural networks to learn and adapt based on the sequence of input spikes, mirroring the learning and memory processes observed in biological nervous systems.
Secondly, in brain-memory-inspired models, replay is a key point of connection between AI and neuroscience. Replay in the brain has been viewed as rehearsal or as sampling from a transition model and is often used to address the continual learning issues [
11] of deep learning models. A straightforward idea is to approximate and recover old data distributions by storing a few old training samples or training a generative model. Typical sub-directions [
12] include experience replay, which saves a few old training samples in a memory buffer; generative replay, which trains a generative model to provide generated samples; and feature replay, which recovers the distribution of old features through saving prototypes, saving statistical information or training a generative model.
In addition to the replay mechanism, active forgetting of the biological brain is employed to overcome catastrophic forgetting with continual learning. For example, by modeling a robust
Drosophila learning system that actively regulates forgetting with multiple learning modules, Wang et al. [
13] proposed a generic approach that appropriately attenuates old memories in parameter distributions to improve learning plasticity and accordingly coordinates a multi-learner architecture to ensure solution compatibility. Furthermore, Kurth-Nelson et al. [
14] proposed that replay in the brain has a more general function: deriving new knowledge through compositional computation. Therefore, they suggested that replay—as a compositional computation embedded in the larger brain system including the cortex (which behaves like a deep neural network in many ways)—should be mapped to machine learning techniques that hybridize compositional computation with deep learning. Additionally, the replay mechanism can be combined with the attention mechanism, where the weights of the attention allocation network are adjusted through newly generated combinations during the replay process, thereby achieving structured reasoning.
Thirdly, the hippocampus is crucial for the formation of human memory, and recent studies have explored the relationships between the hippocampus and promising novel neural network architectures. For instance, Whittington et al. [
15] mathematically associated and compared representation methods of transformer architecture with those of the human hippocampus, providing a fresh perspective on explaining the biological plausibility of transformers. Inspired by such findings, Kim et al. [
16] proposed a new non-linear activation function that mimics
N-methyl-
D-aspartate receptor (NMDAR) dynamics. NMDAR-like nonlinearity shifts short-term working memory into long-term reference memory in transformers, thus enhancing a process that is similar to memory consolidation in the mammalian brain.
Recently, Jiang et al. [
17] suggested that attention should be paid to the self-evolution of AI models. Compared with the use of large-scale data to train models, self-evolution may only use limited data or interactions. Drawing inspiration from the columnar organization of the human cerebral cortex, the researchers hypothesized that AI models could potentially develop emergent cognitive capabilities and construct internal representational models through iterative interactions with their environment. To achieve this, Jiang et al. [
17] proposed that models must be equipped with LTM, which stores and manages processed real-world interaction data. LTM not only enables the representation of long-tail individual data in statistical models but also facilitates self-evolution by supporting diverse experiences across various environments and agents.
3. The neural mechanisms of machine memory
Unveiling and translating human learning and memory principles contribute to the development of M2I. In this section, we elaborate on the key issues and progress in neuronal and neural network pre-configuration, the neuronal and circuit plasticity of learning and memory, and the generative grammar of brain intelligence.
3.1. Key issues regarding the neural mechanisms of machine memory
•The neuronal and neural network pre-configuration of brain intelligence. It is now well-acknowledged that human intelligence originates from the reception of external stimuli and subsequent processing of these input signals for learning and memory. During human brain development, neuroectoderm cells are patterned to variable regional neural progenitors, which then differentiate into specific neuronal subtypes, constructing the basic framework for sensing and processing external signals. Fully elucidating how neuronal and neural networks are pre-configured and how the human brain is constructed step-by-step into neurobiologically and neuropsychologically responsive neuronal ensembles during embryonic development is essential in order to build functional modules for M2I.
•The neuronal and neural circuit plasticity of brain learning and memory. The competence and performance of the human brain’s neurobiological or neuropsychological function are attributed to the temporal and spatial activation of neural networks formed of neurons and synapses. Plasticity at the neuronal or neural circuit level accounts for adaptive processes regarding experiences, resulting in the generation or evolution of learning and memory. Tracing the participating neurons or neural circuits and unveiling their cellular and molecular adaptations in a specific learning and memory paradigm are the key to elucidating brain intelligence.
•The generative grammar of brain intelligence. The newly developed brain is just like a pre-configured blank slate. Immediate experience exposure sequentially activates or inhibits a cluster of neurons, which internally generate representations of the external stimuli. Repeated, similar, or related experience exposures might lead to a replay of these neuronal ensembles and therefore strengthen or amend these internally generated representations. The internally generated representation repertoires are selected upon new stimuli and used to form and express novel representations of the external world. We hypothesize that, with pre-configured neurons and neural networks, which are capable of generating intelligence, there is a generative grammar that writes, erases, and reads external experiences at either the cellular or circuit level.
3.2. Major progress in the neural mechanisms of machine memory
During embryonic development, pluripotent epiblast cells are defaulted into neuroectoderm cells, which are arranged into a single-layer neural plate. The neural plate then curves up, and its left and right edges melt at the dorsal midline to form a neural tube [
18]. It is during this neural tube formation period that, by patterning morphogens, neuroectoderm cells gradually take regional identities, such as the cortical and hippocampal neural progenitors at the dorsal region of the forebrain; and the lateral, medial and caudal ganglionic eminence and pre-optic area neural progenitors at the ventral forebrain [
19]. Neural progenitors at the dorsal forebrain largely differentiate into glutamatergic excitatory neurons, while those at the ventral forebrain mainly differentiate into γ-aminobutyric acid (GABAergic) inhibitory neurons or cholinergic neurons [
20], [
21]. Neurons can either migrate or project to local or distal brain regions, where they form pre-configured neural networks through synapses. The newly developed clusters of neurons and neural networks are pre-configured to sense and process specific external stimuli just like a pre-configured blank slate and purposely receive, record, and rework information from the external world (
Fig. 2).
Owing to the rapid development of single-cell transcriptomic analysis technologies, neuronal identities have been carefully depicted, along with their molecular features and developmental trajectories, in most brain regions of model organisms and of humans during development [
22]. Human pluripotent stem cells, including human embryonic stem cells and human induced pluripotent stem cells, also hold the potential to develop into different brain regional neural progenitors and then neurons in culture [
23], [
24]. The
in vitro generated neurons and neural networks might serve as a pre-configured blank slate for stimuli sensing and processing and therefore provide an unprecedented model for studying or mimicking intelligence generation.
During the learning process, stimuli are sensed by primary neurons, which transmit signals to downstream neurons through action-potential-mediated synaptic activation or inhibition [
25]. In another type of learning, multiple signals can converge and integrate on a downstream neuron through several input neurons [
26]. During the memory process, signals with attention are processed to internally generate representations at either the neuronal or neural circuit level [
27].
Learning and memory behavioral studies are used to characterize learning and learned states in animal models. Calcium imaging [
28], immediate early gene reporter [
29], and
in vivo electrophysiological recording [
30] are recent technologies that are capable of mapping the neurons and neural circuits involved in a behavior event. High-resolution imaging technologies are now used to investigate synaptic adaptations in learning, such as newly generated or decayed synapses and dynamic changes in the number of receptors for neural transmitters at the post-synaptic membrane [
31]. Our recent study developed genetic nuclear membrane labeling followed by a single nuclear sequencing strategy, which can be applied to capture functionally related neurons and determine the underlying molecular mechanisms of learning and memory [
32]. Projection-specific chemogenetic and optogenetic manipulations are now widely applied to reveal functional projections in the acquisition, consolidation, and retrieval of behavior, including learning and memory [
33].
The formation of brain intelligence is somewhat similar to the generation of linguistic performance, where the brain’s neurons can be viewed as letters and its sequentially activated neuronal ensembles as a vocabulary list. The most challenging aspects of brain intelligence are how the brain summarizes and extracts a specific meaning from these vocabulary repertoires, how a sentence is organized with a putative linguistic grammar, and how a strong linguistic performance is acquired to express new sentences to reflect a novel context. To address these aspects, joint efforts from neurobiologists, neuropsychologists, and scientists in machine intelligence might pave a way to decode the brain’s general generative grammar, which could then be applied to unveiling brain intelligence and constructing the next generation of machine intelligence.
4. Associative representations in machine memory
The representation of machine memory is the foundational construct that allows AI systems to encode, store, and retrieve information effectively. The key challenges and main progress in this field can be summarized as follows.
4.1. Key issues related to associative representations in machine memory
•The abstract-concrete association issue. Memory reconstruction theory posits that memory is not a mere copy of past sensory experiences but rather a reconstruction based on semantic content. Indeed, neuroscience research has discovered the existence of an “abstract-concrete” joint representation mechanism in both rodents and humans. Notably, the fan cells in the lateral entorhinal cortex of the human brain can create associations between entities, resulting in associative memories. Consequently, leveraging this representational mechanism to associatively encode various forms of sensory information, such as visual and linguistic data, along with semantic abstractions in machine perception, presents a significant challenge. This endeavor aims to improve the plasticity and flexibility of activation methods within machine memory’s associative representations.
•The spatiotemporal dynamic association issue. Neuroscience research has revealed that human memory storage is characterized by its fragmented and dynamic nature. Spatially, memories are spatially decomposed into distinct components and allocated to different brain regions. Temporally, the complexity of interactions between various brain systems leads to the dynamic updating of memory fragments over time. While dynamic associations across space and time are fundamental to the construction and consolidation of memory in the human brain, current machine learning models represent memory information as specific parametric knowledge, which lacks explicit spatiotemporal encoding and the capacity for associative mechanisms related to memory content. As a result, it is extremely challenging to replicate the human brain’s memory-reconstruction mechanisms to develop a topological structure that evolves and associates along spatial and temporal dimensions in machine memory.
•The efficient hashing access issue. The brain employs the concept of distributed storage to maintain an extensive repository of memories; through an efficient fuzzy hash access mechanism, it can swiftly pinpoint the necessary memories using specific cues or critical information. However, current machine learning models encounter challenges with low access efficiency when confronted with a plethora of memory information, struggling to rapidly retrieve and identify the memories associated with given prompts. Therefore, it is challenging to emulate the brain’s high-performance hashing access mechanism to construct a distributed hash storage structure and a mapping mechanism that is sensitive to localized machine memory.
4.2. Major progress related to associative representations in machine memory
Here, we review the progress that has been achieved in associative representations in machine memory within three areas: the abstract-concrete association of memory, the spatiotemporal association of memory, and efficient memory access.
In the realm of abstract-concrete memory association, researchers from the Institute of Cognitive Neuroscience, University College London, have combined the use of reinforcement learning algorithms and brain imaging techniques, demonstrating a mechanism of abstraction built upon the valuation of sensory features [
34]. Professor David W. Tank at the Princeton Neuroscience Institute, Princeton University, investigated the neural coding mechanisms within the hippocampus that underlie the formation of abstract knowledge [
35], revealing how spatial and non-spatial information is integrated to create a geometric framework for storing and retrieving complex concepts. A cooperative team from Fudan University and the University of York discovered the common neural circuits that assist in the retrieval of semantic memory and episodic memory and thereby enable the brain to flexibly extract long-term memories with different functions [
36]. Professor Neil Burgess from University College London proposed a generative model that captures the abstract-concrete association in memory (
Fig. 3) [
37], illustrating how memories are dynamically formed and reified from the interplay of abstract concepts and concrete instances.
Regarding spatiotemporal memory association, researchers at the University of Oxford have constructed the Tolman-Eichenbaum machine (TEM) as a theoretical model that unifies spatial and relational memory through the process of generalization within the hippocampal formation [
38]. Collaborative research between Peking University and the Chinese Academy of Sciences Brain Science and Intelligence Technology Innovation Center found that the neural activity state space during the maintenance of sequential memory is composed of mutually orthogonal sequential subspaces, revealing the encoding mechanism of brain sequential memory [
39]. Casanueva-Morato et al. [
40] developed a bio-inspired spiking content-addressable memory model based on the CA3 region of the hippocampus with the ability to learn, forget, and recall memories.
As for the mechanisms of memory access, Dasgupta et al. [
41] proposed an efficient hash-based similarity search algorithm inspired by the working mechanism of the olfactory cells of fruit flies, whose input is an odor and whose output is a tag (called a “hash”) for that odor. This work provides a new perspective on efficient access to memory in machines. Professor Tajana Simunic at the University of California, San Diego, proposed a fully binary brain-inspired classifier based on hyperdimensional computing for energy-efficient and high-accuracy classification [
42]. Luo et al. [
43] revealed the interactive mechanisms between the hippocampus and the cerebral cortex during the learning process, discovering an efficient coding method that integrates distributed storage with multi-sensory perception. Inspired by the working principles of brain neurons, researchers have introduced a nanowire network capable of instant learning and memory, marking a crucial step toward the development of highly efficient, energy-saving machine intelligence [
44].
These works not only deepen our understanding of the mechanism of memory encoding but also provide theoretical support for future research on machine memory representation and learning.
4.3. Discussion
4.3.1. What human brain memory research inspired associative representation in machine memory?
From the perspective of information theory, human memory is a process of encoding, storing, and retrieving external input information. Endel Tulving, a Canadian cognitive psychologist and a member of the National Academy of Sciences, proposed the systems of processing information (SPI) theory [
45], [
46] of memory functions. This theory posits that the memory system consists of multiple memory modules that perform specific functions, such as the perceptual input system, working memory system, language processing system, conceptual system, and LTM system. The relationships between these modules are characterized by serial information encoding, parallel information storage, and independent information retrieval (e.g., when recalling an event, we can independently retrieve relevant visual, auditory, emotional, and other types of information as needed).
Inspired by this theory, we aim to explore the associative collaborative representation mechanisms of machine memory, focusing on distributed machine memory representations that combine abstraction and generalization, dynamic spatial-temporal association, and efficient fuzzy hashing retrieval. This research endeavors to achieve distributed machine memory representations that support multi-level abstraction, spatial-temporal association, and efficient retrieval.
4.3.2. How does the M2I model achieve associative representations in machine memory?
The whole-brain mapping theory [
47], [
48] suggests that the brain utilizes the principle of distributed storage to preserve vast amounts of memories. That is, memories stored in the brain are not concentrated in a specific area but are stored in a dispersed manner throughout the neural circuits of the entire brain. Therefore, we intend to construct a distributed hash storage structure composed of multilevel storage nodes for machine memory and to design a consistent hash mapping function for distributed storage nodes, providing a foundation for the efficient and stable retrieval of memories.
Furthermore, drawing on the memory-retrieval mechanisms [
49] of the human brain, we aim to construct an efficient fuzzy hashing mechanism for machine memory, enabling the rapid association and retrieval of similar but distinct memories and ensuring the efficiency, fault tolerance, and robustness of memory retrieval. This approach offers insights into addressing the shortcomings of large models that excessively consume data and computing resources.
Inspired by the reconstructive mechanisms [
50] of human memory, we construct a distributed multi-channel machine memory semantic network and study the generative self-associative connections of abstract-concrete machine memory. On this basis, we aim to establish a machine memory storage mechanism that decouples cues from content and to construct an explicit coding and association mechanism for machine memory across time and space. This mechanism will overturn the traditional structure of machine learning models based on implicit parameterized knowledge, enabling the extraction of specific content or related knowledge. It will also help to address the weakness of traditional black-box models in terms of logical reasoning capabilities.
5. Continual learning in machine memory
Continual learning in machine memory focuses on addressing the catastrophic forgetting phenomenon. It transcends traditional continual-learning frameworks by incorporating biomimetic principles that emulate the human brain’s remarkable capacity for lifelong learning and adaptation. The architectural foundation of this methodology is predicated on achieving a tradeoff between synaptic plasticity and memory stability, thereby facilitating robust knowledge acquisition while preserving previously encoded information. The key challenges and fundamental processes in this domain can be systematically categorized and analyzed through the lens of neuroscientific principles and computational requirements, as delineated in the following.
5.1. Key issues related to continual learning in machine memory
•Adaptive regulation in machine memory control. The adaptive regulation mechanism of the human brain’s memory-learning patterns forms the foundation of memory learning. Neuroscientific research has revealed that different learning modes in human brain memory may excel at various types of scenario requirements and memory objectives. Simultaneously, the memory-learning process usually involves the collaboration of multiple learning modes. Therefore, a significant challenge in the design of machine memory control lies in how to draw inspiration from and simulate the adaptive regulation mechanism of the human brain’s memory-learning patterns. The goal is to automatically select appropriate single or combined learning modes for different scenarios, thereby enhancing the efficiency of machine memory learning.
•Collaborative associative continual learning in machine memory. Humans’ capacity for continual learning forms the basis for both reinforcing existing knowledge and assimilating new information. However, current continual learning models often suffer from drawbacks such as high computational costs and difficulties in knowledge transfer, making them inefficient in handling complex and dynamic big data scenarios. Consequently, the question of how to draw inspiration from the Hebbian learning characteristics of the human brain to establish a collaborative mechanism integrating local and global learning, as well as an associative replay learning mechanism, is a major challenge.
•Updating and reorganization of machine memory. Memory updating and reorganization enable the brain to rapidly adapt to dynamically changing scenarios. However, existing machine learning models lack effective mechanisms for updating and reorganization, making it difficult for them to swiftly adapt to complex and dynamic scenarios. Therefore, a notable challenge that needs to be addressed is how to draw inspiration from Bayesian brain theory and sleep-related neuroscience research to organically incorporate Bayesian principles and memory-replay mechanisms into methods for machine memory updating and reorganization.
5.2. Major progress related to continual learning in machine memory
Replay-based approaches represent a fundamental paradigm in neurologically inspired approaches to continual learning, serving as a critical mechanism for mitigating catastrophic forgetting and facilitating knowledge retention in neural networks. This section focuses on reviewing recent developments in neurologically inspired continual learning from the perspectives of experience replay and generative replay.
Experience replay typically involves storing a subset of old training samples in a memory buffer and strategically utilizing them during the learning process. A great deal of effort has been put toward experience-replay methodologies, with a focus on two principal dimensions: the construction of efficient memory buffers and the effective exploitation of stored information. For example, Rolnick et al. [
51] explored a straightforward solution by using experience-replay buffers for all past events and adopted a mixture of on- and off-policy learning by leveraging behavioral cloning. Rostami et al. [
52] designed a new model by learning a generative model that couples the current task to previously learned tasks through a discriminative embedding space inspired by complementary learning systems theory. Wang et al. [
53] proposed memory replay with data compression (MRDC) to reduce the storage cost of old training samples and thus increase the number of stored samples in the memory buffer. Wang et al. [
54] suggested a brain-inspired deep retrieval and imagination (DRI) framework, in which a generative model is designed to produce imaginary data and leverages knowledge distillation to retrieve past experiences. Boschini et al. [
55] described a simple and effective approach that combines rehearsal and knowledge distillation; this method allows a deep-learning-based model to revise its replay memory to welcome novel information regarding past data, paving the way for learning as-yet-unseen classes. Wang et al. [
56] developed a generic approach that appropriately attenuates old memories in parameter distributions to improve learning plasticity by modeling a robust
Drosophila learning system that actively regulates forgetting with multiple learning modules, as shown in
Fig. 4.
Generative replay usually requires training an additional generative model to replay generated data—that is, recover the old data distribution. Many methods have recently been proposed to implement the generative replay methodology. For example, Shin et al. [
57] suggested a preliminary framework that incorporates the replaying of generated data from the old generative model into the process of learning each generation task; this approach enables the inheritance of previously acquired knowledge. Wu et al. [
58] extended this approach by utilizing replay alignment to ensure the consistency of generative data sampled with the same random noise across both the old and new generative models, in a process that resembles function regularization in enforcing coherence. Rostami et al. [
59] developed a computational model that can efficiently expand its previously learned concepts to new domains using a few labeled samples.Van de Ven et al. [
60] proposed a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network’s own context-modulated feedback connections without storing data. Inspired by the organization principles of the brain’s memory system, Wang et al. [
61] applied a triple network generative adversarial network (GAN) architecture to model the interplay of the hippocampus, prefrontal cortex, and sensory cortex.
5.3. Discussion
5.3.1. What human brain memory research inspired the continual learning of machine memory?
For the continual learning of machine memory, we drew inspiration from the adaptive nature of biological neural networks, which exhibit the ability to dynamically adjust synaptic plasticity in response to changing inputs. This adaptability includes: ① preserving learned synaptic changes to combat interference, which underpins the concept of weight regularization to selectively control parameter adjustments [
62], [
63]; ② expanding or pruning functional connections to accommodate new memories, following a paradigm of expanding and then renormalizing memory space [
64], [
65]; ③ regulating synaptic plasticity based on activity and persistence, akin to meta-plasticity or the adaptability of adaptability, in line with the principles of meta-learning [
66], [
67]; and ④ utilizing inhibitory synapses to temper the activity of stimulated neurons, which resembles the role of binary masks in parameter allocation [
68], [
69]. We were also inspired by the Hebbian learning theory [
70] to design the parameter update mechanism of our machine memory model; that is, only the local parameter is updated, allowing our model to alleviate the issue of catastrophic forgetting.
5.3.2. How does the M2I model achieve the continual learning of machine memory?
Machine memory resolves catastrophic forgetting through three interconnected, brain-inspired mechanisms that work synergistically to maintain knowledge stability while enabling continual learning.
At its core, the adaptive regulation mechanism [
71] mirrors the human brain’s memory-learning patterns by dynamically selecting and coordinating multiple learning modes, automatically optimizing the learning process for different scenarios while efficiently allocating computational resources. This adaptive control is seamlessly integrated with a collaborative associative learning system that implements Hebbian learning principles [
70], combining local and global learning approaches to maintain both specific task performance and overall knowledge coherence. The local learning component focuses on task-specific updates while the global learning preserves broader knowledge structures, with an associative replay learning mechanism reinforcing critical connections and facilitating efficient knowledge transfer across different domains.
This collaborative associative approach overcomes the limitations of current continual learning models, which include high computational costs and difficulties in knowledge transfer, making it efficient for handling complex and dynamic big data scenarios.
The collaborative associative continual learning of M
2I is further enhanced by a memory updating and reorganization mechanism inspired by Bayesian brain theory [
72] and sleep-related neuroscience research [
73], which enables rapid adaptation to dynamic scenarios through probabilistic knowledge updates and selective memory replay. By incorporating Bayesian principles and memory-replay mechanisms, the system can efficiently update and reorganize stored information while preserving essential knowledge structures, significantly reducing computational overhead compared with traditional continual learning approaches. This mechanism, along with the implementation of biologically inspired learning processes and advanced resource-optimization techniques, ensures the robust preservation of existing knowledge while facilitating the incorporation of new information.
6. Collaborative reasoning by machine memory
Based on the efficient associative representations and continual learning of machine memory, the performance of interpretable reasoning based on a task or a problem to be solved and the consolidation and updating of memories are at the core of M2I. The key challenges and main progress in this field can be summarized as follows.
6.1. Key issues in collaborative reasoning by machine memory
•Multi-type machine memory activation for complex scenes and tasks. During the process of memory retrieval in the human brain, the working memory selectively attends to important information during specific scenes and tasks while actively suppressing irrelevant stimuli. Subsequently, various types of memories are activated in brain regions such as the hippocampus and prefrontal cortex based on content relevance. Therefore, the key to designing an efficient mechanism for memory activation in machines has two aspects: first, addressing the problem of information purification and filtering for scenes and tasks in order to enhance the representation of underlying relational elements across different scenes and tasks, thereby achieving the highly scalable encoding of scene and task information; and, second, tackling the problem of efficient and comprehensive activation for multiple types of memory to avoid the vast search space that results from the ambiguity and complexity of scene semantics, thereby improving retrieval efficiency while reducing resource consumption.
•Cooperative reasoning between different types of memory. Although existing large language models (LLMs) are capable of reasoning with various modalities, such as text, images, and three dimensions (3D), they still lack interpretability in the reasoning process. The dual-process collaborative reasoning mechanism of intuition and reasoning in the human brain provides promising insight for addressing the low reasoning efficiency and poor interpretability of existing LLMs. To incorporate this mechanism into machine memory models, the major challenges lie in constructing a generative intuition reasoning system, denoted as system I, and an iterative logical reasoning system, denoted as system II; designing pathway-selection mechanisms for complex reasoning tasks; and devising mechanisms to resolve conflicts between multiple types of memories.
•The design of consolidation and transfer mechanisms for different types of memory. In order to address complex and dynamic tasks in real-world scenes, existing LLMs require task-specific fine-tuning and inference adaptation, which results in high energy consumption. Inspired by the reward mechanisms of learning and memory in the human brain, constructing new associations between memories and consolidating them to achieve rapid intuitive reasoning in system I can greatly enhance reasoning speed and reduce machine inference energy consumption. The challenge lies in how to implement sparse reward modeling for complex reasoning tasks, stimulate memory consolidation, establish mechanisms for generating and transferring associations between memories, and migrate high-energy logical reasoning results to system I to achieve fast intuitive reasoning.
6.2. Major progress in collaborative reasoning by machine memory
The AI community is currently striving to develop various approaches in order to acquire the complex reasoning capability inherent in the notion of intelligence, which currently represents a major bottleneck. Although several influential experts believe that the integration route is the most promising [
74] and propose a so-called neuro-symbolic AI approach [
75], theoretical research on human cognition and decision reasoning, which aims to understand how humans evolved to acquire advanced capabilities and contributes to raising key fundamental research questions for the future development of AI, is still lacking. At present, Kahneman’s theory of thinking fast and slow [
76] (also known as dual process theory) has become a widely recognized principle and is rapidly being absorbed into the AI environment. This section briefly reviews the progress that has been achieved by related work along this line.
Before going further, we introduce a well-known, successful case to illustrate how powerful an intelligent agent with complex human-like reasoning capabilities can be. In 2016, AlphaGo shocked the world by defeating humans in the strategy game Go [
77]. Its huge success is widely believed to come from the adoption of dual process cognitive systems [
78]. To be specific, AlphaGo and its many variants employ a collaborative framework that combines fast deep neural networks with slower tree search methods [
77], [
79], [
80]. The tree search model deduces all possible action outcomes based on the current state and selects the optimal one, while a deep neural network assigns a policy value to specific board configurations based on feedback from the actions selected and played [
81]. As system I, the deep neural network acts as a pre-attention stage, building fast associations to select relevant representations for system II powered by the tree search. Although both systems can operate independently, the best performance beyond human levels is achieved only when they interact in synergy.
Recently, dual process theory has been increasingly integrated with AI, especially in the field of natural language processing. For example, CogKR [
82] adopts a representative dual system architecture, as shown in
Fig. 5, which iteratively coordinates an extension module and a reasoning module to perform multi-hop knowledge graph reasoning in the form of subgraphs instead of individual paths. Hua and Zhang [
83] built a two-system architecture that utilizes neural logic reasoning (system II) on top of representation learning models (system I). CogER [
84] is a cognition-aware knowledge graph reasoning model for explainable recommendation, in which system I generates an intuitive estimation of the next entity, and system II conducts explicit reasoning to select the most promising entity. Motivated by the unprecedented and exciting achievements of modern LLMs, there is growing interest in exploring their abilities in complex reasoning [
85]. The Statler framework [
86] equips LLMs with a persistent, memory-like representation of the world state. By maintaining the world state, LLMs can access context-related information over extended periods, greatly enhancing their reasoning capabilities. Memory-of-thought (MoT) [
87] is an MoT framework in which LLMs save high-confidence thoughts as external memory through pre-thinking and then recall relevant memory to help their reasoning. Despite some empirical success, research on collaborative reasoning in advanced AI is still in its initial stage.
6.3. Discussion
6.3.1. What human brain memory research inspired the collaborative reasoning of machine memory?
As discussed earlier, the data-driven inference approach of deep learning involves all parameters in the computation, which is highly resource intensive. Moreover, this approach primarily learns correlations from data, presenting significant limitations in content neutrality and interpretability. However, a revisitation of earlier symbolic reasoning methods reveals that they excel in these areas. Nevertheless, symbolic reasoning has limitations: ① It often struggles with handling uncertainty and ambiguity, making it difficult to adapt to new data or unseen scenarios and leading to weaker generalization abilities; and ② it is challenging to use to represent knowledge involving high-dimensional data such as text, images, and audio-video content. In fact, this is the motivation behind our approach.
According to the dual process theory [
81] in cognitive neuroscience, the human brain possesses two markedly different information-processing mechanisms. System I relies on experience and intuition and quickly engages in association and pattern matching, akin to deep learning. System II, on the other hand, operates slowly and consciously, requiring more cognitive load and attention; it is suitable for analysis, reasoning, and solving complex problems, similar to symbolic reasoning. Drawing inspiration from the dual system mechanism of the human brain and integrating the parametric knowledge (memory) of deep learning with the symbolic knowledge (memory) from early knowledge engineering holds great promise for addressing the aforementioned issues.
6.3.2. How does the M2I model achieve collaborative reasoning by machine memory?
Admittedly, a substantial body of work has focused on the design of individual reasoning modules, with typical representatives including various types of knowledge graph reasoning models [
88]. However, the great majority of existing techniques—whether embedding-based [
89], [
90], [
91], path-based [
92], [
93], [
94], or rule-based [
95], [
96], [
97]—only involve the learning and reasoning process of a single model for relatively simple tasks such as head reasoning and relation inferring. In contrast, the concept of collaborative reasoning we have elaborated includes multiple machine-memory-related mechanisms and is oriented to more complicated reasoning scenarios such as advanced algebra and complex task planning. To this end, we plan to draw on dual process theory from cognitive science and the neurobiological bases of brain memory to build a reasoning module that contains a system for fast intuitive reasoning (system I) and a system for rational logical reasoning (system II).
The reasoning module can retrieve different types of task-related machine memories, such as abstract semantics and spatiotemporal associations, which are various forms of knowledge organized and stored in a hierarchical and categorized form by the representation module, according to the external scenarios and tasks to be solved. Based on the retrieved memory, the reasoning module integrates the two reasoning patterns of the generative-based system I and iterative-based system II to work collaboratively, during which new relevant memories are extracted as supplements to promote long-chain reasoning. In the process of collaborative work, the dynamic selection of reasoning pathways ensures that the most appropriate system is selected at each reasoning stage, while the conflict-resolution mechanism aims to resolve conflicts between the multiple types of retrieved memories to maintain the rationality and correctness of the reasoning. Furthermore, the module is intended to realize the formation of associations between memories, the consolidation of memories (i.e., the enhancement of correct memories and rectification of incorrect memories), and their migration from system II to system I, enabling fast intuitive reasoning (when faced with the same problem) that significantly improves the reasoning speed and energy efficiency.
Although some efforts have been made to explore collaborative reasoning models [
82], [
98], [
99] for different tasks such as knowledge graph reasoning, multi-hop question answering, and classification under continual learning, such efforts predominantly focus on the iterative interaction between two functionally independent systems, neglecting to consider memory consolidation and its transfer mechanism between systems, which is the key to efficient reasoning in the human brain. Unlike existing studies, the collaborative reasoning we advocate is expected to fundamentally overcome the inherent defect of weak logical reasoning in black box models and achieve advanced reasoning with traceable results and interpretable processes similar to that of human beings.
This kind of human-brain-like collaborative reasoning will greatly enhance the development of advanced AI in various fields. For example, in the medical field, given a series of medical records such as a patient’s temperature sheets, laboratory test reports, medical imaging examination results, and so on, the machine-memory-based reasoning module could automatically derive professional diagnostic opinions for the patient, including a traceable and interpretable diagnostic basis and prescriptions. The experience from each diagnosis will form specific machine memories to promote the accuracy and efficiency of the subsequent reasoning. At the same time, these memories will make various associations with existing related memories, including supplementation, enhancement, and weakening. The organic integration of multiple memories is likely to generate new insights that will help humans identify unknown diseases.
7. Conclusions and future work
LLMs and multimodal models, such as ChatGPT and Sora, have attracted widespread attention from academia and industry due to their robust capabilities for understanding, generating, and interacting, as well as their potential for cross-domain applications. These models represent a pivotal direction in current AI research. However, their reliance on data-driven neural network frameworks inevitably brings inherent limitations, including excessive consumption of training data and computational power, catastrophic forgetting, weak logical reasoning abilities, and a lack of interpretability.
The memory mechanisms of the human brain play a crucial role in human intelligence. Specifically, the brain’s synaptic plasticity, multi-level integration, and associative activation mechanisms endow it with high neural plasticity, preventing catastrophic forgetting and enabling the execution of complex reasoning and planning tasks with low power consumption. Inspired by the memory mechanisms of the human brain, this perspective article introduces the interdisciplinary research direction of machine memory, which is positioned at the intersection of AI, neuroscience, and cognitive science. We identify a series of challenges in this domain, including the neurological mechanisms of machine memory, associative representation, continual learning, and collaborative reasoning. Research in the direction of machine memory aims to address the path dependency on traditional artificial neural networks for achieving machine intelligence, thereby overcoming the issues of brute-force computation, catastrophic forgetting, and weak logical reasoning capabilities inherent in current LLMs and multimodal models. Furthermore, leveraging M2I can help validate theories and hypotheses in brain science, particularly those related to human brain memory, thus promoting the deep integration, cross-validation, and collaborative development of AI and brain science.
CRediT authorship contribution statement
Qinghua Zheng: Supervision, Funding acquisition, Conceptualization. Huan Liu: Writing - review & editing, Writing - original draft, Project administration. Xiaoqing Zhang: Writing - original draft, Investigation. Caixia Yan: Writing - original draft, Investigation. Xiangyong Cao: Writing - original draft, Investigation. Tieliang Gong: Writing - original draft, Investigation. Yong-Jin Liu: Writing - review & editing, Investigation. Bin Shi: Writing - original draft, Investigation. Zhen Peng: Writing - original draft, Investigation. Xiaocen Fan: Writing - original draft, Investigation. Ying Cai: Writing - review & editing. Jun Liu: Writing - original draft, Conceptualization.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (62137002, 62250009, 62202367, 82025020, and 82230072).
PDF
(1661KB)
