Journal Home Online First Current Issue Archive For Authors Journal Information 中文版

Engineering >> 2020, Volume 6, Issue 4 doi: 10.1016/j.eng.2020.02.004

Toward the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes

a National Engineering Laboratory for Video Technology, School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China
b Peng Cheng Laboratory, Shenzhen 518055, China
c Center for Systems Neuroscience, Department of Neuroscience, Psychology and Behaviour, University of Leicester, Leicester LE1 7RH, UK

Received: 2018-09-28 Revised: 2019-04-05 Accepted: 2019-04-22 Available online: 2020-02-21

Next Previous


A neuroprosthesis is a type of precision medical device that is intended to manipulate the neuronal signals of the brain in a closed-loop fashion, while simultaneously receiving stimuli from the environment and controlling some part of a human brain or body. Incoming visual information can be processed by the brain in millisecond intervals. The retina computes visual scenes and sends its output to the cortex in the form of neuronal spikes for further computation. Thus, the neuronal signal of interest for a retinal neuroprosthesis is the neuronal spike. Closed-loop computation in a neuroprosthesis includes two stages: encoding a stimulus as a neuronal signal, and decoding it back into a stimulus. In this paper, we review some of the recent progress that has been achieved in visual computation models that use spikes to analyze natural scenes that include static images and dynamic videos. We hypothesize that in order to obtain a better understanding of the computational principles in the retina, a hypercircuit view of the retina is necessary, in which the different functional network motifs that have been revealed in the cortex neuronal network are taken into consideration when interacting with the retina. The different building blocks of the retina, which include a diversity of cell types and synaptic connections—both chemical synapses and electrical synapses (gap junctions)—make the retina an ideal neuronal network for adapting the computational techniques that have been developed in artificial intelligence to model the encoding and decoding of visual scenes. An overall systems approach to visual computation with neuronal spikes is necessary in order to advance the next generation of retinal neuroprosthesis as an artificial visual system.








[ 1 ] Chapin JK, Moxon KA, Markowitz RS, Nicolelis MA. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat Neurosci 1999;2(7):664–70. link1

[ 2 ] Taylor DM, Tillery SIH, Schwartz AB. Direct cortical control of 3D neuroprosthetic devices. Science 2002;296(5574):1829–32. link1

[ 3 ] Musallam S, Corneil BD, Greger B, Scherberger H, Andersen RA. Cognitive control signals for neural prosthetics. Science 2004;305(5681):258–62. link1

[ 4 ] Moritz CT, Perlmutter SI, Fetz EE. Direct control of paralysed muscles by cortical neurons. Nature 2008;456(7222):639–42. link1

[ 5 ] Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB. Cortical control of a prosthetic arm for self-feeding. Nature 2008;453(7198):1098–101. link1

[ 6 ] Gilja V, Nuyujukian P, Chestek CA, Cunningham JP, Yu BM, Fan JM, et al. A high-performance neural prosthesis enabled by control algorithm design. Nat Neurosci 2012;15(12):1752–7. link1

[ 7 ] Hochberg LR, Bacher D, Jarosiewicz B, Masse NY, Simeral JD, Vogel J, et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 2012;485(7398):372–5. link1

[ 8 ] Collinger JL, Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, et al. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 2013;381(9866):557–64. link1

[ 9 ] Shanechi MM, Orsborn AL, Moorman HG, Gowda S, Dangi S, Carmena JM. Rapid control and feedback rates enhance neuroprosthetic control. Nat Commun 2017;6(8):13825. link1

[10] Nirenberg S, Pandarinath C. Retinal prosthetic strategy with the capacity to restore normal vision. Proc Natl Acad Sci USA 2012;109(37):15012–7. link1

[11] Seeber BU, Bruce IC. The history and future of neural modeling for cochlear implants. Network 2016;27(2–3):53–66. link1

[12] Johnson LA, Della Santina CC, Wang X. Representations of time-varying cochlear implant stimulation in auditory cortex of awake marmosets (Callithrix jacchus). J Neurosci 2017;37(29):7008–22. link1

[13] Yue L, Weiland JD, Roska B, Humayun MS. Retinal stimulation strategies to restore vision: fundamentals and systems. Prog Retin Eye Res 2016;53:21–47. link1

[14] Ha S, Khraiche ML, Akinin A, Jing Y, Damle S, Kuang Y, et al. Towards highresolution retinal prostheses with direct optical addressing and inductive telemetry. J Neural Eng 2016;13(5):056008. link1

[15] Shepherd RK, Shivdasani MN, Nayagam DA, Williams CE, Blamey PJ. Visual prostheses for the blind. Trends Biotechnol 2013;31(10):562–71. link1

[16] Ghezzi D. Retinal prostheses: progress toward the next generation implants. Front Neurosci 2015;9:290. link1

[17] Tang J, Qin N, Chong Y, Diao Y, Yiliguma, Wang Z, et al. Nanowire arrays restore vision in blind mice. Nat Commun 2018;9(1):786. link1

[18] Goetz GA, Palanker DV. Electronic approaches to restoration of sight. Rep Prog Phys 2016;79(9):096701. link1

[19] Baden T, Esposti F, Nikolaev A, Lagnado L. Spikes in retinal bipolar cells phaselock to visual stimuli with millisecond precision. Curr Biol 2011;21 (22):1859–69. link1

[20] Jadzinsky PD, Baccus SA. Transformation of visual signals by inhibitory interneurons in retinal circuits. Annu Rev Neurosci 2013;36(1):403–28. link1

[21] Euler T, Haverkamp S, Schubert T, Baden T. Retinal bipolar cells: elementary building blocks of vision. Nat Rev Neurosci 2014;15(8):507–19. link1

[22] Franke K, Berens P, Schubert T, Bethge M, Euler T, Baden T. Inhibition decorrelates visual feature representations in the inner retina. Nature 2017;542(7642):439–44. link1

[23] Helmstaedter M, Briggman KL, Turaga SC, Jain V, Seung HS, Denk W. Connectomic reconstruction of the inner plexiform layer in the mouse retina. Nature 2013;500(7461):168–74. link1

[24] Baden T, Berens P, Franke K, Román Rosón M, Bethge M, Euler T. The functional diversity of retinal ganglion cells in the mouse. Nature 2016;529 (7586):345–50. link1

[25] Bloomfield SA, Völgyi B. The diverse functional roles and regulation of neuronal gap junctions in the retina. Nat Rev Neurosci 2009;10(7):495–506. link1

[26] Grimes WN, Songco-Aguas A, Rieke F. Parallel processing of rod and cone signals: retinal function and human perception. Annu Rev Vis Sci 2018;4 (1):123–41. link1

[27] O’Brien J, Bloomfield SA. Plasticity of retinal gap junctions: roles in synaptic physiology and disease. Annu Rev Vis Sci 2018;4(1):79–100. link1

[28] Rivlin-Etzion M, Grimes WN, Rieke F. Flexible neural hardware supports dynamic computations in retina. Trends Neurosci 2018;41(4):224–37. link1

[29] Demb JB, Singer JH. Functional circuitry of the retina. Annu Rev Vis Sci 2015;1 (1):263–89. link1

[30] Briggman KL, Helmstaedter M, Denk W. Wiring specificity in the directionselectivity circuit of the retina. Nature 2011;471(7337):183–8. link1

[31] Wei W, Feller MB. Organization and development of direction-selective circuits in the retina. Trends Neurosci 2011;34(12):638–45. link1

[32] Zhang C, Kolodkin AL, Wong RO, James RE. Establishing wiring specificity in visual system circuits: from the retina to the brain. Annu Rev Neurosci 2017;40(1):395–424. link1

[33] Mauss AS, Vlasits A, Borst A, Feller M. Visual circuits for direction selectivity. Annu Rev Neurosci 2017;40(1):211–30. link1

[34] Gollisch T, Meister M. Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron 2010;65(2):150–64. link1

[35] Navlakha S, Bar-Joseph Z, Barth AL. Network design and the brain. Trends Cogn Sci 2018;22(1):64–78. link1

[36] Douglas RJ, Martin KAC. Neuronal circuits of the neocortex. Annu Rev Neurosci 2004;27(1):419–51. link1

[37] Braganza O, Beck H. The circuit motif as a conceptual tool for multilevel neuroscience. Trends Neurosci 2018;41(3):128–36. link1

[38] Werblin FS. The retinal hypercircuit: a repeating synaptic interactive motif underlying visual function. J Physiol 2011;589(15):3691–702. link1

[39] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553):436–44. link1

[40] Sinha R, Hoon M, Baudin J, Okawa H, Wong ROL, Rieke F. Cellular and circuit mechanisms shaping the perceptual properties of the primate fovea. Cell 2017;168(3):413–26. link1

[41] Yan Q, Yu Z, Chen F, Liu JK. Revealing structure components of the retina by deep learning networks. 2017. arXiv:1711.02837.

[42] Maheswaranathan N, Mclntosh LT, Kastner DB, Melander J, Brezovec L, Nayebi A, et al. Deep learning models reveal internal structure and diverse computations in the retina under natural scenes. 2018. bioRxiv 340943.

[43] Brunel N. Dynamics of networks of randomly connected excitatory and inhibitory spiking neurons. J Physiol Paris 2000;94(5–6):445–63. link1

[44] Liu JK, Buonomano DV. Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner. J Neurosci 2009;29 (42):13172–81. link1

[45] Liu JK. Learning rule of homeostatic synaptic scaling: presynaptic dependent or not. Neural Comput 2011;23(12):3145–61. link1

[46] Zampini V, Liu JK, Diana MA, Maldonado PP, Brunel N, Dieudonné S. Mechanisms and functional roles of glutamatergic synapse diversity in a cerebellar circuit. eLife 2016;5:e15872. link1

[47] Bi G, Poo M. Synaptic modification by correlated activity: Hebb’s postulate revisited. Annu Rev Neurosci 2001;24(1):139–66. link1

[48] Koller D, Friedman N. Probabilistic graphical models: principles and techniques. Cambridge: The MIT Press; 2019. link1

[49] Carandini M, Heeger DJ. Normalization as a canonical neural computation. Nat Rev Neurosci 2012;13(1):51–62. link1

[50] Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 1998;20(11):1254–9. link1

[51] Roy S, Basu A. An online unsupervised structural plasticity algorithm for spiking neural networks. IEEE Trans Neural Networks Learning Syst 2017;28 (4):900–10. link1

[52] Kappel D, Nessler B, Maass W. STDP installs in winner-take-all circuits an online approximation to hidden Markov model learning. PLOS Comput Biol 2014;10(3):e1003511. link1

[53] Yu Z, Tian Y, Huang T, Liu JK. Winner-take-all as basic probabilistic inference unit of neuronal circuits. 2018. arXiv:1808.00675.

[54] Yamins DLK, DiCarlo JJ. Using goal-driven deep learning models to understand sensory cortex. Nat Neurosci 2016;19(3):356–65. link1

[55] Marblestone AH, Wayne G, Kording KP. Toward an integration of deep learning and neuroscience. Front Comput Neurosci 2016;10:94. link1

[56] Lee TS, Mumford D. Hierarchical Bayesian inference in the visual cortex. J Opt Soc Am Opt Image Sci Vis 2003;20(7):1434–48. link1

[57] Maass W. Searching for principles of brain computation. Curr Opin Behav Sci 2016;11:81–92. link1

[58] Knill DC, Pouget A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci 2004;27(12):712–9. link1

[59] Simoncelli EP, Olshausen BA. Natural image statistics and neural representation. Annu Rev Neurosci 2001;24(1):1193–216. link1

[60] Wu MCK, David SV, Gallant JL. Complete functional characterization of sensory neurons by system identification. Annu Rev Neurosci 2006;29 (1):477–505. link1

[61] Meyer AF, Williamson RS, Linden JF, Sahani M. Models of neuronal stimulusresponse functions: elaboration, estimation, and evaluation. Front Syst Neurosci 2017;10:109. link1

[62] Chichilnisky EJ. A simple white noise analysis of neuronal light responses. Network 2001;12(2):199–213. link1

[63] Schwartz O, Pillow JW, Rust NC, Simoncelli EP. Spike-triggered neural characterization. J Vis 2006;6(4):484–507. link1

[64] Liu JK, Gollisch T. Spike-triggered covariance analysis reveals phenomenological diversity of contrast adaptation in the retina. PLOS Comput Biol 2015;11(7):e1004425. link1

[65] Sahani M, Linden JF. How linear are auditory cortical responses? Adv Neural Inf Process Syst 2003;15:109–16. link1

[66] Machens CK, Wehr MS, Zador AM. Linearity of cortical receptive fields measured with natural sounds. J Neurosci 2004;24(5):1089–100. link1

[67] Sharpee T, Rust NC, Bialek W. Analyzing neural responses to natural signals: maximally informative dimensions. Neural Comput 2004;16(2):223–50. link1

[68] Pillow JW, Shlens J, Paninski L, Sher A, Litke AM, Chichilnisky EJ, et al. Spatiotemporal correlations and visual signalling in a complete neuronal population. Nature 2008;454(7207):995–9. link1

[69] McFarland JM, Cui Y, Butts DA. Inferring nonlinear neuronal computation based on physiologically plausible inputs. PLOS Comput Biol 2013;9(7): e1003143. link1

[70] Paninski L. Convergence properties of three spike-triggered analysis techniques. Network 2003;14(3):437–64. link1

[71] Rust NC, Schwartz O, Movshon JA, Simoncelli EP. Spatiotemporal elements of macaque V1 receptive fields. Neuron 2005;46(6):945–56. link1

[72] Maheswaranathan N, Kastner DB, Baccus SA, Ganguli S. Inferring hidden structure in multilayered neural circuits. PLOS Comput Biol 2018;14(8): e1006291. link1

[73] Liu JK, Schreyer HM, Onken A, Rozenblit F, Khani MH, Krishnamoorthy V, et al. Inference of neuronal functional circuitry with spike-triggered non-negative matrix factorization. Nat Commun 2017;8(1):149. link1

[74] Jia S, Yu Z, Onken A, Tian Y, Huang T, Liu JK. Neural system identification with spike-triggered non-negative matrix factorization. 2018. arXiv:1808.03958.

[75] Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T, editors. Computer Vision—ECCV 2014. Cham: Springer; 2014. p. 818–33. link1

[76] Smith LN, Topin N. Deep convolutional neural network design patterns. 2016. arXiv:1611.00847.

[77] Hassabis D, Kumaran D, Summerfield C, Botvinick M. Neuroscience-inspired artificial intelligence. Neuron 2017;95(2):245–58. link1

[78] Lecun Y, Kavukcuoglu K, Farabet C. Convolutional networks and applications in vision. In: Proceedings of the IEEE International Symposium on Circuits and Systems; 2010 May 30–Jun 2; Paris, France. New York: IEEE; 2010. link1

[79] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 19th International Conference on Neural Information Processing Systems; 2012 Nov 12–15; Doha, Qatar; 2012.

[80] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014. arXiv:1409.1556.

[81] Kriegeskorte N. Deep neural networks: a new framework for modeling biological vision and brain information processing. Annu Rev Vis Sci 2015;1 (1):417–46. link1

[82] Yamins D, Hong H, Cadieu C, Dicarlo JJ. Hierarchical modular optimization of convolutional networks achieves representations similar to macaque IT and human ventral stream. Adv Neural Inf Process Syst 2013;26: 3093–101. link1

[83] Yamins DLK, Hong H, Cadieu CF, Solomon EA, Seibert D, DiCarlo JJ. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci USA 2014;111(23):8619–24. link1

[84] Khaligh-Razavi SM, Kriegeskorte N. Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Computational Biol 2014;10(11):e1003915. link1

[85] Yan Q, Zheng Y, Jia S, Zhang Y, Yu Z, Chen F, Revealing fine structures of the retinal receptive field by deep learning networks. 2018. arXiv:1811.0229.

[86] Batty E, Merel J, Brackbill N, Heitman A, Sher A, Litke A, et al. Multilayer recurrent network models of primate retinal ganglion cell responses. In: Proceedings of the 5th International Conference on Learning Representations; 2017 Apr 24–26; Toulon, France; 2017.

[87] Vance P, Das GP, Kerr D, Coleman S, McGinnity TM, Gollisch T, et al. Bioinspired approach to modeling retinal ganglion cells using system identification techniques. IEEE Trans Neural Networks Learning Syst 2018;29(5):1796–808. link1

[88] Vintch B, Movshon JA, Simoncelli EP. A convolutional subunit model for neuronal responses in macaque V1. J Neurosci 2015;35(44):14829–41. link1

[89] Antolik J, Hofer SB, Bedner JA, Mrsic-Flogel TD. Model constrained by visual hierarchy improves prediction of neural responses to natural scenes. PLoS Comput Biol 2016;12(6):e1004927. link1

[90] Kindel WF, Christensen ED, Zylberberg J. Using deep learning to reveal the neural code for images in primary visual cortex. 2017. arXiv:1706.06208.

[91] Cadena SA, Denfield GH, Walker EY, Gatys LA, Tolias AS, Bethge M, et al. Deep convolutional models improve predictions of macaque V1 responses to natural images. 2017. bioRxiv 201764.

[92] Klindt DA, Echer AS, Euler T, Bethge M. Neural system identification for large populations separating ‘‘what” and ‘‘where”. 2017. arXiv:1711.02653.

[93] Rowekamp RJ, Sharpee TO. Cross-orientation suppression in visual area V2. Nat Commun 2017;8:15739. link1

[94] McIntosh LT, Maheswaranathan N, Nayebi A, Ganguli S, Baccus SA. Deep learning models of the retinal response to natural scenes. Adv Neural Inf Process Syst 2016;29:1369–77. link1

[95] Thirion B, Duchesnay E, Hubbard E, Dubois J, Poline JB, Lebihan D, et al. Inverse retinotopy: inferring the visual content of images from brain activation patterns. Neuroimage 2006;33(4):1104–16. link1

[96] Naselaris T, Prenger RJ, Kay KN, Oliver M, Gallant JL. Bayesian reconstruction of natural images from human brain activity. Neuron 2009;63(6):902–15. link1

[97] Nishimoto S, Vu AT, Naselaris T, Benjamini Y, Yu B, Gallant JL. Reconstructing visual experiences from brain activity evoked by natural movies. Curr Biol 2011;21(19):1641–6. link1

[98] Wen H, Shi J, Zhang Y, Lu K, Cao J, Liu Z. Neural encoding and decoding with deep learning for dynamic natural vision. Cereb Cortex 2017;28:4136–60. link1

[99] Marre O, Botella-Soler V, Simmons KD, Mora T, Tkacˇik G, Berry MJ II. High accuracy decoding of dynamical motion from a large retinal population. PLOS Comput Biol 2015;11(7):e1004304. link1

[100] Parthasarathy N,Batty E, Falcon W, Rutten T, Rajpal M,Chichilnisky EJ, et al. Neural networks for efficient bayesian decoding of natural images from retinal neurons. 2017. bioRxiv 153759.

[101] Botella-Soler V, Deny S, Martius G, Marre O, Tkacˇik G. Nonlinear decoding of a complex movie from the mammalian retina. PLOS Comput Biol 2018;14(5): e1006057. link1

[102] Gollisch T, Meister M. Rapid neural coding in the retina with relative spike latencies. Science 2008;319(5866):1108–11. link1

[103] Stanley GB, Li FF, Dan Y. Reconstruction of natural scenes from ensemble responses in the lateral geniculate nucleus. J Neurosci 1999;19(18):8036–42. link1

[104] Garasto S, Bharath AA, Schultz SR. Visual reconstruction from 2-photon calcium imaging suggests linear readout properties of neurons in mouse primary visual cortex. 2018. bioRxiv 300392.

[105] Zhang Y, Jia S, Zheng Y, Yu Z, Tian Y, Huang T, et al. Reconstruction of natural visual scenes from neural spikes with deep neural networks. Neural Netw 2020;125:19–30. link1

[106] Onken A, Liu JK, Karunasekara PPCR, Delis I, Gollisch T, Stefano P. Data from: using matrix and tensor factorizations for the single-trial analysis of population spike trains [Internet]. Dryad, Dataset. 2017 [cited 2019 Apr 5]. Available from: https://doi:10.5061/dryad.4ch10. link1

[107] Maass W. Networks of spiking neurons: the third generation of neural network models. Neural Netw 1997;10(9):1659–71. link1

[108] Ghosh-Dastidar S, Adeli H. Spiking neural networks. Int J Neural Syst 2009;19 (04):295–308. link1

[109] Paugam-Moisy H, Bohte S. Computing with spiking neuron networks. In: Handbook of natural computing. Berlin: Springer; 2012. p. 335–76. link1

[110] Jonke Z, Habenschuss S, Maass W. Solving constraint satisfaction problems with networks of spiking neurons. Front Neurosci 2016;10:118. link1

[111] Tavanaei A, Ghodrati M, Kheradpisheh SR, Masqulier T, Maida AS. Deep learning in spiking neural networks. 2018. arXiv:1804.08150.

[112] Hu Y, Tang H, Wang Y, Pan G. Spiking deep residual network. 2018. arXiv:1805.01352.

[113] Qi, Y, Shen J, Wang Y, Tang H, Yu H, Wu Z, Jointly learning network connections and link weights in spiking neural networks. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence; 2018 Jul 13–19; Stockholm, Sweden; 2018.

[114] Xu, Q, Qi Y, Yu H, Shen J, Tang H, Pan G. CSNN: an augmented spiking based framework with perceptron-inception. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence; 2018 Jul 13–19; Stockholm, Sweden; 2018.

[115] Xiao R, Tang H, Gu P, Xu X. Spike-based encoding and learning of spectrum features for robust sound recognition. Neurocomputing 2018;313:65–73. link1

[116] Esser SK, Merolla PA, Arthur JV, Cassidy AS, Appuswamy R, Andreopoulos A, et al. Convolutional networks for fast, energy-efficient neuromorphic computing. Proc Natl Acad Sci USA 2016;113(41):11441–6. link1

[117] Davies M, Srinivasa N, Lin TH, Chinya G, Cao Y, Choday SH, et al. Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 2018;38(1):82–99. link1

[118] Maass W. Lower bounds for the computational power of networks of spiking neurons. Neural Comput 1996;8(1):1–40. link1

[119] Hodgkin AL, Huxley AF. A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 1952;117 (4):500–44. link1

[120] Meyniel F, Sigman M, Mainen ZF. Confidence as Bayesian probability: from neural origins to behavior. Neuron 2015;88(1):78–92. link1

[121] Ernst MO, Banks MS. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 2002;415(6870):429–33. link1

[122] Körding KP, Wolpert DM. Bayesian integration in sensorimotor learning. Nature 2004;427(6971):244–7. link1

[123] Pouget A, Beck JM, Ma WJ, Latham PE. Probabilistic brains: knowns and unknowns. Nat Neurosci 2013;16(9):1170–8. link1

[124] Anastasio TJ, Patton PE, Belkacem-Boussaid K. Using Bayes’ rule to model multisensory enhancement in the superior colliculus. Neural Comput 2000;12(5):1165–87. link1

[125] Rao RPN. Bayesian computation in recurrent neural circuits. Neural Comput 2004;16(1):1–38. link1

[126] Rao RP. Hierarchical Bayesian inference in networks of spiking neurons. In: Proceedings of the 17th Advances in Neural Information Processing Systems; 2004 Dec 13–18; Vancouver, BC, Canada; 2004.

[127] Gold JI, Shadlen MN. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 2002;36 (2):299–308. link1

[128] Deneve S. Bayesian spiking neurons I: inference. Neural Comput 2008;20 (1):91–117. link1

[129] Milton A. The noisy brain: stochastic dynamics as a principle of brain function. Perception 2011;40(8):1015–6. link1

[130] Fiser J, Chiu C, Weliky M. Small modulation of ongoing cortical dynamics by sensory input during natural vision. Nature 2004;431(7008):573–8. link1

[131] Ringach DL. Spontaneous and driven cortical activity: implications for computation. Curr Opin Neurobiol 2009;19(4):439–44. link1

[132] Hoyer PO, Hyvarinen A. Interpreting neural response variability as Monte Carlo sampling of the posterior. Adv Neural Inf Process Syst 2003;1:293–300. link1

[133] Fiser J, Berkes P, Orbán G, Lengyel M. Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn Sci 2010;14 (3):119–30. link1

[134] Bonawitz E, Denison S, Griffiths TL, Gopnik A. Probabilistic models, learning algorithms, and response variability: sampling in cognitive development. Trends Cogn Sci 2014;18(10):497–500. link1

[135] Denison S, Bonawitz E, Gopnik A, Griffiths TL. Rational variability in children’s causal inferences: the sampling hypothesis. Cognition 2013;126(2):285–300. link1

[136] Gopnik A, Wellman HM. Reconstructing constructivism: causal models, Bayesian learning mechanisms, and the theory. Psychol Bull 2012;138 (6):1085–108. link1

[137] Bonawitz EB, van Schijndel TJ, Friel D, Schulz L. Children balance theories and evidence in exploration, explanation, and learning. Cognit Psychol 2012;64 (4):215–34. link1

[138] Beck JM, Pouget A. Exact inferences in a neural implementation of a hidden Markov model. Neural Comput 2007;19(5):1344–61. link1

[139] Yu Z, Chen F, Dong J. Neural network implementation of inference on binary Markov random fields with probability coding. Appl Math Comput 2017;301:193–200. link1

[140] Ott T, Stoop R. The neurodynamics of belief propagation on binary Markov random fields. In: Proceedings of the 20th Annual Conference on Neural Information Processing Systems; 2006 Dec 4–7; Vancouver, BC, Canada; 2006.

[141] George D, Jeff H. Belief propagation and wiring length optimization as organizing principles for cortical microcircuits. Report. Menlo Park: Redwood Neurscience Institute; 2005.

[142] George D, Hawkins J. Towards a mathematical theory of cortical microcircuits. PLOS Comput Biol 2009;5(10):e1000532. link1

[143] Steimer A, Maass W, Douglas R. Belief propagation in networks of spiking neurons. Neural Comput 2009;21(9):2502–23. link1

[144] Litvak S, Ullman S. Cortical circuitry implementing graphical models. Neural Comput 2009;21(11):3010–56. link1

[145] Friston K. The free-energy principle: a rough guide to the brain?. Trends Cogn Sci 2009;13(7):293–301. link1

[146] Friston K. The free-energy principle: a unified brain theory? Nat Rev Neurosci 2010;11(2):127–38.

[147] Friston KJ, Parr T, de Vries B. The graphical brain: belief propagation and active inference. Netw Neurosci 2017;1(4):381–414. link1

[148] Friston K, FitzGerald T, Rigoli F, Schwartenbeck P, Pezzulo G. Active inference: a process theory. Neural Comput 2017;29(1):1–49. link1

[149] Ma WJ, Beck JM, Latham PE, Pouget A. Bayesian inference with probabilistic population codes. Nat Neurosci 2006;9(11):1432–8. link1

[150] Beck JM, Ma WJ, Kiani R, Hanks T, Churchland AK, Roitman J, et al. Probabilistic population codes for Bayesian decision making. Neuron 2008;60(6):1142–52. link1

[151] Ma WJ, Rahmati M. Towards a neural implementation of causal inference in cue combination. Multisens Res 2013;26(1–2):159–76. link1

[152] Beck JM, Latham PE, Pouget A. Marginalization in neural circuits with divisive normalization. J Neurosci 2011;31(43):15310–9. link1

[153] Yu Z, Chen F, Dong J, Dai Q. Sampling-based causal inference in cue combination and its neural implementation. Neurocomputing 2016;175:155–65. link1

[154] Yang T, Shadlen MN. Probabilistic reasoning by neurons. Nature 2007;447 (7148):1075–80. link1

[155] Gold JI, Shadlen MN. The neural basis of decision making. Annu Rev Neurosci 2007;30(1):535–74. link1

[156] Pecevski D, Buesing L, Maass W. Probabilistic inference in general graphical models through sampling in stochastic networks of spiking neurons. PLOS Comput Biol 2011;7(12):e1002294. link1

[157] Buesing L, Bill J, Nessler B, Maass W. Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons. PLOS Comput Biol 2011;7(11):e1002211. link1

[158] Habenschuss S, Jonke Z, Maass W. Stochastic computations in cortical microcircuit models. PLOS Comput Biol 2013;9(11):e1003311. link1

[159] Legenstein R, Maass W. Ensembles of spiking neurons with noise support optimal probabilistic inference in a dynamically changing environment. PLOS Comput Biol 2014;10(10):e1003859. link1

[160] Maass W. Noise as a resource for computation and learning in networks of spiking neurons. Proc IEEE 2014;102(5):860–80. link1

[161] Shi L, Griffiths TL. Neural implementation of hierarchical Bayesian inference by importance sampling. In: Proceedings of the 23rd Annual Conference on Neural Information Processing Systems; 2009 Dec 7–10; Vancouver, BC, Canada; 2009.

[162] Shi L, Griffiths TL, Feldman NH, Sanborn AN. Exemplar models as a mechanism for performing Bayesian inference. Psychon Bull Rev 2010;17 (4):443–64. link1

[163] Yu Z, Huang T, Liu JK. Implementation of Bayesian inference in distributed neural networks. In: Proceedings of the 26th Euromicro International Conference; 2018 Mar 21–23; Cambridge, UK; 2018.

[164] Coppola DM, White LE, Fitzpatrick D, Purves D. Unequal representation of cardinal and oblique contours in ferret visual cortex. Proc Natl Acad Sci USA 1998;95(5):2621–3. link1

[165] Furmanski CS, Engel SA. An oblique effect in human primary visual cortex. Nat Neurosci 2000;3(6):535–6. link1

[166] Yue Y, He L, He G, Liu JK, Du K, Tian Y, et al. A simple blind-denoising filter inspired by electrically coupled photoreceptors in the retina. 2018. arXiv:1806.05882.

[167] Zheng Y, Yu Z, Jia S, Liu JK, Huang T, Tian Y. Probabilistic inference of binary markov random fields in spiking neural networks through mean-field approximation. 2019. arXiv:1902.08411.

[168] Bishop C. Pattern recognition and machine learning. New York: Springer; 2006. link1

[169] Yu Z, Chen F, Deng F. Unification of MAP estimation and marginal inference in recurrent neural networks. IEEE Trans Neural Netw Learn Syst 2018;29 (11):5761–6. link1

[170] Yu Q, Yan R, Tang H, Tan KC, Li H. A spiking neural network system for robust sequence recognition. IEEE Trans Neural Networks Learning Syst 2015;27 (3):621–35. link1

[171] Guo S, Yu Z, Deng F, Hu X, Chen F. Hierarchical Bayesian inference and learning in spiking neural networks. IEEE Trans Cybern 2019;49(1):133–45. link1

[172] Yu Z, Guo S, Deng F, Yan Q, Huang K, Liu JK, et al. Emergent inference of hidden Markov models in spiking winner-take-all neural networks. IEEE Trans Cybern 2020;50(3):1347–54. link1

[173] Onken A, Liu JK, Karunasekara PP, Delis I, Gollisch T, Panzeri S. Using matrix and tensor factorizations for the single-trial analysis of population spike trains. PLOS Comput Biol 2016;12(11):e1005189. link1

[174] Portelli G, Barrett JM, Hilgen G, Masquelier T, Maccione A, Di Marco S, et al. Rank order coding: a retinal information decoding strategy revealed by largescale multielectrode array retinal recordings. Eneuro 2016;3(3):ENEURO- 0134-15.2016.

[175] Maccione A, Hennig MH, Gandolfo M, Muthmann O, van Coppenhagen J, Eglen SJ, et al. Following the ontogeny of retinal waves: pan-retinal recordings of population dynamics in the neonatal mouse. J Physiol 2014;592(7):1545–63. link1

[176] Hilgen G, Pirmoradian S, Pamplona D, Kornprobst P, Cessac B, Hennig MH, et al. Pan-retinal characterisation of light responses from ganglion cells in the developing mouse retina. Sci Rep 2017;7(1):42330. link1

[177] Yan B, Nirenberg S. An embedded real-time processing platform for optogenetic neuroprosthetic applications. IEEE Trans Neural Syst Rehab Eng 2018;26(1):233–43. link1

[178] Victor JD. Spike train metrics. Curr Opin Neurobiol 2005;15(5):585–92. link1

[179] Van Rossum MCW. A novel spike distance. Neural Comput 2001;13 (4):751–63. link1

[180] Shah NP, Madugula S, Chichilnisky EJ, Singer Y, Shlens J. Learning a neural response metric for retinal prosthesis. 2017. bioRxiv 226530.

[181] Li Z. Decoding methods for neural prostheses: where have we reached?. Front Syst Neurosci 2014;8:129. link1

[182] Al-Shamekh S, Goldberg JL. Retinal repair with induced pluripotent stem cells. Transl Res 2014;163(4):377–86. link1

[183] Borooah S, Phillips MJ, Bilican B, Wright AF, Wilmut I, Chandran S, et al. Using human induced pluripotent stem cells to treat retinal disease. Prog Retin Eye Res 2013;37:163–81. link1

[184] Mandai M, Watanabe A, Kurimoto Y, Hirami Y, Morinaga C, Daimon T, et al. Autologous induced stem-cell-derived retinal cells for macular degeneration. N Engl J Med 2017;376(11):1038–46. link1

[185] Yu Z, Kappel D, Legenstein R, Song S, Chen F, Maass W. CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling. 2016. arXiv:1606.00157.

[186] Rutishauser U, Slotine JJ, Douglas RJ. Solving constraint-satisfaction problems with distributed neocortical-like neuronal networks. Neural Comput 2018;30 (5):1359–93. link1

[187] Dong S, Huang T, Tian Y. Spike camera and its coding methods. In: Proceedings of the 2017 Data Compression Conference; 2017 Apr 19– 22; San Diego, CA, USA; 2017.

[188] Bi, Z., Dong S, Tian Y, Huang T. Spike coding for dynamic vision sensors. In: Proceedings of the 2018 Data Compression Conference; 2018 Mar 27–30; Snowbird, UT, USA; 2018.

[189] Lichtsteiner P, Posch C, Delbruck T. A 128 128 120 dB 15 ls latency asynchronous temporal contrast vision sensor. IEEE J Solid-State Circuits 2008;43(2):566–76. link1

[190] Li H, Liu H, Ji X, Li G, Shi L. CIFAR10-DVS: an event-stream dataset for object classification. Front Neurosci 2017;11:309. link1

[191] Orchard G, Jayawant A, Cohen GK, Thakor N. Converting static image datasets to spiking neuromorphic datasets using saccades. Front Neurosci 2015;9:437. link1

Related Research