ambulance bed bolt briefcase calendar chain chevron-left chevron-right clock-o commenting-o commenting comments diamond envelope-o envelope facebook feed flask globe group heart-o heart heartbeat hospital-o instagram leaf map-marker medkit phone quote-left quote-right skype star-o star tint trophy twitter user-md user youtube

LEE LAB

for Biological and Machine Intelligence Research

Publications

Featured

Relative luminance and binocular disparity preferences are correlated in macaque primary visual cortex, matching natural scene statistics

Samonds JM, Potetz BR, Lee TS | PNAS | 2012 | 10.1073/pnas.1200125109 | PDF

Humans excel at inferring information about 3D scenes from their 2D images projected on the retinas, using a wide range of depth cues. One example of such inference is the tendency for observers to perceive lighter image regions as closer. This psychophysical behavior could have an ecological basis because nearer regions tend to be lighter in natural 3D scenes. Here, we show that an analogous association exists between the relative luminance and binocular disparity preferences of neurons in macaque primary visual cortex. The joint coding of relative luminance and binocular disparity at the neuronal population level may be an integral part of the neural mechanisms for perceptual inference of depth from images.


Accounting for network effects in neuronal responses using L1 regularized point process models

Ryan Kelly, Matthew Smith, Robert Kass, Tai S.Lee | NIPS -- Advances in Neural Information Processing Systems | 2010 | PDF

Activity of a neuron, even in the early sensory areas, is not simply a function of its local receptive field or tuning properties, but depends on global context of the stimulus, as well as the neural context. This suggests the activity of the surrounding neurons and global brain states can exert considerable influence on the activity of a neuron. In this paper we implemented an L1 regularized point process model to assess the contribution of multiple factors to the firing rate of many individual units recorded simultaneously from V1 with a 96-electrode “Utah” array. We found that the spikes of surrounding neurons indeed provide strong predictions of a neuron’s response, in addition to the neuron’s receptive field transfer function. We also found that the same spikes could be accounted for with the local field potentials, a surrogate measure of global network states. This work shows that accounting for network fluctuations can improve estimates of single trial firing rate and stimulus-response transfer functions.


Scene statistics and 3D surface perception

Brian Potetz, Tai Sing Lee | In Computational Vision: From Surfaces to 3D Objects. Chapman Hall. Ed. C. W. Tyler. Chapman & Hall/CRC, chapt 1, pp. 1-25, (2010) | 2010 | PDF

The inference of depth information from single images is typically performed by devising models of image formation based on the physics of light interaction and then inverting these models to solve for depth. Once inverted, these models are highly underconstrained, requiring many assumptions such as Lambertian surface reflectance, smoothness of surfaces, uniform albedo, or lack of cast shadows. Little is known about the relative merits of these assumptions in real scenes. A statistical understanding of the joint distribution of real images and their underlying 3D structure would allow us to replace these assumptions and simplifications with probabilistic priors based on real scenes. Furthermore, statistical studies may uncover entirely new sources of information that are not obvious from physical models. Real scenes are affected by many regularities in the environment, such as the natural geometry of objects, the arrangements of objects in space, natural distributions of light, and regularities in the position of the observer. Few current computer vision algorithms for 3D shape inference make use of these trends. Despite the potential usefulness of statistical models and the growing success of statistical methods in vision, few studies have been made into the statistical relationship between images and range (depth) images. Those studies that have examined this relationship in nature have uncovered meaningful and exploitable statistical trends in real scenes which may be useful for designing new algorithms in surface inference, and also for understanding how humans perceive depth in real scenes [32, 18, 46]. In this chapter, we will highlight some results we have obtained in our study on the statistical relationships between 3D scene structures and 2D images, and discuss their implications on understanding human 3D surface perception and its underlying computational principles.


All

Relating functional connectivity in V1 neural circuits and 3D natural scenes using Boltzmann machines

Yimeng Zhang, Xiong Li, Jason M. Samonds, Tai Sing Lee | Vision Research | 2015 | 10.1016/j.visres.2015.12.002 | PDF

Bayesian theory has provided a compelling conceptualization for perceptual inference in the brain. Central to Bayesian inference is the notion of statistical priors. To understand the neural mechanisms of Bayesian inference, we need to understand the neural representation of statistical regularities in the natural environment. In this paper, we investigated empirically how statistical regularities in natural 3D scenes are represented in the functional connectivity of disparity-tuned neurons in the primary visual cortex of primates. We applied a Boltzmann machine model to learn from 3D natural scenes, and found that the units in the model exhibited cooperative and competitive interactions, forming a “disparity association field”, analogous to the contour association field. The cooperative and competitive interactions in the disparity association field are consistent with constraints of computational models for stereo matching. In addition, we simulated neurophysiological experiments on the model, and found the results to be consistent with neurophysiological data in terms of the functional connectivity measurements between disparity-tuned neurons in the macaque primary visual cortex. These findings demonstrate that there is a relationship between the functional connectivity observed in the visual cortex and the statistics of natural scenes. They also suggest that the Boltzmann machine can be a viable model for conceptualizing computations in the visual cortex and, as such, can be used to predict neural circuits in the visual cortex from natural scene statistics.


The Visual System’s Internal Model of the World

Tai Sing Lee | Proceedings of the IEEE | 2015 | 10.1109/JPROC.2015.2434601 | PDF

The Bayesian paradigm has provided a useful conceptual theory for understanding perceptual computation in the brain. While the detailed neural mechanisms of Bayesian inference are not fully understood, recent computational and neurophysiological works have illuminated the underlying computational principles and representational architecture. The fundamental insights are that the visual system is organized as a modular hierarchy to encode an internal model of the world, and that perception is realized by statistical inference based on such internal model. In this paper, we will discuss and analyze the varieties of representational schemes of these internal models and how they might be used to perform learning and inference. We will argue for a unified theoretical framework for relating the internal models to the observed neural phenomena and mechanisms in the visual cortex.


Predictive Encoding of Contextual Relationships for Perceptual Inference, Interpolation and Prediction

Mingmin Zhao, Chengxu Zhuang, Yizhou Wang, Tai Sing Lee | International Conference on Learning Representations (ICLR) 2015 | 2015 | arXiv

We propose a new neurally-inspired model that can learn to encode the global relationship context of visual events across time and space and to use the contextual information to modulate the analysis by synthesis process in a predictive coding framework. The model learns latent contextual representations by maximizing the predictability of visual events based on local and global contextual information through both top-down and bottom-up processes. In contrast to standard predictive coding models, the prediction error in this model is used to update the contextual representation but does not alter the feedforward input for the next layer, and is thus more consistent with neurophysiological observations. We establish the computational feasibility of this model by demonstrating its ability in several aspects. We show that our model can outperform state-of-art performances of gated Boltzmann machines (GBM) in estimation of contextual information. Our model can also interpolate missing events or predict future events in image sequences while simultaneously estimating contextual information. We show it achieves state-of-art performances in terms of prediction accuracy in a variety of tasks and possesses the ability to interpolate missing frames, a function that is lacking in GBM.


Relating functional connectivity in V1 neural circuits and 3D natural scenes using Boltzmann machines

Yimeng Zhang, Xiong Li, Jason M. Samonds, Ben Poole, Tai Sing Lee | Computational and Systems Neuroscience (Cosyne) 2015 | 2015 | PDF

Bayesian theory has provided a compelling conceptualization for perceptual inference in the brain. To understand the neural mechanisms of Bayesian inference, we need to understand the neural representation of statistical regularities in the natural environment. Here, we investigated empirically how the second order statistical regularities in natural 3D scenes are represented in the functional connectivity of a population of disparity-tuned neurons in the primary visual cortex of primates. We applied the Boltzmann machine to learn from 3D natural scenes and found that the functional connectivity between nodes exhibited patterns of cooperative and competitive interactions that are consistent with the observed functional connectivity between disparity-tuned neurons in the macaque primary visual cortex. The positive interactions encode statistical priors about spatial correlations in depth and implement a smoothness constraint. The negative interactions within a hypercolumn and across hypercolumns emerge automatically to reflect the uniqueness constraint found in computational models for stereopsis. These findings demonstrate that there is a relationship between the functional connectivity observed in the visual cortex and the statistics of natural scenes. This relationship suggests that the functional connectivity between disparity-tuned neurons can be considered as a disparity association field. They also suggest that the Boltzmann machine, or a Markov random field in general, can be a viable model for conceptualizing computations in the visual cortex, and as such, can be used to leverage the natural scene statistics to understand neural circuits in the visual cortex.


Late responses of V2 neurons are enhanced by global scene context of natural movies

Jason Samonds, Yuke Li, Yimeng Zhang, Tai Sing Lee | Neuroscience (SfN) 2014 | 2014 | Poster

No abstract for this publication yet.


Sample Skewness as a Statistical Measurement of Neuronal Tuning Sharpness

Jason M.Samonds, Brian R.Potetz, Tai Sing Lee | Neural Computation | 2014 | 10.1162/NECO_a_00582 | PDF


Learning Discriminative Sufficient Statistics Score Space for Classification

Xiong Li, Bin Wang, Yuncai Liu, Tai Sing Lee | Machine Learning and Knowledge Discovery in Databases, Volume 8190 of the series Lecture Notes in Computer Science pp 49-64 | 2013 | PDF

Neural dynamics of image representation in the primary visual cortex

Xiaogang Yan, Ankit Khambhati, Lei Liu, Tai Sing Lee | Journal of Physiology - Paris | 2012 | 10.1016/j.jphysparis.2012.08.006 | PDF


Neuronal interactions and their role in solving the stereo correspondence problem

Jason M.Samonds, Tai Sing Lee | In Vision in 3D Environments, Ed. Laurence Harris, Michael Jenkin, Cambridge University Press | 2011 | 10.1017/CBO9780511736261.007 | PDF

Hybrid generative-discriminative classification using posterior divergence

Xiong Li, Tai Sing Lee, Yuncai Liu | Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on | 2011 | 10.1109/CVPR.2011.5995584 | PDF

Accounting for network effects in neuronal responses using L1 regularized point process models

Ryan Kelly, Matthew Smith, Robert Kass, Tai S.Lee | NIPS -- Advances in Neural Information Processing Systems | 2010 | PDF


Scene statistics and 3D surface perception

Brian Potetz, Tai Sing Lee | In Computational Vision: From Surfaces to 3D Objects. Chapman Hall. Ed. C. W. Tyler. Chapman & Hall/CRC, chapt 1, pp. 1-25, (2010) | 2010 | PDF


The Block Diagonal Infinite Hidden Markov Model

Thomas Stepleton, Zoubin Ghahramani, Geoffrey Gordon, Tai Sing Lee | Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS-09) | 2009 | PDF

Efficient belief propagation for higher-order cliques using linear constraint nodes

Brian Potetz, Tai Sing Lee | Computer Vision and Image Understanding | 2008 | 10.1016/j.cviu.2008.05.007 | PDF

Neural encoding of scene statistics for surface and object inference

Tai Sing Lee, Tom Stepleton, Brian Potetz, Janson Samonds | In Object Categorization: perspectives from human and machine vision, Ed. Sven Dickinson, Ales Leonardis, Bernt Schiele, Michael Tarr, Cambridge University Press | 2008 | PDF

Contextual Influences in Visual Processing

Tai Sing Lee | Encyclopedia of Neuroscience, Ed. M.D. Binder, N. Hirokawa and U. Windhorst, Springer-Verlag. in Press. | 2008 | Doi


Comparison of Recordings from Microelectrode Arrays and Single Electrodes in the Visual Cortex

Ryan C.Kelly, Matthew A.Smith, Jason M.Samonds, Adam Kohn, A.B.Bonds, J.Anthony Movshon, Tai Sing Lee | The Journal of Neuroscience | 2007 | 10.1523/JNEUROSCI.4906-06.2007 | PDF

Neurophysiological Evidence of Cooperative Mechanisms for Stereo Computation

Jason M.Samonds, Brian R.Potetz, Tai Sing Lee | NIPS -- Advances in Neural Information Processing Systems 19 | 2006 | PDF

Efficient Coding of Visual Scenes by Grouping and Segmentation

Tai Sing Lee, Alan L.Yuille | in Bayesian Brain, probabilistic approaches to neural coding. Ed. K. Doya, S. Ishii, R. Rao, A. Pougeti. MIT Press, 141-185. | 2006 | PDF

Scaling Laws in Natural Scenes and the Inference of 3D Shape

Brian R. Potetz, Tai Sing Lee | NIPS -- Advances in Neural Information Processing Systems 18 | 2005 | PDF



Preference of sensory neural coding for 1/f signals

Yu Y, Romero R, Lee TS | PHYSICAL REVIEW LETTERS | 2005 | 10.1103/PhysRevLett.94.108103 | PDF





Decoding V1 Neuronal Activity using Particle Filtering with Volterra Kernels

Ryan C.Kelly, Tai Sing Lee | Advances in Neural Information Processing Systems 15 | 2003 | PDF



Hierarchical Bayesian inference in the visual cortex

Lee TS, Mumford D | Journal of the Optical Society of America A | 2003 | 10.1364/JOSAA.20.001434 | PDF

Computations in the early visual cortex

Tai Sing Lee | Journal of Physiology-Paris | 2003 | 10.1016/j.jphysparis.2003.09.015 | PDF

Adaptation of the temporal receptive fields of macaque V1 neurons

Richard Romermo, Yuguo Yu, Pedram Afshar, Tai Sing Lee | Neurocomputing | 2003 | 10.1016/S0925-2312(02)00799-3 | PDF

Neural Basis of Attentive Perceptual Organization

Tai Sing Lee | In Perceptual Organization in Vision: Behavioral and Neural Perspectives Ed. M. Behrmann, C. Olson and R. Kimchi, Lawrence Erlbaum Associates, 431-457. | 2003 | PDF

Analysis and synthesis of visual images in the brain: evidence for Pattern theory

Tai Sing Lee | In Mathematical methods in computer vision, Lecture notes in Mathematics and its Application. Ed. P. Olver and A. Tannenbaum. Springer-Verlag, 87-106. | 2003 | PDF



An information-theoretic framework for understanding saccadic behaviors

Tai Sing Lee, Stella X.Yu | Advances in Neural Information Processing Systems 12 | 2002 | PDF


Neural activity in early visual cortex reflects behavioral experience and higher-order perceptual saliency

Tai Sing Lee, Cindy F.Yang, Richard D.Romero, David Mumford | Nature Neuroscience | 2002 | 10.1038/nn0602-860 | PDF


Spike train analysis for single trial data

Richard Romero, STai Sing Lee | Neurocomputing | 2002 | PDF

A Hierarchical Markov Random Field Model for Figure-Ground Segregation

Stella X.Yu, Tai Sing Lee, Takeo Kanade | Lecture Notes in Computer Science | 2001 | PDF


Informatics of spike trains in neuronal ensemble

Xiaogang Yan, Tai Sing Lee | Proceedings of the 22nd Annual International Conference of the IEEE | 2000 | Doi

Estimation of temporal kernels for cells in V1

Richard Romero, Pedram Afshar, Tai Sing Lee | Proceedings of Annual Conference in Computational Neuroscience | 2000 | PDF


What Do V1 Neurons Tell Us about Saccadic Suppression

Stella X.Yu, Tai Sing Lee | Journal of Neural Computing | 2000 | PDF

The role of the primary visual cortex in higher level vision

Tai Sing Lee, David Mumford, Richard Romero, Victor A.F.Lamme | Vision Research | 1998 | 10.1016/S0042-6989(97)00464-1 | PDF

Image Representation Using 2D Gabor Wavelets

Tai Sing Lee | IEEE Transection of Pattern Analysis and Machine Intelligence | 1996 | 10.1109/34.541406 | PDF

Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image segmentation

Song Chun Zhu, Tai Sing Lee, A.L. Yuille | Pattern Analysis and Machine Intelligence, IEEE Transactions on | 1996 | 10.1109/34.537343 | PDF

The Role of V1 in Shape Representation

Tai Sing Lee, David Mumford, Song Chun Zhu, Victor A.F.Lamme | Proceedings of the Annual Conference of Computational Neuroscience 96 | 1996 | PDF


Representational Strategy in the Visual Cortex

Tai Sing Lee | Image Processing, 1994. Proceedings. ICIP-94., IEEE International Conference | 1994 | 10.1109/ICIP.1994.413639 | PDF

Texture segmentation by minimizing vector-valued energy functionals: The Coupled-Membrane model

Tai Sing Lee, David Mumford, Alan Yuille | Lecture Notes in Computer Science 588, 165-173, Springer-Verlag. | 1992 | PDF