To survive, organisms must both accurately represent stimuli in the outside world, and use that representation to generate beneficial behavioral actions. Historically, these two processes – the mapping from stimuli to neural responses, and the mapping from neural activity to behavior – have largely been treated separately. Of the two, the former has received the most attention. Often referred to as the “neural coding problem,” its goal is to determine which features of neural activity carry information about external stimuli. This approach has led to many empirical and theoretical proposals about the spatial and temporal features of neural population activity, or “neural codes,” that represent sensory information. However, there is still no consensus about the neural code for most sensory stimuli in most areas of the nervous system. The lack of consensus arises in part because, while it is established that certain features of neural population responses carry information about specific stimuli, it is unclear whether the brain uses (“reads”) the information in these features to form sensory perceptions. We have developed a theoretical framework, based on the intersection of coding and readout, to approach this problem. Experimentally informing this framework requires manipulating patterns of neuronal activity based on, and at the same spatiotemporal scale as, their natural firing patterns during sensory perception. This work must be done in behaving animals because it is essential to know which neural codes guide behavioral decisions. In the first phase of this project (funded by the BRAIN Initiative), we developed the technology necessary for realizing this goal. In the present proposal, we will extend our patterned neuronal stimulation technology and apply it to answer long-standing questions about neural coding and readout in the visual, olfactory, and auditory systems. We will pioneer the capacity to determine which neurons within a network are encoding behaviorally relevant information, and also to determine the extent to which temporal patterns of those neurons’ activity are being used to guide behavior. Finally, we will study these neural coding principles across changes in behavioral state and during learning to determine how internal context and past experience shape coding and readout. The contributions of the proposed work will be three-fold. First, we will provide the neuroscience community with the tools needed to test theories of how neural populations encode and decode information throughout the brain. Second, we will reveal fundamental principles of spatiotemporal neural coding and readout in the visual, olfactory, and auditory systems of behaving animals. And third, our unifying theoretical framework for cracking neural codes will allow the broader neuroscience community to resolve ongoing debates regarding neural coding that have been previously stalemated by considering only half of the coding/readout problem. NIH Brain Initiative Grant number: U19NS107464 PIs: Behtash Babadi, Dante R Chialvo, Tommaso Fellin, Mark H Histed, Patrick O Kanold, Wolfgang Losert, John HR Maunsell, Stefano Panzeri, Dietmar Plenz, Dmitry Rinberg, Shy Shoham