The job of any sensory system is to create objects in the world out of the incoming proximal stimulus energy. The energy is neutral; it does not specify the objects itself. Thus, sensory systems must abstract the energy that does specify objects and differentiate it from the noise energy. The perceptual variables that specify objects for both listening and looking become those of contrast and correlated change across space and time, so that perceiving occurs at several spatial and temporal scales in parallel. Given that the perceptual goals and perceptual variables are equivalent, the rules of perceiving will be the same for all senses.

The goal of this book is to describe these conceptual similarities and differences between hearing and seeing. Although it is mathematical and conceptually analytical, the book does not make explicit use of advanced mathematical concepts. Each chapter combines information on hearing and seeing, and gives a detailed treatment of a small number of topics. The first three chapters present introductory information, including properties of auditory and visual worlds, how receptive fields are organized to pick out those properties, and whether the receptive fields are optimized to pick up the structure of the sensory world. Each subsequent chapter considers one type of perceptual element: texture, motion, contrast and noise, color, timbre, and object segmentation. Each type of perceptual situation is described as a problem of discovering the correlated energy, and the research presented focuses on how humans manage to perceive given the complicated set of skills required. This book is intended for use in upper-division undergraduate courses in perception and sensation, cognitive psychology, and neuroscience. It will fill the slot between textbooks that cover perception and sensory physiology and neuroscience, and more advanced monographs that cover one sense or topic in detail.