Predicting and perceiving magnetic north: A predictive processing account of sensory augmentation

Sensorimotor theory (SMT) constitutes consciousness in the mastery of sensorimotor contingencies (SMC) rather than in properties of internal sensory pathways. Thereby it opens, as a major prediction of the theory, the possibility to substitute and augment sensory signals within perception. In line with this, we recently gave a psychophysical demonstration for changes to the perception of space with a sensory augmentation device (Schumann & O’Regan, 2017). The hearSpace headset signals head alignment to geomagnetic North to a user in a signal coding that mimics the SMC underlying auditory directional localization. Using this new ‘contingency mimetic’ approach to sensory augmentation, we could expand and compress perceivers experiential space. This result is in contrast to classical, non-biomimetic approaches to substitution and augmentation, which have so far established mostly cognitive but not perceptual experiences.

While classical SMT struggles to explain this pattern of results, here we provide an explanation within a predictive processing (PP) account of SMT. Perception corresponds to hidden variables in generative models best explaining sensory input via counterfactual knowledge of SMC. For this, backward connections pass predictions from generative models downwards; and lower-levels pass prediction errors upwards if predictions are not met. Unlike in SMT, in PP this internal loop is central, but it is unclear if classical substitution and augmentation signals manage to meaningfully interface with it. By contrast, contingency-mimetic augmentation “piggy-backs” artificial contingencies on natural contingencies; these act as a natural interface to generative models, thus explaining the perceptual effects obtained via this novel approach. Hence, summing up, contingency-mimetic sensory augmentation supports a predictive processing implementation of sensorimotor theory and provides a promising new tool to study predictive processing in consciousness science.