Images Can Be Reconstructed from Brain Activity

A new study suggests that by analyzing functional magnetic resonance images (fMRIs) of the brain with an elegant mathematical model, it is possible to reconstruct thoughts.

Researchers at Radboud University Nijmegen (The Netherlands) used data from an fMRI scanner to teach a linear Gaussian mathematical model how small volumes (2 x 2 x 2 mm) from the brain scans—known as voxels—respond to individual pixels. By combining all the information about the pixels from the voxels, the researchers were able to reconstruct what the test subject was looking at. The result was not a clear image, but a somewhat fuzzy speckle pattern.


To improve the performance of the model, the researchers supplied it with prior knowledge, by teaching it what the letters actually look like. The model thus compared the letters to determine which one corresponds most exactly with the speckle image, and then pushed the results of the image towards that letter. This improved the recognition of the letters enormously, resulting in an actual letter, a true reconstruction. The study was published early online on July 22, 2013, in the journal Neuroimage.

“Our approach is similar to how we believe the brain itself combines prior knowledge with sensory information. For example, you can recognize the lines and curves in this article as letters only after you have learned to read,” said senior author Marcel van Gerven, PhD. “We hope to improve the models to such an extent that we can also apply them to the working memory or to subjective experiences such as dreams or visualizations. Reconstructions indicate whether the model you have created approaches reality.”

“In our further research we will be working with a more powerful MRI scanner,” added lead author Sanne Schoenmakers, MSc, who is also working on a thesis about decoding thoughts. “Due to the higher resolution of the scanner, we hope to be able to link the model to more detailed images. We are currently linking images of letters to 1,200 voxels in the brain; with the more powerful scanner we will link images of faces to 15,000 voxels.”

Related Links:
Radboud University Nijmegen