Brain-IT Rebuilds Seen Images After One Scanner Hour

Brain-IT rebuilds viewed pictures from one hour of fMRI, matching older methods trained on 40 hours. The Weizmann Institute of Science system, accepted at ICLR 2026, groups brain spots that share a job across people, then a diffusion model paints the scene.

The pizza slices and baseball shots are the demo. The change that moves lab time is the booking: one scan slot instead of 40 hours in a 7-tesla tube.

Volunteers Looked at Photos Inside a 7-Tesla Magnet

Roman Beliy, Amit Zalcher, Jonathan Kogman, Navve Wasserman, and Prof. Michal Irani, in Weizmann’s computer science group, trained the model on public scans. Those scans come from the Natural Scenes Dataset, collected at the University of Minnesota’s Center for Magnetic Resonance Research. In that set, eight people viewed thousands of natural scenes from the COCO photo collection while a 7-tesla scanner sampled the whole brain at 1.8-mm resolution every 1.6 seconds.

Each person saw about 9,000 to 10,000 distinct color photos across 30 to 40 weekly sessions, for 22,500 to 30,000 trials in a year. They were not asked to imagine a secret picture. They looked at a photo and reported whether they had seen it before in the study.

Irani calls “mind reading” a cute, jazzy name for that setup. The paper reconstructs the picture that was on the screen, from blood-oxygen changes in cortex, not a private daydream. Newer scans in the mix use voxels of about one cubic millimeter, she said, against about three cubic millimeters on older machines.

One Hour Now Matches a 40-Hour Record

Older visual decoders needed dozens of hours of a new person’s scans before they could guess what that person was seeing. The Brain-IT paper says results comparable to 40-hour recordings now appear after one hour of that person’s fMRI. On the lab’s project page, Subject 1 already yields meaningful pictures after 15 minutes.

One-hour transfer is not a new idea. MedARC’s MindEye2, from 2024, already adapted a shared model with about one hour of data. Tanishq Mathew Abraham, who worked on that line, wrote that Brain-IT’s 15-minute pictures were the part that was no longer garbage, after MindEye2 had held the one-hour mark. The Weizmann claim is narrower and harder: one hour that looks like a full 40-hour model, with layout and color closer to the photo on the screen.

THE SCANNER BILL

• Old load: About 40 hours of fMRI per new person on prior full-data models.
• New load: About 1 hour of that person’s scans for Brain-IT to reach that quality band.
• Short test: 15 minutes already produces a usable scene on Subject 1.
• Hourly rate: Tommy Sprague of UC Santa Barbara puts scanner time at $600 to $1,000 an hour, or $24,000 to $40,000 for 40 hours.

Sprague studies vision with the same class of machines. He said the 40-hour bill is why labs cannot map every new volunteer from scratch.

None of us can afford 40 hours of imaging for a new subject. It’s something like $600 to $1,000 an hour.

Tommy Sprague, neuroscientist, University of California, Santa Barbara

Scores below average Subjects 1, 2, 5, and 7 from that dataset. Higher is better for PixCorr, SSIM, and CLIP two-way identification (chance is 50 percent).

FULL-DATA SCORES ON FOUR NSD SUBJECTS

The project page says Brain-IT wins 7 of 8 standard metrics against those baselines. PixCorr rises 0.064 from MindEye2. SSIM rises 0.055. CLIP two-way identification rises 3.4 points from MindEye2.

Fake Brain Scans Fill the Training Gap

Eight brains and a few thousand photos each are a small pile by AI standards. The team’s fix is a two-way dictionary: a decoder that turns an fMRI volume into a picture, and an encoder that turns a picture into a predicted volume. Irani said about 70% of the training images never had a real scan sitting next to them.

THE ENCODER-DECODER LOOP

1. New photo: Start with a random picture that nobody saw inside the magnet.
2. Fake scan: The encoder predicts the fMRI that picture would have produced.
3. Rebuild: The decoder tries to recover the same picture from that predicted scan.
4. Repeat: Both models improve on pairs that were never collected in an MRI.

The encoder first chops each brain into about 40,000 voxels and maps those voxels onto shared clusters. A Brain Interaction Transformer then predicts two local feature maps: low-level VGG structure for layout and color, and high-level semantic features that steer a diffusion model. A coarse image from the low-level branch initializes that diffusion pass. The lab posted public Brain-IT code and encoder with a companion universal encoder paper.

128 Shared Visual Jobs Across Different Brains

While that encoder trained, it sorted cortex into 128 functional regions shared across people. Some of those jobs were already in textbooks. Irani said others were new to the team.

One split sits inside the parahippocampal place area, the zone that handles scenes of places. One part of it answered indoor views. Another answered outdoor views. Separate clusters lit up for food photos and for sports photos. The same cluster can sit in a slightly different anatomical spot from person to person, which is why a voxel-by-voxel copy from one brain to the next fails.

That shared map is why a new volunteer does not start from zero. The model already knows the jobs. It only has to learn how this person’s voxels sit on those 128 bins.

A Cake Comes Back as Three Sandwiches

The reconstructions still lie in ordinary ways. On a video call, Irani pointed to a cake that came back as a pile of three sandwiches, and a dog in a bathtub that came back as a goat in a bathtub of the same color. The object class can slip while the layout holds, which is the opposite of older systems that named the right thing and then painted it in the wrong place.

There exist nowadays models that translate brain activity into images, and they can even produce impressive reconstructions that preserve the semantic meaning of the image reasonably well. However, they tend to make mistakes in basic features such as composition and color. The new model we developed outperforms them in reconstructing both the content of the image and its details. What’s more, while every other model requires dozens of hours of brain scans to learn to ‘read’ a new person, our model needs only one hour.

Michal Irani, professor, Weizmann Institute of Science

HOW VISUAL DECODING GOT HERE

1. 2017: A Purdue model guessed which picture a person had seen and drew grainy stand-ins.
2. 2023: Yu Takagi and Shinji Nishimoto at Osaka University mapped fMRI into Stable Diffusion and rebuilt photos four people had viewed.
3. 2024: MindEye2 showed a shared-subject model could adapt with about one hour of a new person’s data.
4. March 1, 2026: Brain-IT’s second arXiv version posts the 1-hour and 15-minute transfer numbers.
5. April 23-27, 2026: The paper appears at ICLR 2026 in Rio de Janeiro as a poster.
6. September 14, 2026: Weizmann’s American committee releases the lab statement from Rehovot.

Each jump used a bigger generative model on the same scarce scan piles. Brain-IT’s bet is that fake scans plus shared clusters beat another year of magnet time.

The Clinic Case Still Lives in the Magnet

Irani wants the same pipeline for people who cannot speak, and later for imagined pictures, audio, video, and dreams. Judy Illes, a neuroethicist at the University of British Columbia, called the clinical idea magnificent. Locked-in communication already exists in a different form, with implanted grids that decode attempted speech, but those systems are surgical and person-specific. A one-hour fMRI setup would be slower and bulkier, and it would still require a person who can lie still in a tube.

The privacy scare skips that tube. Sprague said a covert scan of an unwilling person is still a stretch, because even willing volunteers are hard to keep still. He does think the same approach would likely work on pictures a person is thinking about but not seeing, once the model is calibrated. The path that worries him is EEG, a cap of electrodes, if models trained on magnets start to read the cheaper signal.

Marcello Ienca works on neuroethics at the Technical University of Munich. He said an EEG version would be a game changer for companies that already hold a calibrated cap, and he can imagine courts reaching for mental-image reconstructions as evidence. Irani said she is trying to think only of good things for now. Dreams remain a stated goal, not a result. She noted that an fMRI volume takes about two seconds, while video and dreams throw dozens of frames through the eyes in that window. That timing gap is still open.

WHERE EXPERTS DISAGREE

• Clinical use: Illes sees a path to help people with neurologic conditions; the paper itself only reconstructs viewed photos in a research scanner.
• Thoughts vs photos: Sprague expects the method to extend to imagined pictures after calibration; Irani says inner imagery and dreams are something the lab does not have yet.
• The EEG step: Ienca treats a move off the magnet as the commercial and legal risk; Irani is not treating that risk as current.

Until someone closes the two-second gap, Brain-IT remains a way to turn a booked hour in a 7-tesla magnet into a picture of what was on the screen, at a quality that used to take 40 hours of the same person’s time.

Frequently Asked Questions

Who Wrote the Brain-IT Paper?

Roman Beliy and Amit Zalcher share first authorship, with Jonathan Kogman, Navve Wasserman, and Michal Irani at the Weizmann Institute of Science Department of Computer Science and Applied Mathematics. The GitHub contact listed for the code is roman.beliy@weizmann.ac.il.

Is the Brain-IT Code Public?

Yes. The WeizmannVision brainit-fmri repository publishes the Universal Brain Encoder (image-to-fMRI) and the Brain-IT reconstruction pipeline, with an environment file and an inference script that writes low-level, semantic, and enhanced image folders per subject.

What Is the Universal Brain Encoder?

It is the companion image-to-scan model, posted as arXiv 2406.12179, that predicts fMRI from a photo so the decoder can train on pictures that were never shown in a magnet. The same 128 shared clusters are the units that encoder uses to move across people.

Where Was the Training Dataset Collected?

The Natural Scenes Dataset was scanned at 7 tesla at Minnesota’s Center for Magnetic Resonance Research, using 1.8-mm voxels and a 1.6-second sampling rate, while people did a continuous recognition task on COCO photos. Extra sessions cover rest, retinotopy, and category localizers, which Brain-IT does not use for the picture task.

Can Brain-IT Read Dreams or Silent Thoughts?

Not in the published experiments. Every scored picture was on a screen in front of a volunteer. Irani has said dream and video decoding still collide with the fact that one fMRI volume takes about two seconds, and she describes imagined content as a goal the lab is striving for rather than a current output.