Google’s Fruit Fly Brain Map Is Already Being Used to Play Doom and Mario

news
Google’s Fruit Fly Brain Map Is Already Being Used to Play Doom and Mario

Google Research and HHMI’s Janelia Research Campus have completed a detailed digital map of an adult male fruit fly’s central nervous system, and developers are already experimenting with the data in unusual ways.

The project, called MaleCNS v1.0, contains more than 166,000 neurons and roughly 125 million synaptic connections. It is described as the largest complete fruit fly brain map yet by neuron count and was created after about a decade of work.

Researchers built the map by cutting the nervous system into millions of extremely thin sections, scanning them with electron microscopes, and combining AI based analysis with extensive manual verification.

The original goal is scientific. The open dataset can help researchers study how nervous systems process sensory information, control movement, and respond to damage.

Developers turned the connectome into a game controller

Soon after the dataset became available, software developers began using the mapped neural network as the basis for experimental game control systems.

One of the most notable projects came from software engineer Alex Wormuth, who converted the fruit fly connectome into an active neural simulation capable of interacting with Doom.

DetailInformation
DatasetMaleCNS v1.0
NeuronsMore than 166,000
Synaptic connectionsAbout 125 million
Research groupsGoogle Research and HHMI Janelia
Main scientific purposeStudy neural processing and behavior
Doom experimentNeural simulation controls movement
Other games testedSuper Mario 64, Beat Saber, Minecraft
Dataset availabilityOpen source

In the Doom experiment, game frames were converted into signals that could be sent to the simulated fly’s sensory neurons.

The resulting neural activity was then mapped to movement controls inside the game.

When the simulated player took damage, the system sent an electrical pulse to two dopamine related neurons known as PPL101. That mechanism was used as a simple reinforcement signal intended to distinguish better and worse outcomes.

Super Mario 64 and Beat Saber followed

Other developers quickly adapted the same brain model to additional games.

Programmer Jessica Paquette created a framework that used the roughly 166,700 neuron simulation to control Mario in Super Mario 64.

The results were limited, with the simulated system sometimes sending Mario into walls or struggling to interpret a three dimensional environment. Even so, the experiment demonstrated that the mapped biological network could be connected to an entirely different digital task.

Other projects reportedly used the connectome for movement in Beat Saber and Minecraft.

These demonstrations should not be interpreted as evidence that a fruit fly understands the games in the way a human player does.

The biological map is being simulated and connected to artificial inputs and outputs designed by programmers. The developers choose how game information is fed into the network and how neural activity is translated into controls.

The brain map is primarily a neuroscience project

The gaming experiments have attracted attention because they provide an accessible demonstration of what can be done with a complete neural wiring map.

The scientific value is broader.

A connectome of this scale gives researchers a detailed view of how neurons are physically connected across the brain and central nervous system. That can help with studies of sensory processing, movement, decision systems, and nervous system damage.

The map is also important because it provides a biological network with far more structural detail than simplified artificial neural networks used in many software experiments.

Larger animal brain maps could come next

Google is already considering more complex mapping projects involving animals such as fish and mice.

Those nervous systems would be substantially larger and more difficult to reconstruct, but the fruit fly project shows that large scale connectome mapping is becoming increasingly practical with modern imaging, AI analysis, and data processing.

For now, MaleCNS v1.0 serves two roles. It is a serious neuroscience resource and, thanks to open access, a platform for experimental software projects that test how mapped biological circuits behave when connected to completely artificial environments such as Doom or Super Mario 64.

Discover: News

Discussion (0)

Be the first to comment.