Scientists are collecting massive amounts of images and video to study marine life and assess ocean health. AI can help researchers analyze this deluge of visual data more efficiently. Before AI can be used for ocean exploration, machine learning models need to be trained to identify ocean animals. FathomVerse, a new mobile game, seeks to address this challenge by engaging ocean enthusiasts around the world to help review and label images so AI can correctly recognize ocean animals.
“With an enormous challenge comes an enormous opportunity: How do we scale our capacity for analyzing valuable data about the ocean while realizing a new and inclusive vision for ocean exploration and discovery? By combining expertise in ocean science, human-AI interactions, and gaming, FathomVerse offers a solution,” said MBARI Principal Engineer Kakani Katija, who led the development of FathomVerse.
FathomVerse allows anyone with a smartphone or tablet to take part in ocean exploration and discovery. (Image credit: Lilli Carlsen 2024 MBARI)
To develop FathomVerse, Katija and MBARI software engineers collaborated with game design experts &ranj Serious Games—a Netherlands-based game development studio focused on positive behavioral change through play—and Internet of Elephants—a nature tech enterprise based in Kenya focused on rekindling relationships between people and wildlife.
“With more than 3 billion people playing video games globally, we wondered if we could engage a broader audience in ocean exploration through their mobile devices. We know that games can spark creative and unexpected solutions to real-world problems. With FathomVerse, we merged science with an innovative gaming experience, aiming to inspire curiosity, foster learning, and contribute to ocean exploration,” said GAF van Baalen, game director at &ranj Serious Games.
Cameras on MBARI’s advanced underwater robots have helped our scientists discover remarkable new species, observe unique deep-sea environments, and monitor ocean health. MBARI has amassed an enormous archive of deep-sea video. To analyze the video, researchers in MBARI’s Video Lab comb through thousands of hours of footage to identify and label animals and objects. This treasure trove of visual data includes more than 10 million observations of animals, behaviors, interactions, geological features, marine debris, and more.
Manually processing the enormous amounts of visual data that researchers have collected and will continue to collect is a daunting task, requiring a significant investment of time and resources. AI can help researchers analyze data more efficiently and scale with the ever-growing amount of visual data.
MBARI is supporting this much-needed effort by lending our expertise and resources to help accelerate the development of AI tools for analyzing ocean visual data.
Researchers in MBARI’s Video Lab have combed through thousands of hours of deep-sea footage to identify and label animals and objects. Those annotated images are being used to train AI to identify and classify ocean animals. (Image credit: 2022 MBARI)
Before artificial intelligence can be used to analyze ocean data, it must be trained to identify marine life. MBARI is contributing content from its archive of expertly-labeled deep-sea imagery to an open-source database called FathomNet. This database was developed in collaboration with MBARI software engineers and was seeded with nearly 100,000 labeled images collected by a variety of MBARI research teams over the past 35 years. This effort led to questions about how we can engage a broader audience in verifying the labeled image data needed for training machine learning models.
Manually processing images and video to identify animals and objects is very time-intensive. Machine learning models can accelerate this analysis. (Image credit: 2020 MBARI)
“Inspired by other community science apps like iNaturalist and eBird, our team wanted to design ways to tap into widespread enthusiasm for ocean animals while at the same time inviting a broader community of people to take part in ocean exploration and discovery,” explained Katija.
The team spent 18 months developing FathomVerse. After a successful beta launch in 2023 that drew nearly 1,400 players from 65 countries, the FathomVerse app launched publicly in the App Store and Google Play in global markets on May 1, 2024.
“The FathomVerse team worked closely with beta testers recruited with the help of MBARI and the Monterey Bay Aquarium to develop engaging gameplay. We’ve created a cozy underwater world where casual gamers can explore, play, and learn. Designed to be easy to pick up, you can enjoy it with your morning coffee or while waiting for the bus,” said Ocean Vision AI Engagement Coordinator Lilli Carlsen.
In the new FathomVerse game, players can:
- Launch minigames to find and label ocean animals using real imagery collected by researchers.
- Learn how to identify nearly 50 groups of ocean animals.
- Save favorite images and curate a personal gallery.
- Unlock awards to expand their knowledge of ocean animals and dive deeper into how gameplay improves artificial intelligence.
- Listen to Ocean Radio channels and cycle through different soundtracks inspired by the ocean soundscape.
- Be among the first to view new imagery collected by researchers exploring the ocean.
The FathomVerse mobile game features immersive imagery, engaging gameplay, and cutting-edge science. (Image credit: 2024 MBARI)
FathomVerse is one of three software tools developed as part of the Ocean Vision AI program led by Katija. Ocean Vision AI is an innovative collaborative project that seeks to make AI and ocean research more accessible and impactful. In addition to FathomVerse, Ocean Vision AI has released FathomNet, a database of expertly-labeled images and machine learning models that can be used to identify ocean animals. In summer 2024, Ocean Vision AI will debut the Portal, an online, collaborative tool for end-to-end AI-assisted processing of ocean imagery.
“With a triple threat of climate change, pollution, and overfishing, it’s more urgent than ever that we understand our changing ocean. We need all hands on deck to study the ocean at this critical crossroads. By tapping into our collective curiosity, FathomVerse seeks to transform ocean exploration by engaging a community of ocean enthusiasts to work alongside researchers,” said Katija.
Funding for FathomVerse came from the US National Science Foundation’s Convergence Accelerator, the David and Lucile Packard Foundation, and the Schmidt Marine Technology Partners, a program of the Schmidt Family Foundation.
Learn more about FathomVerse at fathomverse.game.
