If Mark Zuckerberg is right, the metaverse will be the 3D successor to today’s 2D social media. But to get there in style, we’ll need to spruce up our virtual images.

YOOM wants to be your next-generation avatar maker.

In the online gaming world, avatars are a digital representation of a player. They can look like the actual person or be digitally enhanced. Want to breathe fire like a dragon or sport Viking horns? Sure, why not.

In the not-so-distant past, creating a realistic avatar that can move naturally through digital space required a dedicated studio decked out with cameras, bespoke software and enough computers to handle the processing requirements.

YOOM creates the same illusion with a few off-the-shelf cameras, artificial intelligence and computer vision software.

“If I want to be inside World of Warcraft or Roblox [a popular gaming platform with 1.7 billion users], I can use my phone to create my own avatar. I can augment it to look like an elf or to make me appear buffer,” YOOM CEO Gilad Talmon tells ISRAEL21c.

The avatar gets the 3D treatment courtesy of motion capture algorithms, the same technology used to make Golem in The Lord of the Rings or Caesar in the Planet of the Apes movies appear realistic. The effect works best when the player is wearing virtual reality (VR) goggles – or at least an augmented reality (AR) headset.

It’s not just for games.

“If you want to watch a Travis Scott rap performance in Fortnite, you can go as yourself and spend time with your friends,” Talmon explains.

Just like being there? Not quite, but as the metaverse gains complexity, will we eventually be unable to tell the difference?

The algorithm does all the work

Professionals can use YOOM to build avatars for players to choose from. That’s not an easy task. Consider Keanu Reeves in the video game The Matrix Awakens.

“They had to do motion capture of his face and full body, and they needed a 3D artist to clean up the capture and the motion,” Talmon explains.

“It’s a long and costly process to reach high-end levels. With our technology, it’s just filming the video and you get super high fidelity. The algorithm does all the work and then it applies it to all the pipelines you have.”

Established in 2016 as Tetavi, the company was renamed YOOM to better describe what it does: “volumetric capture,” or capturing images in three dimensions. When run in sequence, these images create the impression of motion. (A video, by comparison, is a sequence of 2D pictures.)

“It’s very fluid motion, not like the motion you usually see in games,” Talmon says. “The facial expressions look real.”

Talmon came to YOOM from aerospace contractor Elbit, where he worked on projects that “brought the connected helmet from the fighter pilot world to civilian uses such as search and rescue.”

3 petabytes of data

A YOOM-created avatar can be “gamified” by developers to do additional tasks the player cannot do in real life. Like jump over tall buildings in a single bound, perhaps?

This requires a lot of data, something YOOM has in abundance.

“We’ve scanned thousands of people,” Talmon says. “Different age groups, different regions, different body shapes and hair styles. We have a repository of three petabytes of data.”

Artificial intelligence fills in most of the gaps. “If you see creases on the front of my shirt, you can assume they will continue to the back. If there’s a player number on the front of a jersey, you can be pretty sure that it will be there on the back, too. But if I have a tattoo on my back, you’ll never guess that.”

The professional version of YOOM’s software is “sub-millimeter accurate,” Talmon claims.

The home version, to be released this year, is less exacting. But the stakes aren’t as high, either. “If you missed a small bit of hair or a bit of ponytail, you don’t care. All you have to do to onboard yourself into a game is take a picture. The algorithm does the rest.”

Music videos, sports

There are lots of possible use cases for YOOM.

For instance, musicians Besomorph and Riell have created videos using YOOM’s technology.

Most music videos today use a green screen to place the artist in a not-actually-there background. “It’s something that can take hours to set up,” Talmon says.

YOOM allows developers to build the musician’s motion-realistic avatar and then place it into a 3D environment where it can operate almost independently.

Want to sing a duet with Beyonce where you’re both in the video? YOOM can do it. (Beyonce would have to be a YOOM customer, of course.)

“It creates a new level of interaction between artists and their fan bases,” Talmon says.

In 2020, at the height of the Covid pandemic, then-Israeli President Reuven Rivlin recorded an Independence Day greeting using YOOM (when it was still called Tetavi) technology, inviting citizens to snap a selfie with a virtual version of himself.

The Los Angeles Kings used YOOM software to become the first NHL team to enter the metaverse.

YOOM took its portable production studio to the Kings’ practice rink in California and filmed players practicing in full gear as eight cameras captured their motions.

The same process was used to film the Kings’ lion mascot, Bailey, banging a drum and dancing around. The final product was shown on the arena’s video screens.

Big plans

YOOM has raised a total of $50 million and has 80 people on staff in New York, Tokyo, Los Angeles and Tel Aviv. The company has a licensing and royalties model similar to game publishers such as Unity, Roblox and Epic Games.

“We are essentially trying to change the way humans communicate,” Talmon tells ISRAEL21c.

“I think of human communication as transferring messages from one person to another. Most of human communication today is through flat screens. We’re trying to bring that to the 3D world where it’s more natural, as if the conversation were taking place sitting together in the same room.”

We’re just at the tip of the iceberg, Talmon stresses. “We don’t have a sense yet of where this will go. We’re focusing on gaming first because that already exists. Long term, it will truly change the way humans communicate.”

While the metaverse is still five to 10 years away, Talmon adds, “this immersive virtual environment is here today. When we think about where the world is going, this has the potential to a bring a huge net good to society.”

For more on YOOM, click here.