Follow up: Building the Open Metaverse

Rev Lebaredian, VP, Simulation Technology, NVIDIA, kicked off the session. Rev covered recent developments on open standards at NVIDIA. He explained that recent advancements in computer graphics have been in entertainment, but with advancing AI, graphics are now useful for everything AI is useful for. 

“It's not reasonable to embed 3D into the existing 2D web: we need to build open standards around a full 3D web.”

Using the concept of LEGO pieces, Rev explained that no one company or organization can achieve everything necessary to build the metaverse, but we have the potential to build the standards and “plumbing” collaboratively. NVIDIA will leverage its strengths in rendering, physics, etc. to build tools that are additive to current technologies. “We need to try a bunch of stuff and fail fast.” Simulating complex virtual worlds is difficult, and we need standards that allow us to blur the lines between content creation and content consumption. 

Royal O'Brien, GM of Digital Media and Games, Linux Foundation, explained that having hubs for growth and scaling are important. Open sharing enables us to innovate faster. Royal explained the difference between open standards and open source and said open standards are critical in order to build future-proof content that is interoperable and scalable.

"Openness benefits both corporate and community interests and allows implementations to speak for themselves and creates an open and decentralized community." 

Vladimir Vukićević, Director, Software Engineering, Unity, wrapped up the section by stressing the need to build the infrastructure of the metaverse. 

“The open metaverse is going to be built from content of all kinds, content that is going to need to be world-aware and interoperable.  We're going to need to come to a common understanding of not just asset data, but attributes and behaviour, in order to let this content interact and truly connect with both the virtual and the real-world.”

Teddy Bergsman, Senior Director, Quixel at Epic Games briefly described the work being done digitizing our world to make digital world-building more accessible, with the long-term ambition to enable anyone to make high-quality scans for the Metaverse.

For a long time the creation of photorealistic digital worlds have only been available to people with access to expensive equipment. Teddy described his mission to scan representative textures from all biomes, beginning with a team of 200 people across the globe. Believing that all people should be able easily bring anything from the real world into the digital world with an acceptable level of quality via smartphones or other accessible technologies along with automating cloud processing, he stressed that scanning the world will be a job done by many. 

Shehzan Mohammed, Director of Product Management at Cesium said that creating a digital version of our very real and physical world is core to so many ideas and use-cases in the metaverse. ​Shehzan addressed the challenges of the metaverse including the need for high-resolution real-world data, the ability to process it, and then stream it over the web. We believe that 3D Tiles, the OGC open standard created by Cesium for streaming massive 3D datasets, will be the spatial index for the metaverse. 

Next, Mark Sagar, CEO and Co-Founder of Soul Machines, showed off Baby X, a digital human learning to behave like a real person with the power of AI. “The metaverse is coming," he said, and human-driven interactions will not be enough to populate the many virtual worlds of the metaverse.

"There is another way - digital agents that are Autonomously Animated. As AI and Digital Brains become more sophisticated, this will be the way to scale the metaverse. Soul Machines is creating this future to ensure that the metaverse will be useful and engaging,” Mark said.

Morgan McGuire, Chief Scientist at Roblox, started off by stating that “human communication is the heart of the metaverse.” He said, "Roblox built an exciting, safe, and civil metaverse in which our community of hundreds of millions of players explores and creates. Our research team is now inventing scalable graphics distributed systems, AI content creation tools, and natural language processing to power the next generation of online social experiences."

Next, Samantha G Wolfe, Founder of PitchFWD and Adjunct Professor at NYU, outlined how to explain the value of the metaverse in relation to the gaming, virtual reality, and augmented reality industries so people can relate the future possibilities to technologies they're already familiar with.

Ash M Richter, Senior Technology Architect for In-Q-Tel said, "Increased digitization and the rise of the Metaverse presents a unique technical and social challenge to the human species. To get something so vast and so important right and to do so in the service of globalized efforts, we-as a species- are going to have to break many of the self-imposed traditional barriers. We will need to break down the walls between silos: academia, industry, and government. We will need to build collaborations across nations and international organizations. And we will need to strategically look at a different timescale than we are used to in the modern era—planning for the longue durée of human civilization and the future of identity, record-keeping, storytelling, and the monetization and use of all of the above."

The next three presenters discussed open standards for the metaverse, beginning with Neil Trevett, President of The Khronos Group and VP, Developer Ecosystems at NVIDIA. Neil said that The Khronos Group has been building standards for the foundation of the metaverse for years without knowing it. Neil stressed that no one design or roadmap can serve all of the needs and solve all of the problems: we need a decentralized model that allows for a diversity of platforms and experiences. “Bringing together so many advanced technologies at scale requires standards for interoperability." A PC requires hundreds of standards that are taken for granted. So does plugging it in, and connecting it to the web. “The only way to build the metaverse is to build it together,” Neil said.

The following is a statement from Neil:

The metaverse will bring together many advanced technologies, with decentralized control, and so will need an unprecedented level of interoperability between platforms and products. Open interoperability standards have always been the foundation on which ecosystems are built to pervasive scale, and so we will need cooperative standards from many standards organizations—there is no other way to ensure that the metaverse will become ubiquitous.

Open source and open standards are complementary, and almost always leverage each other, but the two are often confused. An open standard is a shared specification, enabling implementations to be built and shipped by multiple companies for a technology to become pervasively available. Open-source projects by contrast enable multiple contributors to cooperate over building a SHARED implementation, creating an engine or application that could often not be built by a single company. As we build the metaverse, open standards can provide the broad connective foundation on which products and open-source projects can thrive and innovate.

Proprietary products are effective at driving the Darwinian evolution of new techniques and technologies. Effective standardization only emerges when there is a recognized, wider need for a proven, well-understood technology. In the long view, an open standard that is not controlled by, or dependent on, any single company can be the thread of continuity for sustained industry forward progress as technologies, platforms and market positions swirl and evolve.

Christine Perey, Spime Wrangler at PEREY Research & Consulting described the useful components of the metaverse such as continuous contextual search and movement along the space time continuum without leaving the physical world, although the physical world is the most important anchoring point for all of us. She talked through upcoming challenges technologists face, including technical challenges such as real-world capture and distributed architecture, societal concerns like privacy, user experiences, governance (compliance), and business.  “For a full interoperable metaverse, we need not just standards, but a full constellation of standards,” she said.

Finally, Dr. Nadine Alameh, CEO of the Open Geospatial Consortium, said she was happy to participate in this session on behalf of 500+ organizations at OGC who believe that location is a critical element for building the metaverse. She sees this as an opportunity for this expert community to contribute expertise with time and space, but more importantly the values of building ecosystems based on the FAIR, CARE, and open principles. 

“We have here a once in a lifetime opportunity to build something that is bigger than any of us and that none of us can build on our own! So if we genuinely want to build a sustainable operational metaverse at scale, we need to combine our strengths to ensure that our pieces, though independently developed and maintained, can actually integrate smoothly. And that is why we need standards!” she said. 

Dr. Alameh said that we are talking about integrating in a metaverse that is constantly evolving, which means that the standards that support this integration also have to evolve. “At OGC, we are also evolving how standards get developed so that we can deliver standards at the pace of today's innovation. The only way that we can all keep up and make the metaverse sustainable and scalable is by getting involved in this collaborative agile process,” she said.