On December 8th 2021, Niantic announced an upgraded feature for their smash-hit mobile game, Pokemon Go! When players are using Pokemon Go! a mechanic of the game requires them to ‘spin’ pokestops which reward them with items that can aid in catching Pokemon or in battling with them.
These Pokestops are based on landmarks in the area, they even feature a photo, or occasionally an advertisement – it might be anything from London’s Big Ben, to a mailbox at the end of a street. For some time, Niantic has been asking players to scan these landmarks using the smartphone camera, essentially mapping the landmark in 3D space.
The new feature introduced more thoroughly rewards players for scanning the landmarks, by ‘powering up’ the poke stop connected to the landmark, giving out bigger and better items to players. This ‘power up’ builds in tiers; after five scans the poke stop enters level one, at ten scans it reaches level two and level three is met after twenty five scans.
‘Why does this matter?’ Well, because changes like this, which incentivise players to collect real world visual data is a major step in Niantic realising their vision for the ‘real world metaverse’.
Image © Niantic
The ‘Real World’ Metaverse
In August 2021, Niantic Labs founder and CEO John Hanke stated that the metaverse – one seen through a Virtual Reality headset – is a “dystopian nightmare”. Hanke went on to say “At Niantic, we believe humans are the happiest when their virtual world leads them to a physical one” and further explained that “unlike a sci-fi metaverse, a real-world metaverse will use technology to improve our experience of the world as we’ve known it for thousands of years.”
Niantic’s goal is to provide technology to enhance living, Niantic’s games urge players to leave the house to exercise and socialise. Hanke emphasises the original visions of the metaverse, seen in novels such as Snow Crash and the works of William Gibson, which are being held up as comparisons to many corporation goals currently, “served as warnings about a dystopian future of technology gone wrong”
As one of the world leaders in Augmented Reality technology, what Hanke envisions is a digital layer to sit atop our physical world. For now, this is seen through our phones. We hold up our smartphone and see the physical world through the camera, but an app would overlay the digital world for interaction or delivery of information. We already see this at its most basic when using the AR+ feature of Pokemon Go! as the Pokemon appears on the screen, superimposed over the local park in which we’re playing.
This technology can improve our lives exponentially by pairing AR features and 5G technology. By layering information over the ‘real world’ through smartphone apps it would remove any need for expensive additional equipment such as Virtual Reality headsets, plus allow users to still experience the world we live in through natural sensory input.
Some examples of a ‘real world’ metaverse could be; viewing a historical building through your phone, which changes the buildings appearance to show how it looked over the years. By raising your smartphone over a restaurant menu, you could tap a dish and see a photo of it, along with nutritional information. Looking at a billboard could transform it into a 30 second advert, better explaining the product or information.
Lightship and the 3D World
While Niantic are the leaders of the AR development world, they don’t want to do it alone. Launching Niantic Lightship, the company is putting their tools into the hands of smaller development teams across the world. Lightship’s ARDK tools allow for anyone to implement Augmented Reality into their games or apps, improving “depth, physics, occlusions and semantic segmentation.”
There are three core technologies within the Lightship ARDK which can enhance games and apps:
Real-time Mapping: Meshing and Semantic Mapping.
Using the smartphone camera, the Lightship tools can project a grid onto the world and objects within it. This helps the app or game to gauge distances and predict collisions. This is an important tool in achieving a level of realism as objects in the physical world, such as trees, buildings, benches, are seen as immovable. These tools mean that kicking an AR ball over a physical hedge would hide the ball from view. Or, the ball could bounce off the hedge and roll back to the user.
Semantic mapping is a little different and it ties into the Pokestop scanning from above. By combining scans of the physical world, taken by different users, the Lightship tools begin to craft a fully 3D digital space based on the real world. This is integral in defining pedestrian areas or examining architecture details. This mapping happens in real time, so as the user explores, the program combined with AI, can deliver a rough 3D model within seconds. This tool also ensures that all users experience the same interactions within the space.
Semantic Segmentation.
This tool uses computer vision-based capabilities to instantly recognise aspects of the physical world – the sky, trees, bodies of water, grass, artificial grounds. These elements dictate how they would then interact with the game or app being used and aid bringing AR creations to life. By understanding the physical world at a glance, the program knows how to ensure interactivity between digital and physical.
Multiplayer API
A large aspect of a ‘real world metaverse’ will be how people interact with each other and how they each view the same AR object. The ARDK tool is backed up by P2P networking and a managed server. This removes the need for developers to write their own multiplayer code. The tool currently allows for up to five people to enjoy and cooperate within the same AR experience with all of them seeing how they are interacting together within both the physical world and a digital space.
AR Opportunities
Companies are lining up to use the ARDK tools to elevate their brand presence and it’s clear to see why. Augmented Reality provides a great opportunity for businesses to interact with their audience in a more organic way. The music festival Coachella plan to use the AR tools allowing festival goers to watch stages come to life as the artist performs. Historic Royal Palaces are crafting an app which transforms the moat around the Tower of London, filling it with flowers adding “a dose of beauty, whimsy and history into our lives.”
The possibilities for brands are far reaching and seemingly endless. As the metaverse meets the physical world brick and mortar businesses can implement large-scale screens on the exterior of their buildings, turning any tiny city premises into Times Square, without damaging the natural facade. These advertisements can be projected into the sky above streets without the light pollution, or transform ‘ugly’ objects like a manhole cover into a piece of artwork advertising the latest blockbuster movie.
Image © ESB-Professional-Shutterstock.com
For developers utilising Lightship technology, they could easily pair their game or app with Admix’s In-Play SDK. This would implement our billboards and advertising opportunities, overlaid onto the physical world, earning the creators a passive income. This Augmented Reality ‘real world metaverse’ hopes to be a subtle mix of technology and the natural world. Non-intrusive monetization will be key for development teams who want to earn valuable revenue while putting the user experience first.
In order to have metaverse aspirations, a company or developer will need to work not only with available technology, but also with the world around them. Augmented Reality is an easy first step in the right direction, as a cost effective and accessible tool for many. Smartphones are important for realising the vision of Niantic. Their goals, and the tools with which they hope to achieve them, can expand learning, commerce, social interactions and entertainment, crafting a ‘real world metaverse’ that is as approachable as a battle in Pokemon Go!
