ARM Community: Creating the Augmented World with ARM Mali GPUs (Part 1 of 3) - ARM Community

The ARM® Mali™ Ecosystem today has a broad community of developers and Partners who have created visually-compelling applications for Android™-based platforms using Mali GPUs. The Samsung Galaxy S2, the best Android smartphone in the market today, is showing how our Partners can develop high-end games and stunning user interfaces, creating an unparalleled user experience.

ARM has developed a strategy to provide the best AR solutions by working across the Augmented Reality (AR) value chain with key Partners.

Augmented Reality is one area that greatly benefits from platforms which provide high-performance CPU plus GPU solutions within a low-power profile, as AR applications require responsive image tracking, 3D graphics and a sustained battery life to make the technology viable.

The “Augmented City”, jointly developed by ARM and metaio, is an example of a platform that combines an efficient, low-power hardware platform (such as the Galaxy S2 which is powered by a dual-core ARM Cortex™-A9 CPU and a quad core Mali-400 MP GPU) with metaio’s unifeye SDK to create a realistic experience in a virtual city.

In following the hype around the AR landscape, I have observed that the broad mobile ecosystem has been fanatically searching for a “killer app” and there is plenty of speculation as to what it could be.

Will it be a ‘Face Recognition’ add-on for Facebook using a head-mounted device, or a high-end mobile game that allows two or more players to interact with virtual characters superimposed on real objects in a large room, or will it be a location-based shopping experience like the ‘Minority Report’ Gap store?

We possess the technology today to make this dream a reality. As a consequence, we see AR at the peak of the hype curve, so will it all be downhill from here?

Maybe not.

Where is AR today?

AR is the method of overlaying virtual content over a real environment and augmenting it with user-relevant, location-based and interactive content using digital ‘senses’, namely: the camera, GPS, and an accelerometer coupled with an intelligent computing device. In a nutshell, this is the smartphone today. It embodies all the features required to make the phone aware of its environment to connect the user to the virtual content superimposed on the real world.

Critical Success Factors

However, technology is not sufficient to make AR succeed in the marketplace. For instance, GPS + Google Maps would be of no real use if there wasn’t a reliable data network to support the feature. Therefore, there are aspects of the smartphone which are essential to make AR technologically capable and commercially successful.



Anyone developing an AR solution today should carefully consider these aspects and ensure that the AR platform offers the user a “realistic and immersive” experience in the augmented world.

However, a vendor, typically an OEM, wanting to provide AR solutions on mobile phones faces a daunting scenario, as we see it: a fragmented AR landscape (see below) that consists of several platform providers and middleware suppliers, a wide but disconnected community of AR developers, and a large consumer market with little knowledge of, or access to, efficient and cost-effective AR solutions.

Today, platforms are available at a low cost but middleware (SDKs) tends to be quite expensive. Application developers are expensive while OEMs (and others) are willing to invest substantially into their AR solutions. This has resulted in even more hype with tier-1 OEMs wanting a part of the multi-billion AR market, as speculated by ABI Research and Juniper:

• “Market size of mobile AR will grow from $21 million in 2010 to $3 billion in 2016” – ABI Research
• ”The mobile augmented reality market will reach $732 million by 2014, fueled by paid application downloads, subscription services and advertising” - Juniper

In addition, metaio has predicted that ‘By 2014: Augmented Reality will be on every Smartphone’



The strategy to provide a solution to the end customer is to get a lucid understanding of the AR value chain and assess where we can add value. We believe that we should then use a bottom-up approach and provide solutions which are optimized, scalable and cost-effective to make it widely adopted by the developer community. They can in-turn provide uniform, robust and cost-effective solutions to the end consumer.

Working within the strong Mali Ecosystem, ARM has developed a unique strategy to help its Partners extract the maximum value from the AR value chain to reduce their cost of development and AR deployment (time to market).

More in Part 2 of this 3 part blog series…

Interested? Agree/disagree? Please let us know your thoughts.

Sri Kannan currently drives the Augmented Reality (AR), User Experience and GPU Compute Ecosystem at ARM’s media processing division. Sri has spent over a decade in the semiconductor industry as a CPU/DSP architect. He believes that graphics capability in an SoC will be key to enhancing the user experience on next generation smartphones because it makes a user’s interaction - be it gaming, shopping, navigation or social media - exciting and ‘true to life’.