ARM Community: Why OpenCL will be on Every Smartphone in 2014 - ARM Community

What is the killer app for OpenCL? It's a question we get asked all the time. Recently, I figured out the answer. The killer app for OpenCL, and for GPU computing in general, is… are you ready? are you sitting down? It’s graphics.

Open C-What?

Let’s back up a bit. OpenCL, if you hadn’t heard, is the latest API for general-purpose computing on GPUs. That is, it lets you use the graphics chip in your laptop / tablet / cell phone to run all kinds of computations that used to run a CPU – and in many cases, to run them a hundred times faster. OpenCL was created by the Khronos Group from a draft specification contributed by Apple. It ships standard with the latest version of Mac OS X, and implementations are available for Windows and Linux as well. (If it’s new to you, check out our previous posts on the topic.) It’s very cool technology. The question is, why do I, my kids, my mom, and (by the way) about a billion other consumers want this technology? Why is it going to be a must-have feature of every 2014 smartphone?

Everyone in our industry is enough of a techie to see that OpenCL has huge potential: given that the average graphics processor A.) is programmable, and B.) can deliver tens, hundreds, or even thousands of gigaflops to an application, surely it must be useful for something besides drawing triangles? And surely that something is cool enough to create a must-have product? But when you actually try to nail it down, to say exactly why we're all going to want this capability on our cell phones in a couple of years, it’s hard. We can all see that OpenCL is going to be, shall we say, insanely great - but we don’t all see, or at least don't agree on, what it is going to be insanely great for. That thing – the use case that turns cool technology into something everybody has to have – is the Killer App.

The Quest for the Killer App

When the OpenCL Working Group gathered to start work on version 1.0, sitting around in a dingy hotel in Cupertino, the room was full of guys who had no doubt about what the killer app for OpenCL was: it was weather simulation, finite element analysis, turbulent flow modeling, and generally running the hell out of dusty old C code, ported from Fortran IV in the dawn of time by troglodytes. That's why the base language is C, rather than something more modern and maintainable. (Yes C hackers, I'm winding you up. Put up yer dukes, and come out fighting!) Don't get me wrong - the scientific computing community does vital work that makes our world a better place, which is why the rest of us on the committee let them have their way. But with all due respect, those aren’t applications that will make most of us run out to buy an OpenCL-capable cell phone.

There are other ideas out there. My boss at ARM, Jem Davies, thinks the killer app will be advanced imaging and, perhaps, augmented reality. A lot more people think it'll be, specifically, the branch of image processing known as computational photography. Some of them have done impressive work to enable those applications on mobile devices – see for example the 'FrankenCamera' project at Stanford (image here) and they have some compelling arguments. There are also people who think the killer app will be computer-vision-on-the-phone, or on-device navigation, or even on-the-handset financial modeling.

They’re All Wrong

Like I said at the beginning, all of these guys – especially my boss – are wrong. The killer app for OpenCL is graphics.

This truth first hit me a month or two ago, when I was at the High Performance Graphics (HPG) workshop in Saarbrücken, Germany. HPG draws a lot of ray tracing freaks, so naturally there were lots of papers describing GPU-based ways to build Bounding Volume Hierarchies and cast rays through them. But there were also papers on GPU-based stochastic rasterisation (useful for motion and camera defocus blur), on Reyes (micropolygon) rendering, and on useful technical nuggets such as procedural noise generation. The truth hit me again at SIGGRAPH, the biggest academic conference and trade show in the business. There was actually a session with the hilarious title "GPU Rendering" – I’ve got a novel idea, let's use the GPU to do graphics! - but looking further, there were papers and course sessions on using GPUs for cloth animation, for real-time radiosity, for crowd navigation, for hair and fur rendering, and even for synthesizing the sound of a shattering ceramic piggy bank. (Is that last one really graphics? Well, sort of…)

Of course, I should have figured this out years ago. At SIGGRAPH 2008, a team from AMD described the “March of the Froblins” tech demo. In the demo, a couple of thousand frog-goblins lead varied (if somewhat pointless) lives in a network of mountain valleys. They find and eat mushrooms when they’re hungry, they find and collect gold when they’re not, and they go to their camp and sleep when they’re tired. They do this while avoiding obstacles (such as each other) and choosing sensible routes from place to place, all while doing more traditionally graphics-y things like animating their skeletons, casting shadows, and reflecting light in interesting ways. And all of it, of course, is done on the GPU. The work is two years old, but it remains an impressive tour de force, and is a great way for the non-specialist to get a sense of just how flexible and powerful modern GPUs are. There’s also a very accessible paper.

The Killer App for OpenCL is Graphics

OK, an army of frog-goblins isn’t a killer app. Neither is awesome defocus blur, or cloth simulation, or even a shattering piggy bank. At least, not for most of us – they might make me (and a few other graphics geeks) run out and buy new phones, but not (I hate this word) “normal” people. However, normal people will buy a new phone that has a stunningly beautiful interface, or that runs stunningly beautiful applications. And the marriage of GPU computing and graphics, in the hands of real artists, can produce images that are truly stunning. So, in my view, the thing that drives OpenCL into every serious 2014 smartphone isn’t going to be a camera that makes me look ten years younger, or that can produce a panoramic image when I wave the handset back and forth. Nor is it going to be a phone that recognizes people I see only at weddings and whispers their names to me. It’s going to be a user interface that makes me smile every time I look at it. It’s going to be games that make my jaw drop, and web content (yes, OpenCL will come to the web) that makes me want to climb inside the screen. Of course, I do want the computational photography, the augmented reality, and the panoramic imaging, and I’ll gladly buy the apps when they’re available; but they’re not the killer app. The killer app is graphics. You heard it here first.

Agree? Disagree? Just like to fight? Let me know…

Tom Olson is Director of Graphics Research at ARM. After a couple of years as a musician (which he doesn't talk about), and a couple more designing digital logic for satellites, he earned a PhD and became a computer vision researcher. Around 2001 he saw the coming tidal wave of demand for graphics on mobile devices, and switched his research area to graphics. He spends his working days thinking about what ARM GPUs will be used for in 2013 and beyond. In his spare time, he chairs the Khronos OpenGL ES Working Group.

Shortlink to this post: http://bit.ly/bFlcbf