Gaming and Android TV (Ubiquity Dev Summit 2016)
Hi everybody. I can’t really see you, but hello. My name is Krispy. I am a developer advocate for Google. I work in probably one of the coolest roles I think Google has. And that is, I am a games developer advocate. I play video games, and I’m paid to play video games. But not only that, I get to help our partners build their video games and release them.
Basically, my role as– it’s an engineering role. It’s to help our third party game partners integrate Google technology into what they do. So why am I here? What am I going to talk about today? Well, I am going to talk about game development and Android TV, something that I’m actually very, very passionate about. I’m going to talk about the constraints involved and give my thoughts on how this is a boom for game developers who are interested in making an impact on the evolving era of new TV. I will also endeavor to give you exposure to some helpful code that should allow you to jump in quickly and experiment. You’re all game experts, I’m assuming.
And it’s my hope that by the end of this talk, you’ll be able to do what you do already even better. And perhaps a good place to start with this is maybe a discussion about where video games first met TV. You know, the Nintendo Entertainment System, right? You might be inclined to think that the NES was groundbreaking. And it was. I’m not discounting that. It came out in 1987. And it was a phenomenal success. I mean, we still talk about it today. I don’t know if you saw my belt buckle. Right? This has influenced my career. So why is this important? Because it was influential. But we actually have to go back in time a little bit and look at the Magnavox Odyssey. 1972. All right? This is cited as the first video game console for the home. And the Odyssey was initially built with discrete components. This was before a deal with Texas Instruments that led to more condensed multi-chip design. Now, one side note here is that National Semiconductor proposed a single chip design to go into the Magnavox back in 1975– the first SOC, they were proposing.
But due to the existing deal that Magnavox had with Texas Instruments, the Odyssey 100 and 200 ended up shipping with Texas Instruments multi-chip solutions. But in 1976, the Odyssey 300 came out. Now, this version was one of the first game or TV game systems that included a single chip design. It was the first one, but it was one of the first. The owner of the first SOC in a video game system goes to the Coleco Telstar, which used the same chip as the Odyssey 300. And this chip was produced by General Instruments, and it was the AY-5-8500 chip. I mean, I’m not going to go into the specs of that. You can look that up on the tubes. But the thing that’s amazing about this is General Instruments– now defunct, by the way, they don’t exist anymore– was based in Pennsylvania and specialized in semiconductor and cable television equipment manufacture.
So what we have here is a case where TV is influencing video games, which is influencing TV, which is influencing video games. And they’re sort of ratcheting it up. All right. So I need to pause here, or else this is really going to turn into like a history lesson on video games. And that would be awesome, but it’s really not what I’m here to talk about today. So if you’re interested in this kind of thing, there’s a great book I would personally recommend to you called “Console Wars.” Use your favorite search engine.
Find a retailer. Pick it up. It’s a great read. And I felt that you would appreciate stuff like this, this level of detail, given that you’re interested in game development. Let’s go back a little farther in time. Let’s shift gears a bit and talk about old TV sets. For instance, did you know that the first electromagnetic, or electromechanical, television was demoed here in San Francisco on September 7, 1927? That’s pretty cool. We’re in the same city this stuff was first demoed in. Now, these TVs obviously were very primitive. But we’re looking at it from the future. At the time, this was groundbreaking. This was revolutionary. And also, the fact that we’re evaluating it as primitive kind of makes sense, given the available technology of the time. Certainly video games couldn’t exist without the developments that took place in early TV sets like this.
And the thing I want to highlight here is that the industry evolution of TV hardware is actually markedly different than that which took place in the computer industry. Simply put, the TV hardware space did not evolve as quickly as the computer industry did because of the use cases that were involved. TV is markedly different in a use case than a computer or a console. So if we just look at this quickly on a spectrum timeline, we can see electromechanical TVs hit retail shelves in 1928 to 1934. That’s when they were really popular. And that technology evolved into purely electronic televisions– yeah, they had transistors through vacuum tubes and stuff– in the mid 40s. And then we have the fast forward like 25 years, almost 30 years, to the Magnavox Odyssey before we see this next step in evolution of interactivity with television.
I mean, what you are doing in these early days was like turning a knob and maybe getting one or two channels. But the evolution proceeded. I didn’t include the modern stuff. I’m assuming all of you are more familiar with things that took place in the 90s and so on. I’ve mentioned use cases a number of times. Let’s dive in there just a little bit. Let’s look at, say, a console use case here on the left. With a video game console, what are you doing? You’ve got maybe a 16 button controller. And you’re pressing those buttons probably pretty rapidly. And you’re internalizing, as a user, what’s going on on the screen in front of you. And you’re building a mental model, and you’re reacting very quickly. And you’re pushing more buttons. And there’s this cycle, this repeated cycle, that goes very, very quickly. And you’re leaned forward when you’re engaged with this. You’re thinking. You’re active. You’re involved directly in the action that’s happening on the screen.
Now if we contrast that with what’s going on when you’re chilling out, you’re leaned back on your couch. You have your 40 button remote control. I don’t know why they have 40 buttons– honestly, too many buttons. But they’re also super high latency. How many of you have experienced the frustration of pressing a button, and you’re not quite sure if the TV really picked it up? Low latency. You’re not going to be making a lot of decisions very quickly with the TV. Usually it’s, I want to turn the volume up. I want to turn the volume down. I want to select something to watch. And that’s pretty much it. These are very different use cases, right? So why is this important? Why am I talking about this? And actually, you know what? I haven’t even talked about phones yet. Phones need to enter this discussion, I think. Now again, I don’t have time to go into the whole history of phones with the telegraph and where all this came in.
So I’m going to just basically say the mobile game market is huge. All right? I feel I don’t need to make the case for this. It’s self-evident. And given the successes that have happened in the mobile space with their evolution of technology, and engagement, and market success for a lot of the game development companies, the question naturally arises. Hey, can this be applied to TV? Can this whole model work in the TV space? I think it can personally, but with a caveat. And the caveat being, we need to set the right expectations for both the development communities and the consumer communities in order for this to be successful. I feel that currently a lot of expectations that come from other platforms, and other devices that you’re regularly using, both to watch media and play games, are affecting what your expectations are in the TV space. So I want to talk about a constraint, or what constraints are here. I’ll give you a second to internalize that. So I’m going to state clearly that Android TV is not trying to be a games console system. It’s not a games console platform, just like Android is not a games console platform.
But there are games that run very well on both of these types of form factors, the mobile and the TV space. I feel game developers need to be mindful and self-aware about targeting features that are specific to certain devices when those features are maybe not compatible with a broader set of devices within an Android or mobile ecosystem. For example, building a game that only works with a specific type of controller, or uses a specific hardware feature on a specific device, means that you’re adding your own constraints to your development process when you shouldn’t be. Don’t pass on your constraints to your users. I’m going to transition here into something slightly different. Let’s look at the spectrum of current Android TV hardware and what exists out there, because I don’t feel this gets enough service when we’re talking about this space.
There are really three classifications of hardware that we should consider, especially when developing games. There are operator boxes that we see here on the left. These are lower performance GPUs, CPUs. An operator box is typically made and given to subscribers of a cable operator’s service. They are meant to deliver the service value that is being promised by the service provider. There is an incentive for the service provider to produce something that is cost effective for their business. Ergo, you’re not going to get high performance chips inside an operator box, OK? The refresh cycle for these is completely dependent on the service operator and can typically range anywhere from 2 to 10 years. It depends. You can get a lot of longevity out of some of these boxes. And they’re optimized for video playback. Now if you look at the middle here, integrated TVs.
Integrated TVs are televisions that are smart. And they have an operating system. In the case of Android TVs, I think the most prolific one right now is the Sony Bravia line, which includes Android TV in it. These are great. I love these things. They’re huge. The color’s great. The experience is wonderful. They fit right in the middle when it comes to performance. This doesn’t mean they’re under-powered, but it also means they’re not overpowered.
You’re not going to be able to have 100 different shaders operating on your multi-dimensional BSD trees of tons of models being rendered in real time, and adaptive physics with physics-based rendering on top of that. No. You’re going to be able to get a good gaming experience like you would on a mid to low range phone or tablet. That’s the kind of performance you can expect on that. OTT boxes, over the top boxes– which are different than operator boxes. Over the top boxes are more console-esque in nature, in that they are hardware that a consumer typically buys on their own and adds to the TV in order to extend its functionality in some manner. So I’ve just put the example here of the NVIDIA Tegra.
That’s the SHIELD TV. This thing’s a beast. It’s got a lot of performance characteristics. It’s really powerful, and it’s definitely targeted towards gamers. I’ll talk more about that in a minute. Let me just go back to the TVs, the integrated TVs. I didn’t mention the refresh cycle here. TVs are typically refreshed, and this is sort of a global average, anywhere between six and nine years– every six and nine years, typically. And there’s not a lot of refresh cycle generations in the modern TV sense. But you can expect the user will upgrade their TV system every six to nine years. Different than a phone, which is every one to two years. Over the top boxes, this kind of fits in the middle ground. It could be anywhere from two, to four, to six years, depending on the features that it’s offering and how quickly those change.
So yeah, this is really important to consider as a game developer. Because you should be positioning yourself right in the middle for the broad distribution that I think everybody wants when they release a game. Let me just talk about the SHIELD TV for a moment. I think the hardware that NVIDIA’s created is great. I’m a gamer. I’ve got every game console under the sun. And I really enjoy playing with this. I’ve got a few of these at home. And while this is great at playing AAA content with a game controller, I’m not sure that you want to tie yourself necessarily to one specific vendor. The whole case that I’m trying to make here is you need to consider constraints in the broader sense of game design to allow yourself to reach the biggest audience that you can.
I’ll let you read this for a second. The larger market will be with integrated screens, such as what I showed you a couple of slides ago. This middle market flavor of Android TV is the one that I’m encouraging you to explore. In order for your game to be available to all potential users in the mid-range and above, the constraints are a bit more strict. The SOCs, systems on a chip, in the mid-range TVs were simply not designed for AAA games.
They were designed to create great media experiences and game experiences at a price point the middle market consumers can accept. The blending of use cases can be a little disorienting to game developers who are coming at this from a different experience model. Typically, you’re targeting a console. You are targeting a computer spec. TV is a different space. There’s a theme here. Now, if we talk about porting for a second, porting games from the mobile market into the TV market. This makes a lot of sense because, well, there’s a whole plethora of libraries out there. There’s code base there. There’s a lot of successful games that would make a lot of sense running on TV. However– however, however, however– I’ve seen this happen too many times when I review games. I will get a submission, and the simplest things throw me off. How many of you have touch screen TVs? Or even if you did– don’t answer that. Even if you had touchscreen TVs, would you want to get up from where you’re sitting on your couch, walk the 10 feet, and mash your greasy palm on that beautiful, slick glass screen? Probably not.
OK? So make sure that the games you’re making embrace the constraints of the TV and that they reflect it in the user model that you’re presenting. Now just to cap this off, I want to talk about two games. These aren’t TV games, necessarily. Well, “Badland” actually is on Android TV, but I’m calling these out for a different reason.
“Candy Crush,” I’m sure you’re familiar with the name “Candy Crush,” if you have not played it yourself. “Candy Crush” is amazing. And it’s amazing to me for one simple fact. They embraced a constraint of swipe and totally owned it. They made an experience that is phenomenal on mobile devices, because all you have to do is swipe. Everything else was sort of not focused on. It was the swipe experience and optimizing it for the most enjoyment. That was their constraint. In the case of “Badland,” what did they do that stood out in my mind? They made a game that works amazingly well with one button.
You can do a single tap or you can do a single click. That’s why it works on TV with a remote. And they totally owned the experience around that. It’s not overly complicated, but it is highly engaging. And the important thing is both of these games are played by people who are not gamers. They don’t self-identify as gamers. What they do is they enjoy an experience. And again, this goes back to why I’m talking about the middle market. All right. In the interest of time I must move on. But now we’re going to talk about some of the technical bits. These aren’t so much constraints as pieces of advice that I hope to imbue upon you to help you along your journey in developing games that either target TV first, or help you in porting your games from the mobile space.
So let’s talk about one that I think came up in the last presentation, but I’m going to beat on this point again. It has to do with this. Remember we talked about TVs not having touch screens? Yeah. If you have a game that’s, say, in the Play store that needs a touch screen when you’re on mobile, but you’re going to adapt it to work on TV. What do you do so it doesn’t get filtered? You do this. You set uses feature for Android hardware touchscreen required equals false. Now, why am I actually spending more than just a sentence on this? Well there’s one important thing here. Android developers haven’t had to worry about this particular uses feature before. Why? Because all Android applications implicitly assume touchscreen exists and is required. So if you’re going to build for TV and support mobile, you need this. This is the most important thing. Don’t forget this. While we’re at it, here’s a couple other things that TVs don’t have that you probably want to feature out.
I don’t know about you, but I don’t have a camera on my TV. And I don’t think I want one in my TV. So yeah, if you’re integrating middleware packages like some sort of ad library or what have you, some of those require camera. Remember to set its uses feature required false, or you’re going to filter yourself off of TV. I had a partner recently run into this problem. They’d released the game, Then they added an advertising package that required camera. And then they couldn’t figure out why their game wasn’t showing up anymore. So just quickly analyzing the manifest revealed, hey guys, you’ve got a camera requirement. TVs don’t have cameras.
Location. GPS. Unless someone is actively stealing your television, chances are it’s not going to move. So you probably want to remove, or at least set the required equals false for GPS. Microphones, I get a question about microphones occasionally. Microphones exist in the remote control, some remote controls. But believe it or not, the remote control is not part of your Android TV. It is an accessory. And so the device itself does not have a remote control, or at least current generation devices don’t have a remote control.
I don’t know what the future’s going to bring. But again, going back to that middle market, you want to reach as many TVs as possible. It’s a good idea to set required equals false on this. NFC. I don’t know. Again, why would you walk up to your TV with something and touch it? Maybe there’s a use case there I haven’t thought of. But again, you’re going to want to set required equals false for NFC, and obviously telephony. Yeah, my TV doesn’t do phone calls. And I’m glad for that. And then touchscreen obviously I talked about already. Now– pardon me. Now that we’ve turned off all these required features, what if I have code that uses these features? What do I do now? Am I totally boned? Like, ugh! No, no.
You just use a little bit of code. Now, I’m showing you Android code here, but this could be adapted for other languages. Unity has facilities to call into the Android stack and do these exact things as well. So you want to get the package manager. You want to ask the package manager, hey, do you have this feature that I think I want to use in this particular case? And you can branch from there. You can decide, hey, I’m going to use this or I’m not. In this case, this example that I’m showing you, we’re actually doing sort of a branch logic here. Hey, does it have a touch screen? And if it has a touch screen, does it have telephony? Then there’s a good chance this is a phone. If it has the touchscreen but not telephony, there’s a good chance that it’s a tablet. Well, what about the other case where it doesn’t even have a touchscreen? Does that mean it’s a TV? No. It just means that it doesn’t have a touch screen. Uh-oh. Now what do we do? Well, what if you need to detect if you’re running on an Android TV or a mobile device? What is a reliable way of doing this? The thing that we recommend is actually going to the UI mode manager and querying it for what UI mode it’s currently in, because Android TVs operate in what’s called lean back mode.
You know, when you’re leaned back and you’ve got your remote control, and you’re just chilling watching Netflix. So, yeah, again, very easy, light code that you can apply to your game logic. You might want to take a picture of this. This is important. With this check combined with the other checks around using features, you can quickly determine what the capabilities are of the device that you’re currently running on. This could be important for a multitude of reasons, inputs, or decisions about what you’re going to do with the UI, how constrained it is. One side note here, this is really useful for determining if you want to add sort of a– wow, my brain’s drawing a blank.
There’s a border you want to put around things to prevent overscan from affecting your game UI. Because developing for phone, you’ve got pixel perfect screens. You can push pixels right up to the edges and it’s fine. But on TVs, not all TVs are created equal because of this thing called overscan. Again, I don’t have a lot of time to go into it. But it can eat up to 5% around your entire border of your display space and you have no control over it. So the best thing to do is detect if you’re running on TV. Either give the user the option to adjust or scale the display area accordingly, or if you don’t want to provide that, you don’t want to build it in, just assume a 5% margin for yourself. And put all the important information more towards the center. While I’m on the topic of things we should do with TV UIs, we should probably talk about network state. This is more of a personal gripe that I have around what I see in a lot of games that I download on mobile.
Games that require network access and then only check for Wi-Fi make me really angry, because I have very high speed cell data plans that I could perfectly, or I could for sure take advantage of. But the games that prevent me from actually using it really bother me. On TV it’s a different story. You might have Ethernet or Wi-Fi. So now we’ve got this crossover spectrum of, it could be 3G. It could be Wi-Fi. It could be Ethernet. What do we do now? You should really just be checking, hey, am I connected to the internet? Do I have network access? This is the way to do it.
Don’t go after Wi-Fi. I mean, I’m not saying don’t ever check Wi-Fi. I’m saying for the general connectivity to the broader world of the internet, you should just do, am I connected? And then if you want certain features to only be available in Wi-Fi, then you can do a Wi-Fi check. Remember to always add the user’s permission, entry permission, access network state when doing this, or this won’t work. You need to put that bottom part in your manifest.
So what if you need to check for Ethernet? That’s actually not that much more code at all, actually. You can just, again, get the active network info and check it against type Ethernet in the connectivity manager constant. I’ll give you a second to take a picture or scribble anything because we’re going to shift gears again. I’m good at that, right? Shifting gears? No. Let’s go back to the infrared remotes. TV remotes have been around for a long, long time.
I think the first ones were actually wired right into the TVs. And I believe, if memory serves me, it was 1954. And NEC TV was introduced in the US that had a long cable with a channel up and channel down that actually had a mechanical actuator on the channel knob, and it would turn it to the different channels. We’ve progressed a long way from then. Most of the remotes that you get now actually have an 8-bit microcontroller in them to deal with the codes and do the IR blasts. But infrared remotes are a reality for today’s TV environment. And you have to consider that as a game developer.
Maybe you want to have a game that operates as a strategy game, something that’s less twitchy, that could take advantage of being played with a simple IR remote. If you can do that, you’re going to be compatible with almost everything. So consider this an optional design constraint, that if you can pull it off, you could be very successful with people, again, who are not labeling themselves gamers. All right? I’m going to quickly go through these last parts, because I’m running short on time. Likewise, maybe you want to step it up a notch and consider just regular remotes that come with Android TV. So you’ve got either Wi-Fi direct or Bluetooth-connected remote controls. Or maybe you want to go to the 16 button game controller. As we’re stepping up here though, we’re cutting off chunks of our potential audience.
So just keep that in mind. I’m going to close out here, the last technical piece before I bid you adieu. The game controllers that I mentioned on the end there, you’d think this would work. Right, if you wanted to specify that your game requires a game controller, this makes sense, yeah? And it uses feature Android hardware Gamepad. It follows the pattern of everything else.
Except you’d be wrong. This does not do what you would assume it does. And I’m about to break your brain. So we were talking before about accessories and microphones and stuff. And so I got this Android TV here. And I got my paired Bluetooth 16 button controller. And the reality is the controller doesn’t exist as far as Android as a platform is concerned. So what you actually need to do– and I realized I’m missing the slide for this now. Silly me. You actually need to specify that if your game requires a Gamepad, you have to put uses feature Android name, Android hardware Gamepad, required equals false. So if you require a Gamepad, the requirement has to be set to false. Does that make any sense? It’s something that I want you really just to remember. The reason for this has to do with a lot of legacy stuff that went on with the way the permissions system works and the filters work. But I want to convey it on to you. Please, if you’re going to build a game for Android TV that requires a Gamepad, set uses feature, Android hardware Gamepad, required equals false.
Just remember that. Because at the end the day, it all comes down to the experience that you’re going to provide to those in front of the screen. All right? Couple of resources here for you. We’ve got a great site on building TV games. Highly encourage you to check this out. It’s got a lot of information that is valuable beyond what I’ve talked about. And given that I’ve given you a few history lessons, that stuff is not on this site. I did a Medium post maybe six months ago just on how to detect Android TV and Unity. That might be useful to you. And lastly, we have an amazing site that’s been put together with a lot of hard work from our developer advocacy group on making games with Google technology that I’d love for you to check out and see if there are things in there that your games can benefit from. So I’m going to do questions offline.
As found on Youtube