I recently finished a port of my game Exterminator from Windows Phone 7 (WP7) to the Android platform, what follows is an account of my experiences I promised to a few people.
The Windows Phone is excellent and the development tools are outstanding but Android has a significant lead in the marketplace that I wanted to tap into. (iOS to follow soon...?)
I wrote my WP7 game using XNA after installing the WP7 development tools on top of Visual Studio 2010. I chose XNA over Silverlight for the app technology - my real job involves lots of Silverlight expertise and the game could have easily been written in it and perhaps finished even sooner but I decided I wanted to expand my XNA knowledge (this later turned out to be a good decision, allowing my port to Android to be much simpler). The documentation and samples for XNA are fantastic and the fact I could get a bouncing ball up and running on my WP7 in less than 5 minutes spurred me on to do a full game.
Development of even a fairly simple project should always be done using a source control system and I prefer to use Subversion because of it's simplicity and lightweight. You might want to consider TFS or GIT (the latter allowing you to contribute directly to the monogame project by sending them fixes if you so desire). The time spent setting up and learning to use a source control system will pay for itself many, many times over.
After submitting my game to the Windows marketplace I decided to port my game to Android and looked for cross platform solutions to do this. I found Mono for Android (MFA) which is a port of the .NET API and MonoGame which is an "in progress" port of the XNA api (with iOS being more complete than Android at this point). I purchased MFA from Novell, before Xamarin existed, and downloaded MonoGame. In my game's existing Visual Studio solution I added a new top level "Mono for Android" application and added all my existing game code files with "Add as Link" so that I only had one real copy of the file shared between the two projects. This would mean I could edit something only once and it would be affected in both the Windows Phone and Android versions at the same time.
After a little messing about - chopping out incompatible code, tweeking a few things here and there (ie. very little work really) - I had a sort of working Android version of my game. In an ideal world this would be "it" - we'd be done and it is getting there...
I started out using emulators for Android and quickly found that to be a complete pain. I appreciate the ability to test on various versions of the platform and with various screen resolutions, etc. but the speed of the emulations and debugging on them caused me so much frustration that I abandoned it and instead begged, borrowed and bought several Android devices to debug on. This is a much better solution, it's quicker, more stable and more representative of what your users will see.
Running my game on Android very quickly crapped out. The Android logs (use either using DDMS or 'adb logcat') showed "excessive JNI global references (2001)". I tracked this down, using the memory leak checking tools in DDMS, to be in the Accelerometer code (thread here) I initially thought this to be a monogame problem but turned out to be an issue in MFA which as of 4.01 has still not been addressed. The logs were telling me that something was leaking rather badly in the accelerometer code and it capped it at 2001 leaks. If I ran under release on a real device the game would run for much longer but still crap out at over 50K leaked refs, so the debug leak checking code in the "dalvik" VM was doing its job and you should not ignore these - the issue needed to be addressed. I ended up circumventing MFA for the accelerometer handling and rolled my own using JNI (more on JNI later).
Several of my textures in the game showed just fine but a lot of the others did not show at all, I tracked this down to a requirement (because of the underlying version of OpenGL used by Monogame) that textures must have dimensions of powers of Two, some of my textures did by happen chance and so they worked. I made the fix to not require pow2 textures in my copy of monogame and informed CartBlanche who then included it in the Android branch of the monogame project. If I was more proficient at GIT I could have pushed (or pulled??) the change to him directly.
Some of my font spritesheets were just slightly over 1Mb. Because of the version of Android used you cannot compress any resource greater than 1Mb in the APK file, there is a setting in the project properties for MFA to exclude certain extensions from compression and I tried using it. Unfortunately it didnt work and I had to bug the folks at Xamarin (they are very responsive and helpful - the 'Johns' in particular are great to work with). Thread Here. I could tell they were not being left uncompressed by looking in the final generated APK file. An APK (android package?) file is just a ZIP file. You can rename it as ZIP and open it up to look at the classes and resources contained within. The XNB's for the spritefonts were being compressed event though I added .xnb as a "dont compress this" extension.
Getting Paid (AdMob integration)
Because of the poor conversion rate of free to paid versions of my app to date (1 in 600) I figured I wouldn't bother with a Ad-Free version on the Android marketplace and instead just use AdMob for ad revenue. I found a very good article by Greg Shackles which showed me exactly how to do that. I signed up for an admob account and got this running very quickly (thanks Greg!) - his article also started teaching me how to use JNI.
Note: this is the old AdMob component, the new one will not integrate properly because of an Android 3.2 dependancy (new changeConfig's), unfortunately you can't target Android 3.2 with MFA and have to target 4.0 instead. Then you find out that the underlying InputEvent changed from MotionEvent and when you hit the screen your app will now crash (you target 4.0, because you had to - but play on a 2.2 device, boom!). So you're stuck with the old AdMob component, this turned out to be a bit of an issue - see later (the last bug).
Tip! Make your game play itself!
You will get sick of playing your game, you will do it literally hundreds of times and it'll start to drive you nuts, no matter how good it is. For this reason I put a mode in my game to play itself, I built an AutoPlayer component which acted very much like a real player (with several varying skill levels), I simulated visual acuity, hit accuracy, response times etc. (the "names" of my players varied from "Mr.Magoo" to "Terminator" :) ). I spent over a day making this simulation player but again, it paid for itself many times over (and saved my sanity). Using this AutoPlayer I could test hundreds of games over many hours, exercise my high score service, get good ideas of MTTF (mean time to failure), find bugs and generally increase the reliability of my game. If you make your random number generator start with a known seed you can even create reproducible crash scenarios as the autoplayer will always play the same "random" game for a given seed. So if it crashes at 25minutes and you miss it, you can restart, let the autoplayer do his thing and come back in 25 minutes, you *should* see the same problem. This is a very powerful trick for debugging.
Always keep the x86 version of your game running.
The windows phone is a very prescribed device, the resolution (as yet) is fixed, 480x800 and since I first wrote the game for the Windows Phone this is what every coordinate in my game was based
on. Android devices come in all sorts of shapes and sizes and you will need to accomodate for that. I built a class that did very simple scaling between 480x800 to the actual resolution of the android device. The best way to do this was to "extend" my numeric and geometric primitives (int/float/rectangle/etc) with a Scaled() attribute using C# extension methods - what used to be...
new Rectangle(100,200,40,50);
simply became ...
new Rectangle(100,200,40,50).Scaled();
Debugging this scaling was a bit of a pain to do on the device itself and I still had the x86 project working from when I originally created my XNA project so I switched to that, spoofed the devices resolution and used the x86 app to shake out the problems - much quicker.
Know thy JNI
Java Native Interface. This is someting a .NET developer would probably rather avoid but this is the way you can call into classes created with Java from your C# code. Mono for Android itself basically makes a bunch of calls into the underlying Android OS via java librarys using JNI and expose it like the .NET API - you can see it yourself using Reflector on the Mono assembly and this is a great way to learn how to use JNI.
You will need to use JNI when you want to do something that has not already been exposed in Mono, for instance, if you want to post to facebook using the facebook API jar then you will need to use JNI. If you need to work around any bugs you find in Mono you may also need to use JNI.
Any java classes you create yourself you will likely want to wrap in a C# class that uses JNI. The best way I have found to find those required funny signature strings for functions is to take the resulting DEX file generated from say Eclipse and use DEXDUMP it will spew out the functions contained along with the parameter and return value signatures.
This brings up another good point, you can compile java in Visual Studio by setting the Build Action to AndroidJavaSource (dont forget to make your .java file ASCII using 'Save with Encoding' from within Visual Studio - java doesnt like Visual Studio's UNICODE text files), you can use JAR files by setting the BuildAction to AndroidJavaLibrary. but using java from within Visual Studio is a pain, you will want to download Eclipse and play with your java in there first. When it's all working bring it over to Visual Studio. Eclipse also gives you the tools to make dialogs using axml visually.
Use Logging
The Android OS and apps are very chatty when viewed through their logs. Run DDMS from the Android SDK and click on your device in the window, you will see a very informative spew. You can log to this yourself using
Android.Util.Log.Info("YOURTAG","Hello World");
There are also variants for .Error, .Warn, .Debug.
In DDMS you can filter on YOURTAG so you can see just your logs. If you experience a crash in your app the chances are very high you will see a usable stack trace in the logs. Use logging!
Tombstoning/App Lifecycle problems with OpenGL/monogame
The Android App lifecycle allows the system to shut your app down at any time and rehydrate it later, you have to listen to several events and store/restore state. I did this but when my app came back alive none of the textures that were used before the tombstoning event showed correctly (they showed as rectangles instead). It appeared the surfaces were lost in the shutdown and not restored. This is a bug in monogame (or OpenTK?) which needs to be addressed. I had all the existing state preservation code from my WP7 implementation which I could use so I simply forced a kill process "onPause" so that it has to be restarted from scratch if you rerun the app. This breaks the Android app lifecycle model somewhat and is much slower but I had no choice. My way the app has to be restarted so all the textures are built from fresh. I had even tried rebuilding all the resources onResume but that too did not work. Dean (monogame) is looking into this issue, you may have to do what I did.
Localizing resources
My application is translated into French, Spanish, Italian and German using the good old tried and true .RESX localization schema. ie. Strings.resx with Strings.<lang>.resx for all the other languages. This is great for .NET but unfortunately MFA does not map this over to the Android localization schema. If you want to only have one set of string resources you will need to do some massaging to generate the Android resource equivalents of the localized files. I put together a little command tool that takes my resx files and generates the required Android Values-<lang>\Strings.xml files as well as a spoofed Strings.Designer.cs file so that I can leave my string resource code looking exactly the same between my WP7 and Android apps. I run this tool as a Build step from within Visual Studio. Hopefully soon MFA will handle this as part of their build scripts.
As an aside, I originally released my app in English, even for the other locales, when Zune marketplace was updated they disallowed this and demanded localized texts. Having very limited resources I had to rely on Automatic translation and a bit of common sense to accomplish this but - something unexpected happened - previously everyone seemed happy, even the French - with an unlocalized English UI, but when I released the "machine" localized one I heard a lot of complaints about poor localization!!! Its true, the machine localization is not great, but I realized then that not localizing at all is better than half localizing. Hmmm... must save up for professional localization...
The ugliest hack I have ever made.
One of my Android devices is an Optimus S, it's a very low end, small screen, Android 2.2 device and I use it as my "min-bar". My game would run on this device but then randomly crash with a stack
ending in AndroidGameView relating to setTime (as gleamed form the logs in DDMS).
OpenTK.Platform.Android is a component written by Xamarin in the OpenTK project, unfortunately they havent released the source so I couldnt just "fix" it - Reflector to the rescue! I disassembled AndroidGameView.RunIteration I could see that they determine the update and draw times by comparing to DateTime.Now values. If, however, the system, in between Draws or Updates adjusted the system clock (which it does quite often on my Optimus S for some reason) these times could come out negative and "boom". Unfortunately these values were stored in private fields. My "hack" - wait for it - every 1/60s I analyse these private fields using reflection and check them against DateTime, if they go negative I "poke" a value into them which is (DateTime - 1/30 sec). Yuck. But it seems to work, my Optimus can run for much longer now.
Apparently AndroidGameWindow is getting a complete re-write in an upcoming version of MFA. I will definately be upgrading to that so I pull out this heinous hack.
Sounds not working on Android
It was only at this point that I realized I was probably the first to really do a game for the Android using monogame, since sound simply did not work. There was a bug in the code that accessed the
resources for the MediaPlayer, I fixed the bug and sent it to CartBlanche. I had a problem with the use of MediaPlayer for SoundResources so I rewrote the Android monogame sound to use SoundPool, this has a problem in that it will not play long sounds but in my case this was not required. The SoundPool implementation also made it back into the monogame code base.
MFA?/VS?/ADB? - memory usage in Visual Studio while debugging
One of the problems that slowed me down the most was a nasty bug (x64 related perhaps?) apparently either in VS or MFA (or Android SDK? ADB? ...????) which causes memory usage for VS to rapidly cycle up and down between 700MB and 2GB, this thrashes my machine and brings me to a stop. I have to keep rebooting my machine, this doesnt always happen but when it does it's a complete blocker.
You can tell by all the question marks that I dont know where this bug really lies but I do see that memory profile going crazy so something is tweeked.
The last bug (that I know of...)
My game has 16 levels - each taking a minute to complete, with the other screens in between it can easily take 20 minutes to play my game end to end. The game was looking really good but after 10 or so levels there was a slightly perceptible delay between hitting the screen and the touch being registered. I let my game play itself for an hour and the delay became progressively worse, up to the point of making the game unplayable. This was a tough one to track down and took me the best part of 3 days. To make a long story short to track this down I had to start commenting out chunks of my app, running the autoplayer for long periods of time and seeing if the problem went away. An unsatifyingly primitive technique that consumed wads of time.
Eventually I pulled the AdMob component (yes, I should have tried that first, it's obvious in retrospect) - ran the game all night and in the morning... no lag! Gotcha!
I tried updating the AdMob component to the latest one to see if they had fixed the issue but because of the issues described above I could not use the later component. I now tear down (dispose) the ad component each level and new it up again, this seems to get rid of the bug.
Time to release!
Releasing your Android game
Xamarin has good instructions on how to post to the Android marketplace so I'll not reproduce that here. I also posted to the Amazon marketplace that is currently running a free to upload service (usually $99/yr) but my app is stuck in the review process as I write this - probably because it is the holiday season.
Both the Amazon and Android marketplaces allow you to point to a YouTube video as part of your promotion so you might want to think about doing that. Here's mine.
Tools Used
Any java classes you create yourself you will likely want to wrap in a C# class that uses JNI. The best way I have found to find those required funny signature strings for functions is to take the resulting DEX file generated from say Eclipse and use DEXDUMP it will spew out the functions contained along with the parameter and return value signatures.
This brings up another good point, you can compile java in Visual Studio by setting the Build Action to AndroidJavaSource (dont forget to make your .java file ASCII using 'Save with Encoding' from within Visual Studio - java doesnt like Visual Studio's UNICODE text files), you can use JAR files by setting the BuildAction to AndroidJavaLibrary. but using java from within Visual Studio is a pain, you will want to download Eclipse and play with your java in there first. When it's all working bring it over to Visual Studio. Eclipse also gives you the tools to make dialogs using axml visually.
Use Logging
The Android OS and apps are very chatty when viewed through their logs. Run DDMS from the Android SDK and click on your device in the window, you will see a very informative spew. You can log to this yourself using
Android.Util.Log.Info("YOURTAG","Hello World");
There are also variants for .Error, .Warn, .Debug.
In DDMS you can filter on YOURTAG so you can see just your logs. If you experience a crash in your app the chances are very high you will see a usable stack trace in the logs. Use logging!
Tombstoning/App Lifecycle problems with OpenGL/monogame
The Android App lifecycle allows the system to shut your app down at any time and rehydrate it later, you have to listen to several events and store/restore state. I did this but when my app came back alive none of the textures that were used before the tombstoning event showed correctly (they showed as rectangles instead). It appeared the surfaces were lost in the shutdown and not restored. This is a bug in monogame (or OpenTK?) which needs to be addressed. I had all the existing state preservation code from my WP7 implementation which I could use so I simply forced a kill process "onPause" so that it has to be restarted from scratch if you rerun the app. This breaks the Android app lifecycle model somewhat and is much slower but I had no choice. My way the app has to be restarted so all the textures are built from fresh. I had even tried rebuilding all the resources onResume but that too did not work. Dean (monogame) is looking into this issue, you may have to do what I did.
Localizing resources
My application is translated into French, Spanish, Italian and German using the good old tried and true .RESX localization schema. ie. Strings.resx with Strings.<lang>.resx for all the other languages. This is great for .NET but unfortunately MFA does not map this over to the Android localization schema. If you want to only have one set of string resources you will need to do some massaging to generate the Android resource equivalents of the localized files. I put together a little command tool that takes my resx files and generates the required Android Values-<lang>\Strings.xml files as well as a spoofed Strings.Designer.cs file so that I can leave my string resource code looking exactly the same between my WP7 and Android apps. I run this tool as a Build step from within Visual Studio. Hopefully soon MFA will handle this as part of their build scripts.
As an aside, I originally released my app in English, even for the other locales, when Zune marketplace was updated they disallowed this and demanded localized texts. Having very limited resources I had to rely on Automatic translation and a bit of common sense to accomplish this but - something unexpected happened - previously everyone seemed happy, even the French - with an unlocalized English UI, but when I released the "machine" localized one I heard a lot of complaints about poor localization!!! Its true, the machine localization is not great, but I realized then that not localizing at all is better than half localizing. Hmmm... must save up for professional localization...
The ugliest hack I have ever made.
One of my Android devices is an Optimus S, it's a very low end, small screen, Android 2.2 device and I use it as my "min-bar". My game would run on this device but then randomly crash with a stack
ending in AndroidGameView relating to setTime (as gleamed form the logs in DDMS).
OpenTK.Platform.Android is a component written by Xamarin in the OpenTK project, unfortunately they havent released the source so I couldnt just "fix" it - Reflector to the rescue! I disassembled AndroidGameView.RunIteration I could see that they determine the update and draw times by comparing to DateTime.Now values. If, however, the system, in between Draws or Updates adjusted the system clock (which it does quite often on my Optimus S for some reason) these times could come out negative and "boom". Unfortunately these values were stored in private fields. My "hack" - wait for it - every 1/60s I analyse these private fields using reflection and check them against DateTime, if they go negative I "poke" a value into them which is (DateTime - 1/30 sec). Yuck. But it seems to work, my Optimus can run for much longer now.
Apparently AndroidGameWindow is getting a complete re-write in an upcoming version of MFA. I will definately be upgrading to that so I pull out this heinous hack.
Sounds not working on Android
It was only at this point that I realized I was probably the first to really do a game for the Android using monogame, since sound simply did not work. There was a bug in the code that accessed the
resources for the MediaPlayer, I fixed the bug and sent it to CartBlanche. I had a problem with the use of MediaPlayer for SoundResources so I rewrote the Android monogame sound to use SoundPool, this has a problem in that it will not play long sounds but in my case this was not required. The SoundPool implementation also made it back into the monogame code base.
MFA?/VS?/ADB? - memory usage in Visual Studio while debugging
One of the problems that slowed me down the most was a nasty bug (x64 related perhaps?) apparently either in VS or MFA (or Android SDK? ADB? ...????) which causes memory usage for VS to rapidly cycle up and down between 700MB and 2GB, this thrashes my machine and brings me to a stop. I have to keep rebooting my machine, this doesnt always happen but when it does it's a complete blocker.
You can tell by all the question marks that I dont know where this bug really lies but I do see that memory profile going crazy so something is tweeked.
The last bug (that I know of...)
My game has 16 levels - each taking a minute to complete, with the other screens in between it can easily take 20 minutes to play my game end to end. The game was looking really good but after 10 or so levels there was a slightly perceptible delay between hitting the screen and the touch being registered. I let my game play itself for an hour and the delay became progressively worse, up to the point of making the game unplayable. This was a tough one to track down and took me the best part of 3 days. To make a long story short to track this down I had to start commenting out chunks of my app, running the autoplayer for long periods of time and seeing if the problem went away. An unsatifyingly primitive technique that consumed wads of time.
Eventually I pulled the AdMob component (yes, I should have tried that first, it's obvious in retrospect) - ran the game all night and in the morning... no lag! Gotcha!
I tried updating the AdMob component to the latest one to see if they had fixed the issue but because of the issues described above I could not use the later component. I now tear down (dispose) the ad component each level and new it up again, this seems to get rid of the bug.
Time to release!
Releasing your Android game
Xamarin has good instructions on how to post to the Android marketplace so I'll not reproduce that here. I also posted to the Amazon marketplace that is currently running a free to upload service (usually $99/yr) but my app is stuck in the review process as I write this - probably because it is the holiday season.
Both the Amazon and Android marketplaces allow you to point to a YouTube video as part of your promotion so you might want to think about doing that. Here's mine.
Tools Used
Visual Studio 2010 (Express is free)
Mono for Android ($400) - cross platform implementation of the .NET api
Monogame (free!) - cross platform implementation of the XNA api (in progress)
Beyond Compare ($30) - Superb diffing tool! nice integration with VS/SVNMonogame (free!) - cross platform implementation of the XNA api (in progress)
TortoiseSVN (free!)
AnkhSVN (free!)
VisualSVNServer (free!) - Server side component simplifies making respositories on a server
SpriteFont (free!) - create the spritefont sheets for the fonts in my game
Reflector - invaluable, it's the swiss army knife of .NET assemblies
Eclipse (free!) - Java development environment
CLR profiler
ReSharper
CryptoObfuscator - I could not use Dotfuscator for a non Windows Phone project, this seems to work
Thanks!
Update:
I have been asked for the code I wrote to convert .RESX files to Android compatible string resources for Mono For Android. I suspect (hope?) MFA already does this conversion by now but in case this is of any use to anyone I have Zipped up my code here. You may have to massage it some.
12 comments:
Really interesting stuff.. especially nice to know about your AdMob implementation and the tips for JNI.
Thankfully, I haven't noticed the Visual Studio issue yet. I'm running a fairly powerful Win7-64 bit laptop with VS 2010 Pro. But I'll keep a lookout for it. I have noticed that occasionally on load while debugging, my game will just hang. If I press pause in the debugger it can't find the disassembly where it breaks into the code... but then on continue, it loads without problem. So, if this happens.. I just quickly press pause then continue in the VS debugger and get past the hang.
Also, I've yet to look at my Android logs at all. I'm a little worried about what I might find. I know the accelerometer memory leak issue is something I'm putting off dealing with... hoping it'll be fixed by MFA sooner rather than having to face fixing it myself.
I really appreciate you paving the way. I'm probably about a month or so behind you.
Thanks for the great article providing your insights.
I am also currently porting my game to Android, running into the same issues. Still unfixed for me is the setTime crash, which also only appears on my low end test device. Can you perhaps tell in more detail or provide code in how your fix works?
Also AdMob is giving me a headache as it always returns no ads even though it runs in testmode. When I run your game on the other hand I see ads...
John, thanks! The logging bit is very important so I added that to the blog too.
Tim, sure I can share my "solution" to the setTime problem. Where would you like me to send it?
Tim, are you sure you are adding the adView to the container View with .addView()? Can you check your logs to see if the ad component is actually fetching ads but you are just not making them visible?
Yes, the AdMob log says:
I/AdMobSDK( 305): Making ad request in test mode
I/AdMobSDK( 305): No fill. Server replied that no ads are available (358ms)
On the AdMob SDK page they write, that you need the new SDK for all apps created after October 14, 2011, otherwise it wont return ads. Did you register your app before that date?
Perhaps you can attach your setTime crash work around to the Monodroid bug report you entered at Xamarin. Thanks!
Excellent pieces. Keep posting such kind of information on your blog. I really impressed by your blog.
Excellent article, one of the most useful I've found that goes beyond the total beginners guide and gives real practical advice.
How I've missed DDMS till now I don't know, but I'm pleased you pointed it out!
Hi Warren,
I don't suppose you could post up your source code for the post build step where you spoof the *.Designer.cs files?
I'm in the same situation, it would save me reinventing the wheel. Thanks!
Hi Beringela, with regard to your request for the string conversion code I have updated the blog at the end with a link to a ZIP file containing it. Hope it saves you some time.
Cheers - Warren
Hey Warren, if you need a localization tool to collaboratively translate .resx files, you should have a look at the localization management platform https://poeditor.com/ it has a great translation interface.
Post a Comment