For the past couple of months, I've been working on a much-delayed update to my Android application Scribblify. The original Android version was released back in early 2013 and hadn't been updated on Google Play since then. Although it paled in comparison to the iOS counterpart, the first Android version still received positive reviews and was downloaded 200,000 times. Having released a major 4.0 update to the iOS version earlier this year, it was time to bring the Android version up-to-speed.
One of the greater challenges experienced with Android development occurs when it comes time to test. I previously wrote about the increasingly fragmented iOS platform, but even with a handful of different iOS devices sporting different resolutions and technical specs, nothing quite compares to Android device dispersion. In my updated Android app, for instance, I support all Android devices from API Level 15 (Android 4.0.3) up. According to Google Play, this makes the app compatible with 9,098 Google-certified devices. This list encompasses at least four years of devices including phones, tablets, phablets and more. Understandably, with such a large range of devices comes an equally large range of technical specs.
So, when it comes time to test an Android app, the more devices available to experiment with the merrier. It is particularly worth testing against the multitude of different device resolutions and DPI specifications to ensure that all assets scale appropriately, as well as testing against various OS versions to ensure nothing breaks. Google provides the Android Virtual Device (AVD) Manager, available through Eclipse, Android Studio and as a standalone tool. AVD Manager allows you to create custom-tailored virtual devices spanning the entire core spectrum of Android units, provided you've installed the necessary packages via SDK Manager. However, Android emulation via AVD is traditionally sluggish and will not always behave as expected. There are also more dedicated and responsive Android emulators for testing including Bluestacks, which I wrote a tutorial about previously (other popular Android emulators include Andy, Xamarin, and YouWave).
Testing on Physical Devices using SAMSUNG Remote Test Lab
For more assured Android app testing than software emulation can reasonably provide, it is always favorable to test on physical devices where available. Basic Google-certified tablets with the latest Android OS can now be purchased for as little as $30-$75 at many retailers and work wonderfully for testing. Even some of the more high-end phones and tablets can be affordable when bought used online at sites like eBay. As long as the device is Google-certified and supports Google Play, it is ripe for app testing.
Still, there is another avenue for development testing against an assortment of popular and new Android-powered products–without having to deplete your life's savings while amassing a mountain of test devices. Cloud-based device testing is a growing way to test your app on physical Android units around the world. Popular players in this field include Keynote by Dynatrace, Sauce Labs and Remote TestKit. These and other related services are generally subscription-based but offer free trial periods.
Sometimes though, as the saying goes, the best things in life are free. One free cloud-based testing service that I've grown to love is the SAMSUNG Remote Test Lab. This completely free service, provided by SAMSUNG, allows you to interface with a couple dozen different phones and tablets of diverse specifications.
Getting Started
To get started, you will need to register a free account in the SAMSUNG Developer Portal. After registering and verifying your email address, you can make your way over to the Remote Test Lab landing page. The testing application has a few minimal technical requirements including JavaScript, Java Web Start and a browser from this decade. If any of the configuration is missing the page will alert you.
The SAMSUNG Remote Test Lab offers free app testing on a couple dozen Android devices.
From this page, you can browse through the various devices or narrow down the results by category using the tabs across the top. Most of the test devices are available in multiple different OS versions,making them ideal for testing against OS changes that may affect particular features of an app. The range is substantial, from the now archaic 2.3.3 through 5.1.1 at the time of this writing. When you have arrived at an attractive device that you wish to test against, select the appropriate parameters from the drop-down lists (OS Version, Device List [Location], Reserve Time).
The free testing system works using Credits. Every day that you login to the SAMSUNG developer portal, you will receive 20 additional credits towards testing. These credits accrue so if you sign-in three different days you will accumulate 60 credits. Each credit is good for 15 minutes of physical device testing, and you have to commit to 30 minute minimum sessions (you can also cancel device reservations early to receive unused credits back). In any case you are not likely to run out of testing time too quickly, considering each day of logging in buys you another five hours of testing time.
When you are ready to launch the test device, just hit the Start button.
Running the Hardware Simulator
After starting a device simulator from the SAMSUNG Test Lab, a Java applet will launch. Depending on your browser and OS configuration, you may be prompted a couple of times to open and run the script (say Open / Yes / Run where appropriate). If you are currently not signed in you will still be able to do a 5-minute Pre-Test, which is a good idea to ensure the selected device is functional and usable on the simulator.
When the device first launches, hit OK after reviewing the terms and then choose the device's language (a great way to test translations, but it is generally not changeable until you start a new session). Basically, SAMSUNG asks that you clean up any of your test files and data from the device before exiting so that they do not get cluttered with test content or interfere with other users. The devices are not reset after each session so it is relatively common to find remnants of past testers when connected, including their own signed-in accounts at times.
The test device after initially launching it.
From here out, you can interact with the device's screen using your mouse to simulate touch. Some devices may support alternative touch simulators such as the use of a stylus. In my test cases, I did come across a couple devices that would not respond to mouse control due to various causes including a prior tester changing the unlock method (another reason why trying them without being signed-in first is a fine idea).
Uploading and Installing your App
The SAMSUNG simulator software uses right-click context menus to configure its setup and manage applications.
- Right-click on the device's screen to adjust screen properties including device orientation, and to capture screenshots and video.
- Right-click on the device's frame (chrome) to access test-related controls including app management and logs.
To easily upload, install and run your app to the device, do as follows:
- Right-click on the device's frame (i.e., on the SAMSUNG logo in the screenshot above).
- From the context menu, select Manage > Application to load the Application Manager.
- From the Application Manager view, click New and then double-click on your desired APK to upload it.
- With your uploaded app selected in the Application Manager list, click Install.
- Once installed, make sure the test device is unlocked and that your app is selected in Application Manager, then click Start.
- You can alternatively browse to the application directly on the device, once installed, to launch it.
If all was successful, your app should then launch and you will be able to interact with it as if using a real device. If the device does not have working WiFi connectivity, simply right-click on the frame and select Manage > Reset WiFi. You can switch the orientation to landscape by right-clicking the screen and selecting Orientation > Landscape.
The SAMSUNG device with my development app (Scribblify) installed.
Advanced Tools
The test device environment supports device logging akin to LogCat in Eclipse / ADB. To view the logs, you can right-click the device frame and select Test > Logs. All of the logs will then appear in a new window and can be filtered by log level and/or string. You can also clear the log or save the data to a file just as you could from Eclipse. In addition, the SAMSUNG devices support remote debugging by selecting Test > Remote Debug Bridge and then connecting to it from your local ADB shell (this is the same manner described in my BlueStacks tutorial).
When you have completed your tests, you can uninstall your app(s) from the Application Manager by selecting it and clicking Uninstall. To exit the emulator, right-click the frame and select Exit. Finally, it is a polite gesture to then cancel the reservation in the SAMSUNG Test Lab site (under "My Reservations") so that the device can be used by other parties before the entire duration is up.
This free service is tailored to SAMSUNG devices and does not have nearly the number of available devices as some of the paid services mentioned previously. Even so, it offers a wide-enough spectrum of test devices to rather adequately test most of the problem areas of device. Another very promising and free cloud testing service on the horizon is Google's Cloud Test Lab, which promises to offer virtual access to many devices as well as advanced automated tests. This service is not yet available publicly but is certainly worth keeping tabs on.