Scott BurgholzerIntroduction
As I have been busy with my classes for my doctorate and work, I haven’t had much time to write articles like I had hoped. I was hoping to have a project done that I could discuss how I created a Web App using multiple AWS services for others to learn how they all can interact. As that project is still ongoing, I decided to focus on something else.
I choose AWS Device Farm because eventually I would need to test that web app on mobile devices and on desktop browsers. Device Farm allows us to use physical devices to test our apps. Not only can we test web apps, we can also test Android and iOS apps on physical devices. There is a lot to learn for Device Farm, so I started very basic with using it, and hence this article was born. I hope, while this article just uses a simple app and the defaults, it can expose more people to this AWS service and see if it’ll work in their use cases.
View this article as a “I’ve never heard or seen AWS Device Farm, so I want to start from the basics and work my way up” as that is exactly what I’m doing with you!
What is AWS Device Farm?
AWS Device farm is a cloud-based testing platform that allows developers to test their mobile and web apps on physical devices instead of simulators. There are two main ways of using Device Farm: automated testing of apps and remote access of devices [1].
Automated app testing
AWS Device Farm allows developers the ability to upload their own tests or to use built-in and script-free compatibility. AWS Device Farm performs testing in parallel, tests on multiple devices begin in minutes. As these tests are completed, a report is generated that contains high-level results, low-level logs, pixel-to-pixel screenshots and performance data. Device Farm supports the testing of native and hybrid Android and iOS Apps. Some frameworks that work with Device Farm are PhoneGap, Titanium, Xamarin, Unity, and others. You also get remote access of Android and iOS apps for interactive testing.
Remote access interaction
This allows you to swipe, gesture and interact with a device through your web browser in real time. An example of where this is beneficial is a customer service rep can guide customers through the use or setup of their device. You have the ability to install apps on a device running in a remote access session and reproduce bugs or other issues. While the session is live, Device Farm will collect details about the actions that are completed on the device. Logs with those details and a video capture of the session are available at the end of the session [1].
Note about regions: Per the documentation, Device Farm is only available in the us-west-2 (Oregon) region [1].
Walk-through
I’m going to skip over the setting up section. If you wish to go through it in detail (signing up for an account, create or use an IAM user) you may read that here. Below is an example IAM policy [1] to allow full access to Device Farm. For this walk-through, this is fine, but if you are going to implement this service, you are going to want to restrict it as much as you can to the least privilege access.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"devicefarm:*"
],
"Resource": [
"*"
]
}
]
}
Device Farm Console
Step 1: Create a project
- Go to Device Farm in the console
- Expand the left hand navigation if necessary
- Click on Projects underneath Mobile Device Testing
- Click on Create mobile project
- Enter a project name and assign any tags you wish the project to have
- Click Create
Step 2: Create and start a run
You will need an app in order to start the run. I have created a very simple Android App on my GitHub. Go ahead and download that if you do not have your own Android or Apple App (note in order to get an .ipa file for iOS you will need to have an Apple Developer Account.
Stay on the Automated tests page and click on Create Run
Device Farm uses the app name as the default run name, feel free to choose your own name! Click Next
You’ll be taken to a configure page to setup your test framework. You have the ability to choose one of the testing frameworks provided or built-in test suites. Since we are not worrying about our own tests for this example, we can use the Built-In: Fuzz for our case (can keep all the defaults). When you start writing your own test packages, you can choose the corresponding testing framework and upload the file with your tests. This is not applicable in our example. Click Next
For our example, we will use the Top Devices Device pool. You do have the ability to create your own device pools, but for our simple app, Top Devices is just fine. Click Next
We are going to keep the next page as is. You can read more about the options in the Developer Guide, which you can find at the end of this article tagged [1]. Click Next
We will leave the execution timeout to the default value provided and will then click Confirm and start run. This can take a while, so wait patiently.
Run Results
Step 3: View the run’s results
— When the run is done, you will see the results of the runs. In my case 3 tests were ran and all passed.
— The unique problems would list any problems that were discovered. A video can be downloaded to view the recording of the test that had the problem. There are additional data provided that I won’t get into detail in this article.
— Screenshots will display any screenshots Device Farm took during the run.
Where to go from here?
The next thing I personally would suggest, and plan on doing, is to learn more about test packages. There are frameworks that work with both iOS and Android, and some that only work for only Android or only iOS. This is where you are able to specify your tests the way you want them instead of using just the built-in tests AWS provides. This gives you complete control over how you want the tests done and what you want tested. This is where the real power of AWS Device Farm comes into play.
The other thing to look into is the Desktop Browser Testing for any web apps. Personally for me, my project is going to start as just a web app, so Desktop Browser Testing is of personal importance. As always, you need to figure out your use cases and determine what your needs are.
There is so much more to learn about AWS Device Farm, and I will continue posting about it as I go through it and learn it myself. For now, happy developing and testing!
