A quality mobile app can enhance brand image and significantly increase revenue

According to Statista, between January 2015 and July 2020, application downloads have grown rapidly. In the last 5 years, registered downloads increased from 15.9 billion to 37.8 billion between both app stores. This growth isn’t slowing down any time soon it seems. A new Sensor Tower report revealed that consumer spending on mobile apps and app installs grew significantly during the first half of 2020, reaching $50 billion globally.

However, that doesn’t mean that all mobile applications downloaded by users end up being successful. For that and other reasons, it’s very important to test mobile applications in order to raise their level of quality and enhance the user experience.

Why is it Important to Test Your Mobile App?

• An app with minimal issues has a greater chance at success, meaning more downloads and positive reviews, thus more revenue.
• Testing reduces project risk since developing an app can be quite expensive and if it doesn’t meet the users’ expectations, it can lead to great financial losses.
• A quality app with a good UX/UI helps to enhance the brand image and positions the company as more trustworthy and reliable.
• Testing will help to check the app’s compatibility with different devices and operating systems such as iOS and Android.
• An investment in testing can cut down costs in the future: Detecting bugs in the earlier stages is best as that is when they’re easier and cheaper to fix. In contrast, if a user reports a bug, the cost to fix it at that point will be significantly higher and the app’s reputation will suffer.

The Different Types of Mobile Applications 

Mobile applications are not as simple to test as desktop web applications as they’re more complex and varied. There are three types of mobile apps: native, hybrid and web. Let’s go over each of these!

Native Applications

A native application will run on the device faster and be more advanced in terms of functionality than a mobile web app, yet it must be downloaded before use. 

Since native apps are platform-specific, they must be developed using specific IDEs (integrated development environments) and languages. Android uses languages ​​like Java or Kotlin and Android Studio or Eclipse as the IDE. Apple uses languages ​​such as Objective-C or Swift and XCode as the IDE.

Some examples of the most widely used native applications are Google Maps, LinkedIn, Twitter, Telegram, PokemonGo, etc.

Web Applications

A web application can be accessed through any mobile browser; this means you can use it without having to download anything. Just open the internet browser on your smartphone and navigate to the specific URL. Like web applications, mobile web applications are generally built with JavaScript, CSS, and HTML5.

Some examples of popular web applications are nytimes.com, medium.com, and yelp.com

Hybrid Applications

Hybrid apps combine different features of native and web applications. They’re usually developed as a mobile web application using JavaScript, CSS and HTML5. Packaged in a native environment, the same code can be used for different platforms. Like native apps, hybrid apps must be downloaded from an app store. 

Some examples of hybrid apps are Facebook, Instagram, Uber, and Slack.

What to Consider Before Testing a Mobile App?

Software testers and/or developers must take into account the following considerations before testing a mobile application:

• App Type: Whether it’s a native, hybrid or web app, knowing this information will help define what types of tests can be performed.
• OS Versions: It should be clear which versions of iOS and Android should be covered in the test strategy. 
• Devices: Regardless of the type of app, you should choose a set of real mobile devices to test with. Since having a set of only real devices can be very expensive, it’s recommended to expand device coverage by running tests with simulators and emulators as well. 

What to Know About Mobile Device Simulators and Emulators 

Simulators and emulators are limited in terms of what you can test. Therefore, you can’t use them to test the holistic user experience, but they are great tools for expanding test coverage.

A simulator like the one provided by Xcode simulates or imitates devices with an iOS operating system (like an iPhone or iPad). On the other hand, an emulator – referring to what Android Studio offers – acts as the configured device (running the Android operating system), emulating its operating system and hardware.

Another alternative is to use a cloud testing tool, although they’re paid. Some of the best options are Sauce Labs and BrowserStack. These tools offer the ability to test apps through their real device cloud services, mobile emulators/simulators, and live application tests.

How Do You Choose Which Devices to Test On? 

This BrowserStack article includes lists that combine insights from the usage statistics of over 2 million developers on BrowserStack and the latest global market trends.

One of the basic packages that BrowserStack offers, “Starting Up,” includes the following devices:

• Apple iPhone 8 (Cellular) iOS 13.0
• Apple iPhone XR (Cellular) iOS 12.0
• Google Pixel 3 (Cellular) Android 9.0
• Samsung Galaxy S9 Plus (Cellular) Android 8.0
• Samsung Galaxy S8 (Cellular) Android 7.0
• Apple iPad 6th (Tablet) iOS 11.0

With this set of devices alone, you can achieve the following market coverage:

Where Can You Download the Mobile Application You Want to Test?

Testflight

Only available for iOS, Testflight is an application on the App Store that you must download and install on the test device. With an invitation from the developer, you can download the version to test.

Visual Studio App Center

Visual Studio App Center is a platform where you can upload and download applications. Its tools are designed for Android and iOS applications built on Swift, Objective-C, Java, React Native, UWP, and Xamarin. 

Cloud Storage (for Android)

APKs (Android Package Kits) aren’t limited regarding where they can be published, so you can use a cloud storage service such as Google Drive, One Drive or Dropbox.

How Can You Capture Evidence of Bugs?

If you’ve only used Android before, you may not be sure of how to take a screenshot on an iPhone, or vice versa. 

For Android, the screenshot functionality is a native one and all you have to do is press the power and lower volume buttons simultaneously but, this combination may vary depending on the manufacturer. 

If you want to go further than that and take a screen recording, this functionality isn’t native to Android. Some brands like Samsung, Xiaomi, and Huawei include it in their phones. Check out the many ways to do a screen recording on Android here. 

If the test Android device doesn’t have this functionality, check out the following applications: Screen Recorder, Vysor and HiSuite (HUAWEI).

On iOS, the screenshot functionality is native and depends on the iPhone model; but you can take a screenshot in either of these ways in the image shown below:

The screen recording functionality is also native to iOS since the release of iOS 11 and is carried out as follows:

Types of Software Tests for a Mobile Application

Different apps will require several different kind of tests, but every tester should first take into consideration these ones when getting started: 

Functional Testing

These are the tests to focus on first since they ensure that the mobile application works as indicated in the requirements and/or user stories. Some of the functionalities to be tested are:

Sign Up / Login

• Verify that the user can register and log in
• Verify any integrations with applications such as Facebook, Twitter, Snapchat or Google

Vertical (Portrait) and Horizontal (Landscape) Mode

• Check if the app has the landscape option activated
• Does the app adjust quickly and appropriately to changes in orientation?

Basic Actions, Menu Options, Titles, Texts and Buttons

• Ensure that all buttons, links and other elements of the User Interface (UI) work as expected
• Review the integrations with other applications such as payment applications (PayPal, Stripe, etc.)
• Test the forms, verify that each entry is validated (mandatory fields vs. optional fields)

Notifications, Error/Success Messages

• Check that push notifications are processed correctly

Basic Gestures

• Check how the application behaves by performing some basic gestures on the screen
• A helpful resource: Luke Wroblewski, Product Manager at Google, created the Illustrated Reference Guide with the different gestures that can be performed on touch screens

Common Bugs Found in Functional Testing

• Integration with social networks in the Sign Up and Login functionalities: When testing this integration, it sometimes happens that after signing up or logging in through a social media account, the user doesn’t return to the app, but instead remains on the social media site. It’s recommended to test these scenarios with the social media apps installed and uninstalled. 

• Enabling the option to rotate screen: When testing this functionality, the tester may find that the content doesn’t adapt to landscape mode, since it’s not configured.

• Performing a basic gesture on the screen: When trying to perform a specific gesture, the app breaks. This can happen when the gesture isn’t configured properly.

Usability Testing

This type of testing focuses on ensuring that the end user has a good experience with the application. Usability testing ensures that the mobile app ends up being built in an intuitive and easy-to-use way. It’s advisable to incorporate users from the moment that the mockup or interface design is released and get their feedback on it. 

Also, it’s a good idea to incorporate user acceptance tests (UAT) into the testing process. It’s vital to obtain feedback from users who do not know the application before going to production. Our role as testers is to help coordinate, prepare test scripts and guide users through UAT Tests.

This valuable feedback will help the business make decisions such as whether or not to launch the app to market already or make more changes beforehand. 

Connectivity Testing

Web applications don’t work offline and the same rule applies to mobile web applications. In this case, see if they work well against different internet connection speeds.

When it comes to native mobile apps and some hybrid mobile apps, it’s necessary to check if they still work properly in offline or airplane mode as well as how they work with 3G/4G/5G or Wi-Fi.

The scenarios that can be tested during app connectivity testing include: 

• How it behaves connected to a mobile network 
• How it behaves connected to a Wi-Fi network
• How it behaves connected to a mobile network and a Wi-Fi network
• How it behaves when the connection is lost
• How it behaves when the signal is weak

Performance Testing 

Performance tests help ensure that the mobile application performs as expected under different and specific workloads. The performance of a mobile application is considered a critical factor in its success.

Nobody wants to wait too long and many users uninstall an application if it starts up slowly or if it takes too long to load the content.

These are some areas to pay attention to:

• The amount of time in seconds from when the app is opened until the first screen is displayed (shouldn’t exceed 2 seconds)
• Check the battery consumption: how much battery did the device have before and after using the app
• Check if the mobile device is overheating
• Review memory usage

The scenarios listed above should be tested with a physical mobile test device. 

Fortunately, you don’t have to be an experienced performance tester to obtain this information about an app.

There’s a free tool, Apptim, that captures performance data for you automatically as you explore the app.

Below, you can see what kind of performance metrics you can obtain from an Apptim report which is automatically generated after you finish performing functional testing with the tool. Click here to see a sample mobile test report from Apptim.

Common issues reported in performance tests:

• The app takes a long time to load
• The first time opening the app, it takes a long time to load the first screen
• The app crashed

Interruption Testing

This type of testing ensures that the mobile application works as expected when faced with abrupt and unexpected events. There are many possibilities for an interruption, for example, when a user receives a call or push notification while using the app. 

Some of the scenarios that can be tested are:

• What happens when the user receives a call or an SMS? What happens when they make a call or send an SMS? Does the mobile app continue to work or does it crash?
• What happens after the user turns off their device or it shuts down unexpectedly, does the mobile app open properly afterward?
• How does the app behave when receiving an alert or notification?
• What happens to the application when the user interacts with other apps at the same time?

Compatibility Testing 

For this type of testing, it’s recommended to create a combined strategy between physical or real devices in order to get the right coverage.

You must define which versions of the operating system the mobile application supports and test the following:

• How it behaves in different versions of the operating system
• How it works on different mobile devices
• That the app looks good and runs smoothly on different resolutions and screen sizes

Regardless of the type of application, every mobile app’s design must be responsive. Like desktop web apps, mobile web apps must be tested across multiple browsers. Some of the most popular mobile browsers are Safari, Chrome, Firefox, Edge, Opera, and Samsung.

Common incidents that are reported in compatibility tests:

• The app is not responsive: its design does not adapt correctly to the size of smaller or larger screens
• The content of the web app isn’t displayed correctly: The fields of the page are not configured in a certain browser and operating system

Download, Install and Update Testing

Even the mobile app with the best concept in the world will be useless if users cannot download, install or update it effortlessly. Therefore, remember to verify the following:

• Can the application be downloaded?
• How does the mobile application behave during installation, uninstallation and reinstallation? Are there any interruptions?
• Can the mobile application be updated? Is the stored data consistent after the update?
• Can the app be updated when there are multiple updates available? How does it behave when other mobile apps are also updating?
• Does the mobile app work properly after updating the device’s operating system? It’s vital to verify that an update to the operating system doesn’t break the application.

Localization Testing

Localization testing ensures that the mobile application works as expected in different markets, countries and regions. Some of the scenarios that we can test are:

• Verify that the translations are correct and that they’re not automatically generated translations, since users will recognize this immediately and won’t enjoy using the app as much.
• Verify that the application shows the time correctly in different time zones.
• Confirm that the texts and elements of the interface look fine and work in different languages ​​without any problems.

Bugs commonly reported in localization tests:

• Automatic translations deliver the wrong message (In some cases, an automatically generated translation could even be inadvertently offensive!) 

Lessons Learned in Mobile Application Testing

From my experience as a mobile tester at Abstracta, here are some key takeaways to sum up all of the above:

• The type of mobile application and predicted OS versions it will be used on will dictate the tests that you plan to run
• You should always have at least one physical or real mobile device to test it on, complemented with simulators or emulators and tools in the cloud. A combined strategy of using both real and virtual test devices will be a practical way to achieve the ideal level of test coverage.
• Don’t skip usability testing. It’s critical to find a group of users who can evaluate the mobile application before going to market; the tester’s opinion alone won’t suffice.
• Always consider different connectivity scenarios in testing such as airplane mode, Wi-Fi and different networks like 3G, 4G, etc.
• It’s not necessary to be a performance engineering specialist to warn the development team of irregular behavior of the app on a mobile device thanks to mobile performance testing tools like Apptim.
• Always check how the app will behave after an update. It’s vital to make sure if the correct version was installed.
• Remember that testing a desktop application is not the same as testing a mobile app. We have to consider several more scenarios when carrying out software testing for a mobile application.

---

There you have it, an introduction on how to test a mobile app! If you need more help to get started or determine the right strategy, Abstracta’s test engineers are here to help! Contact us today.

This article is translated from its original version (in Spanish) published on the Abstracta Chile blog.

---

Recommended for You

5 iPad Gestures Testers Should Know
9 Tips for Successful User Testing (and Some GIFS)