If you are testing a mobile app you are probably not sitting behind a desk with all your standard reporting tools and you don’t have all the test cases writing down. So how can you collect your mobile app test results?

Goal
My goal when testing is to give insight into where the defects are and make that as clear to all the stakeholders. By giving this information the possible risks can be evaluated. When testing I can’t explain all the steps I take on a mobile device? I could of course prepare all my test case during a test analysis period and during execution refer to these steps. I try to add print screens and movies to the test results to help a developer finding solutions for the found defects.

Less then ideal
The first problem I have is that the design and the software are developed at the same time (Agile/scrum team), so there is no separate preparation period. The second problem is that I usually can’t prepare every test case in detail because the number of situations and the relation between all the variables are far to complex to grasp in simple steps. How can I verify a defect on one device to all the other devices. Beside the platform and manufacturer differences there are many more factors that determine if there is a defect and/or the defect to manifest. Think about type of network and speed, location, movement, number of apps running, amount of free memory, who is executing the test, his or her emotional state. And there are many more.

One versus a group
If you are testing an app or part of the app on a group then the results need to be clustered and generalized. This is more a process a and quality item then a tooling or statistical issue. I would organize sessions with the group, slice the app and prepare tours with different perspectives. Look for more info about tours at Cem Kaner article. A tours focus on the tasks a user can perform with the app. Use a technique like equivalence class to spread the setup over the tours.

Setup
After a product risk analysis the chosen priority and area’s become clear. These results is translated by me into a setup. This setup is used during the (preparation) of the test cases/tours. An example of a setup is shown below. Every column in the table contains the chosen variation.

Devices	Users	Network type	Motion	Tours	Perspectives	Platforms	Sprints
iPhone 5s – 8.0.2	Product owner	offline	Binnen – zitten	App in its life time	User Scenario’s	iOS	1
iPhone 5s – 8.1	Interne tester	wifi	Binnen – lopen	Change your mind	Interruptions and interactions	Android	2
iPhone 6 – 8.1	Persona – Daan	3G	Buiten – lopen	Comparison	Function	Cross platform	3
iPhone 6p – 8.1	Persona – Lieke	4G	Buiten – rijden	Connectivity	Store submission		4
HTC One M7 – 4.4.3	Externe gebruiker	Multiple		Feature	Data		5
Nexus 5 – 5.0.1				FedEx or CRUD Tour	Network conditions		6
Sony Xperia Arc HD – 4.1.2				Guidebook or Competitor tour	Communication

Tooling
A developer needs all the context you can report to create a solution. Here is a list of the tools that I use to record the context of the defect:
– Product risk analysis
– A test setup
– (logical) Test cases on high level
– Multiple devices with SIM cards and a data bundle
– Multiple persons that execute tests
– Test data in all sort and flavors that can be created or searched for on the spot
– software to report, record and analyse like Xmind for test ideas and coverage, Mobizen and Reflector for mirroring, Shou screen recording, Postman plugin for API analysis, ADB and iTools for log files, Jira for defect reporting, Hip chat for informal communication.

Before I report a defect I need a reference situation to compare my situation to. Usually this is the functional perspective. From this point you start varying using the test setup, experience, hints for the team and your feeling.

Conclusion
Reporting the test results is more than using an application making movies. I like the combination of high level preparation and exploratory testing. I’m in direct contact with the developers and can create my own test data. I have a fysical test lab, test in side and out side. I use multiple devices and my laptop so I can repeat the scenario, analyse and compare the results.

Links
For note taking I’m still looking into iTester app but this is only available on iOS. More info about note taking and exploratory testing:

• http://thesocialtester.co.uk/explaining-exploratory-testing-relies-on-good-notes/
• http://university.utest.com/note-taking-tools-exploratory-testing/
• http://altom.ro/itester#SBTM
• http://ios.wonderhowto.com/how-to/record-your-ipads-iphones-screen-no-jailbreak-computer-required-0156864/ and https://emu4ios.net/
• http://kaner.com/?p=96

It’s not only handy to mirror the screen of your smartphone to a laptop or projector for demo but also needed for all different kind of mobile app testing like user test, reporting a defect or working with a distributed team. Streaming is mirroring the screen to another screen in real time, without any delay and preferred wireless. Handy to know is that mirroring is also called streaming, casting or sharing.

The most problems I have is with mirroring an Android device. There is not one solution for all Android devices. But the options that work best for me I will describe below. If you are looking for a solution for your self look at the list with choices below:

• Wireless or with a wire
• Only iOS or also Android
• Which Android manufacturer
• Rooted or un-rooted devices
• Freeware or payed solution
• Mirroring to a laptop, TV, projector or media player
• Realtime or some delay/lag is aloud
• Images or video/gaming

If you want a wireless connection the The device should be in the same WiFi as the laptop. More precise on the same access point. Otherwise the client (read smartphone) cannot find the server (read laptop). Also for wireless connection the port numbers should be open in the firewall and two way connection should be aloud.

If you want a wired connection then use a connection cable for iOS from Apple (lightning to VGA and HDMI) and a connection cable from (Micro usb naar VGA/HDMI kabels like http://www.slimportconnect.com/). Also make sure you have a projector with a HDMI port.

Android
The preferred option for me is to use a TV or monitor with a tuner, and a Google chrome cast dongle. With this setup the Nexus smartphone mirroring is wireless, sharp and real-time. If the Chromecast dongle isn’t working with the local WiFi then I would choose a cable between the smartphone and the HDMI cable to the projector.

If you look further the the following wireless setups are also working, but with a small presentation delay:

• VNC soltution with http://www.vmlite.com/ server on the smartphone and a viewer like TightVNC viewer on the laptop or
• Mobizen see https://mobizen.com/

There is a new solution from airserver with Miracast solution for Windows. This solution makes Windows laptop a receiver for any Mirocast supported device. I’m not able jet to get my Windows laptop configured correct. More info about airserver and Miracast, see http://www.airserver.com/.

For all Android solutions to be able to mirror your screen the ADB USB driver from Android Studio is needed. See

iPhone
For iOS the solution is not so complex as with Android. The best option is Airplay. This is build into the platform for mirroring an iPhone to an external display like an Apply TV, a Mac laptop. Airplay can be used form iPhone 4 (http://en.wikipedia.org/wiki/List_of_iOS_devices). The iPhone is the sender (client) and the TV or laptop is the receiver (server). On the laptop an air server should be running (http://www.airserver.com/).

Problem and an alternative
Most of the time the problem is that there is no wireless connection between the client (device) and the server (laptop). This can be because the needed port is not open or supported protocol is not supported. Another problem is that in many offices the total wireless network exits of many access points. So its possible that the client is connect to a different access point than the server.

So if the wireless access point is like Fort Nocks the alternative is the setup a personal hotspot with your smartphone and connect with your laptop. This is a costly option and your phone constant needs power and gets very hot.

Conclusion
The main message is there it’s not one solution for all devices, not even on a platform. Also try every option in the room your self. To see if the configuration works, the speed is okay and all the scenarios in the mobile app work. The advise is always have a back-up plan if a connection is not working at moment supreme.

Links

• http://www.droid-life.com/2014/07/09/cast-screen-button-now-works-for-screen-mirroring-to-chromecast/
• http://www.samsung.com/uk/consumer/mobile-devices/smartphones/smartphone-accessories/EAD-T10UDEGXEU
• http://www.airserver.com/ or http://www.airsquirrels.com/reflector/
• http://droid-at-screen.ribomation.com/
• http://www.mirrorop.com/
• https://mobizen.com/
• http://chromecast.wonderhowto.com/how-to/enable-chromecast-screen-mirroring-feature-0156160/
• http://www.vmlite.com/ en http://www.tightvnc.com/download.php