Testing is an integral part of any software development. In today’s post, we will discuss how to add a Xamarin UITest project to an existing Xamarin app solution, the basics of writing UITests and finish off by automating our tests in app center or azure devOps.
Add Xamarin UI test project
We will use Page-Object Pattern (POP) architecture when implementing our tests as it has proven to be simpler and more scalable when writing Xamarin UITests. The basics of adding a UITest project to an existing Xamarin solution are outlined below but you can refer to the getting started with Xamarin UITest for a more detailed instructions.
- Right-Click on solution > Add > New Project
- Select UI Test App
- Add Xamarin.UITest.POP nuget to use the page object pattern. Alternatively, you can manually add the following files to your project as explained in page object pattern.
- Delete the AppInitializer. App manager class will replace this.
- Add two new folders, Pages and Tests
Platform specific Setup
You also need to add reference to your Xamarin.ios and Xamarin android projects
|Add internet permissions in AndroidManifest.xml||Add the Xamarin.TestCloud.Agent nuget.|
|Ensure the Use shared mono Runtime is disabled
Ensure you have set the ANDROID_HOME and JAVA_HOME system environment variables
|Add the ENABLE_TEST_CLOUD code snippet below in the AppDelegate class.
The compiler symbol ENABLE_TEST_CLOUD was added to our iOS debug build configuration and not the release mode as this will cause the app to be rejected by apple.
|For more info, please refer to this post on how to prepare Android for test Automation||For more info, please refer to this post on how to prepare ios for test automation|
Xamarin UITest Setup
Not much is required to setup the Xamarin UITest project besides customizing the binary/artifacts location. You can update these location to the default location of the binaries which is outlined below:
- Apk file location – the default location when running the emulator on release mode is “../../../Decobile.Android/bin/Release/com.decobile.decobileMobile-Signed.apk”;
- App file location – Default location when running the simulator on debug is “../../../Decobile.iOS/bin/iPhoneSimulator/Debug/device-builds/iphone11-13.4/Decobile.iOS”;
However, the problem with this approach is that the default location is not constant especially for iOS that is dependent on the simulator type and build configuration. One solution around this issue is to add a reference to the Xamarin.iOS and Xamarin.Android project as described in the getting started with Xamarin UITest.
Thereafter, all we need is to give the tests a reference to app bundle Id as shown below.
In this way, if we change code in the main project e.g add a new automation id, all we need is to run the app in the simulator/emulator, then stop the debugging (otherwise an exception will occur when you attempt to run the tests while the app is still debugging), and run the tests. The goal is to install the latest version of the app in the simulator/emulator and when we run the tests, the app with the given bundle id will be launched.
However, if you are simply writing the tests and have no access to the main project e.g for automation testers, then a better outcome would be to configure a constant binaries file path. This will make it faster to write and run the tests as the app does not have to rebuild each time. But as earlier stated, iOS path is not constant so how can we overcome this issue? We create a new folder in the root directory and either manually copy these binaries to that folder or better yet use custom scripts that runs after a build to move the binaries to the folder.
For more info, on binaries location see the Xamarin UITest blog
The first step in writing tests is to decorate your xaml with automation ids where applicable. Automation Id is a Xamarin forms element property used to uniquely identify each element. For instance, in a login page, you might want to add the automation Id to the username, password and login button as these elements are mostly likely to be referred to from the UITests project.
This is a folder where all the test fixture will live. Create a tests.cs class and inherit from BaseTestFixture. Your test fixtures should look like this:
Ensure page object classes inherit from BasePage:
This is a handy tool when writing uitests as it allows us to inspect the app’s elements. To use it, simply add the app.repl command in our test class as shown below.
Running the tests will start the repl tool and you can run various commands as described in this AppQuery class
There are two ways to run the test automation i.e. via app center or azure devops. From app center, we will need to define our test run first. This involves setting up the device configuration and test series. Thereafter, we can opt to run the tests whenever the master branch builds. This can be achieved by adding a post build script.
Alternatively, we can use azure devops pipeline. The main advantage of using azure devops pipeline over app center is that we can configure a build configuration that is not dependent on a branch. In the previous step for example, we configured the test to run on master branch but this can be expensive as device build hours are billable. If we instead opted to run the tests on a release branch, this would be better as the tests would not run as often but the downfall is that we have to replicate the release configuration each time e.g. on release branch 1.0.0, release 1.0.1, release 1.0.2 and so on.
This problem is automatically solved in azure devops as we only need to create one release pipeline that will run the tests in app center. If you have not setup any build configuration in azure devops you might to set that up first. Once we have build and packaged our artifacts in azure devops, all we need to do now is to add the App Center Test Task.
App Center Test Task
You can follow the Azure DevOps for UITesting for details on integrating app center test task to existing pipelines. Key things to note:
- Binary application file path:$(agent.builddirectory)/**/*.ipa
- Build directory: Decombile.UITest/bin/$(BuildConfiguration)/
- App centre connection service: Needs user token not app token
- App slug:
- The .ipa file does not seem to be linked with calabash framework – Ensure your build configuration is set to debug is you added the ENABLE_TEST_CLOUD symbol.
- Error: Cannot find test-cloud.exe – Ensure your Xamarin.uitest is building and nuget is restores. You can confirm this by looking at the Xamarin.ios logs and search for the Xamarin.UITest installation. Thereafter you can provide the exact file path i.e $(build.sourcesdirectory)/packages/Xamarin.UITest.3.0.7/tools
- Error: failed to load list of organizations – If you are using app center connection service, ensure the api token used is of user type not app type as described in app center api documentation.
- Android SDK not found. Please install it and if it is still not located, please set the ANDROID_HOME environment variable to point to the directory. This exception might occur if debugging on Xamarin UITests on a physical device. Ensure you have the ANDROID_HOME setup in Environment variables as shown here
- System.Exception : Java Development Kit (JDK) not found. Please make sure that it is installed and if it’s still not located, please set the JAVA_HOME environment variable to point to the directory.
Microsoft Xamarin video link