Photo by Daniele Levis Pelusi on Unsplash
Once reviewed the remote configuration and the native setup, we’ll finish this series by checking the CI/CD pipeline configuration required for test automation. So let’s go!
Create Firebase Project Configure project by adding an account service Enable Cloud Tools Result Update underlying native Android project Configure underlying native Android Test Runner
- Set up Codemagic pipeline
6. Set up Codemagic pipeline
Codemagic from Nevercode is a CI/CD tool designed initially for Flutter. Nowadays though, it offers support for almost any native or cross-platform mobile framework as well.
Codemagic automates the process of building, testing and delivering apps. Furthermore, all the build operations take place on their servers, so we don’t need any infrastructure on our side.
Once we register on their site, we can easily link the source code repositories we want to integrate into our pipeline. Futhermore, this tool offers both free and paid solutions, so it’s suitable for a large range of projects.
On Codemagic, build workflows can be configured in 2 ways:
- locally, by editing the “codemagic.yaml” file inside our project
- remotely, interacting directly with the editor available on the web
Overview of “codemagic.yaml”
For this project, we’ll use the “codemagic.yaml” file to customize our build mechanism.
When this file is added to our project repository, it gets detected by default, so every time we launch a build then configuration is fetched from it automatically.
The following screenshot shows the complete structure of our configuration file:
Structure of “codemagic.yaml”
The root component of the file is the “workflows” section. A single file can include many flows (for instance, one for Android and another one for iOS), and each one of them can have its own settings.
Specific workflow settings are defined using different headings or subsections, such as:
- general props: sets both name and tools versioning.
- environment: declares secrets or keys required for the build.
- scripts: specifies the step-by-step build process.
- artifacts: lists all the components generated after building (.ipa, .apk)
- publishing: states the channels used to distribute our app.
This block is mainly used for:
- give the workflow a descriptive name
- specify the type of machine we want to use when building
- set the build duration time
Any credentials, API keys or secrets can be defined here, either as encryped data or plain text (not recommended). Once declared, these values can be used on any section of the file.
Aditionally, variables can be gathered into groups, so we can manage them as a whole. As a result, when importing a group, we’ll get immediate access to all of its values.
For the sample project, we have defined:
- a group named “google_credentials”, holding variables for the GCloud configuration file
- a single variable called “FIREBASE_PROJECT”, with the unique identifier of our Firebase site
This section is the core of our build pipeline, since it contains the commands we want to execute in order to build our app.
For instance, if we need to update the project dependencies, we can add the following instruction:
name: Get packages script: | cd . && flutter packages pub get
When defining the build, order is important, since the script blocks are executed sequentially, one after another.
In order to run our integration tests, we must add a script to build a test apk (apart from the standard apk):
name: Create both debug and test APK... script: | set -ex cd android ./gradlew app:assembleAndroidTest ./gradlew app:assembleDebug -Ptarget="$FCI_BUILD_DIR/integration_test/app_test.dart"
After that, we have to include another script so the generated apk is uploaded to Firebase Remote Test Lab:
name: Upload to Firebase... script: | set -ex echo $GCLOUD_KEY_FILE | base64 --decode > ./gcloud_key_file.json gcloud auth activate-service-account --key-file=gcloud_key_file.json gcloud --quiet config set project $FIREBASE_PROJECT gcloud firebase test android run \ --type instrumentation \ --app build/app/outputs/apk/debug/app-debug.apk \ --test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk \ --timeout 5
As shown in the snippet, here we’re using the variables/groups and the apk’s generated on previous step:
- we retrieve the variable $GCLOUD_KEY_FILE, decode its contents and save the result to a local file. When building, this file will be used to activate the service account
- we retrieve the $FIREBASE_PROJECT to set the project identifier
- we specify the apk and the test apk used when running our tests
Nothing fancy here, since it only contains the directory for the output files created after building.
Codemagic supports different options when it comes down to distributing the build results.
Apart from email notifications, it also offers webhook compatibility, so it can be connected to messaging apps such as Slack or Discord.
Bonus: robo and instrumentation tests
By slightly changing the build script, we can execute the different types of tests on Firebase Remote Test Lab.
Running instrumentation tests
gcloud firebase test android run \ --type instrumentation \ --app build/app/outputs/apk/debug/app-debug.apk \ --test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk \ --timeout 5m
Running robo tests
gcloud firebase test android run \ --type robo \ --app build/app/outputs/apk/debug/app-debug.apk \ --timeout 5m
The gcloud firebase test command defines optional properties that allow us to add custom tailored customization.
For instance, we can declare the devices we want to use when running our tests by adding:
gcloud firebase test android run \ --type instrumentation \ --device=model=NexusLowRes \ --...
A complete reference for this command and all its properties can be found here.
On this series, we have defined an automation workflow using free tools such as Firebase and Codemagic. As you can see, is not as complicated as it may seemed at the beginning.
Moreover, automation has a lot of advantages over manual testing, so I think is “the way” to go. Give it a try and feel free to share your thoughts and results!
As usual, you can check out the following repository for the source code:
Write you next time!