Streamline Your Flutter CI/CD: Integration Testing with Firebase Test Lab on GitHub Actions (Part 1)

Amarjit Mallick June 1st, 2024

Putting it Together: How Integration Testing Works

flutter create my_app

import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      title: "Firebase Test Lab Sample",
      home: LoginPage(),
    );
  }
}

class LoginPage extends StatefulWidget {
  const LoginPage({super.key});

  @override
  State<LoginPage> createState() => _LoginPageState();
}

class _LoginPageState extends State<LoginPage> {
  final _formKey = GlobalKey<FormState>();
  final _emailController = TextEditingController();
  final _passwordController = TextEditingController();

  @override
  void dispose() {
    _emailController.dispose();
    _passwordController.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Login'),
      ),
      body: Padding(
        padding: const EdgeInsets.all(16.0),
        child: Form(
          key: _formKey,
          child: Column(
            children: [
              TextFormField(
                key: const Key('email_field'),
                controller: _emailController,
                decoration: const InputDecoration(
                  labelText: 'Email',
                ),
                validator: (value) {
                  if (value!.isEmpty) {
                    return 'Please enter your email address';
                  } else if (!RegExp(
                          r"^[a-zA-Z0-9.a-zA-Z0-9.!#$%&'*+-/=?^_`{|}~]+@[a-zA-Z0-9]+\.[a-zA-Z]+")
                      .hasMatch(value)) {
                    return 'Please enter a valid email address';
                  }
                  return null;
                },
              ),
              const SizedBox(height: 16.0),
              TextFormField(
                key: const Key('password_field'),
                controller: _passwordController,
                decoration: const InputDecoration(
                  labelText: 'Password',
                ),
                obscureText: true,
                validator: (value) {
                  if (value!.isEmpty) {
                    return 'Please enter your password';
                  } else if (!RegExp(r"^(?=.*[A-Za-z])(?=.*\d)[A-Za-z\d]{8,}$")
                      .hasMatch(value)) {
                    return 'Password must be at least 8 characters long and contain at least one letter and one number';
                  }
                  return null;
                },
              ),
              const SizedBox(height: 16.0),
              ElevatedButton(
                onPressed: () async {
                  if (_formKey.currentState!.validate()) {
                    Navigator.push(
                      context,
                      MaterialPageRoute(
                        builder: (BuildContext context) {
                          return const HomePage();
                        },
                      ),
                    );
                  }
                },
                child: const Text('Login'),
              ),
            ],
          ),
        ),
      ),
    );
  }
}

class HomePage extends StatelessWidget {
  const HomePage({super.key});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(),
      body: const Center(
        child: Text(
          key: Key('Home Page'),
          "Home Page",
        ),
      ),
    );
  }
}

Set up Integration Testing Dependencies

• Open the pubspec.yaml file.
• Under the dev_dependencies section of the pubspec.yaml file, add the integration_test package and save the file.

dev_dependencies:
  flutter_test:
    sdk: flutter
  integration_test:
    sdk: flutter

• Open your terminal and navigate to your project directory (if you’re not already there) and run the following command to install the newly added dependencies:

flutter pub get

Write the Integration Tests

• Inside your project’s root directory, create a new directory named integration_test. This directory will hold your integration test files.
• Inside the integration_test directory, create a new Dart file (e.g., integration_test.dart). This file will contain your integration test code.
• Inside the integration_test.dart file we will write our integration test code like this:

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

import 'package:my_app/main.dart';
import 'package:integration_test/integration_test.dart';

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  testWidgets('Login and navigate to home page', (WidgetTester tester) async {
    // Build the app and trigger a frame.
    await tester.pumpWidget(const MyApp());

    // Enter the email and password.
    await tester.enterText(
        find.byKey(const Key('email_field')), '[email protected]');
    await tester.enterText(
        find.byKey(const Key('password_field')), 'Password123');

    // Tap the login button.
    await tester.tap(find.byType(ElevatedButton));
    // Wait for the home page to load.
    await tester.pumpAndSettle();

    // Verify that the home page is displayed.
    expect(find.text('Home Page'), findsOneWidget);
  });
}

Preparing to run tests from an APK file

To check if the app has the same behavior across multiple devices and configurations, we’re going to use Firebase Test Lab for. Also, to automate the process of running the tests whenever a new commit is pushed to a git repository, we’re going to use GitHub Actions.

In order to run the tests purely from an APK file, we need to do a little bit of a setup for the native Android project of the Flutter app. In the native Android world, there are instrumentation tests and we are basically going to hook up the multiplatform Flutter integration tests to run as if they were native instrumentation tests on Android.

• Inside android/app/src create two new folders androidTest/java. Then create nested folders with names according to YOUR project’s package name. You will be able to find the package name inside the MainActivity.java or MainActivity.kt file. In our case the package name is com.example.my_app .
• Now we will create MainActivityTest.java file. The path of the file will look like androidTest/java/com/example/my_app/MainActivityTest.java .
• Inside this MainActivityTest.java file we will write some code as follows:

package com.example.my_app; // Change the package name according to your package name

import androidx.test.rule.ActivityTestRule;
import dev.flutter.plugins.integration_test.FlutterTestRunner;
import org.junit.Rule;
import org.junit.runner.RunWith;
import com.example.my_app.MainActivity; // The MainActivity should be imported from your package

@RunWith(FlutterTestRunner.class)
public class MainActivityTest {
 @Rule
 public ActivityTestRule<MainActivity> rule = new ActivityTestRule<>(MainActivity.class, true, false);
}

Note:

• Update the package name
• Import the MainActivity from the same package

After this your project structure should look like this:

• Now open the app level build.gradle file (android/app/build.gradle) and update the code as follows:

plugins {
    id "com.android.application"
    id "kotlin-android"
    id "dev.flutter.flutter-gradle-plugin"
}

def localProperties = new Properties()
def localPropertiesFile = rootProject.file("local.properties")
if (localPropertiesFile.exists()) {
    localPropertiesFile.withReader("UTF-8") { reader ->
        localProperties.load(reader)
    }
}

def flutterVersionCode = localProperties.getProperty("flutter.versionCode")
if (flutterVersionCode == null) {
    flutterVersionCode = "1"
}

def flutterVersionName = localProperties.getProperty("flutter.versionName")
if (flutterVersionName == null) {
    flutterVersionName = "1.0"
}

android {
    namespace = "com.example.my_app"
    compileSdk = 34
    ndkVersion = "26.1.10909125"

    compileOptions {
        sourceCompatibility = JavaVersion.VERSION_1_8
        targetCompatibility = JavaVersion.VERSION_1_8
    }

    defaultConfig {
        applicationId = "com.example.my_app"
        minSdk = 21
        targetSdk = 34
        versionCode = flutterVersionCode.toInteger()
        versionName = flutterVersionName
        // Add these lines
        multiDexEnabled true
        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" 
    }

    buildTypes {
        release {
            signingConfig = signingConfigs.debug
        }
    }
}

flutter {
    source = "../.."
}

dependencies {
     // Add these dependencies
     implementation 'androidx.multidex:multidex:2.0.1'
     androidTestImplementation "androidx.test:runner:1.2.0"
     testImplementation 'junit:junit:4.13.2'
     androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
     androidTestImplementation 'androidx.test:rules:1.2.0'
     androidTestImplementation 'com.android.support.test:rules:1.0.2'
}

Build the apk files

• Open the terminal in your project directory and run the following commands:

flutter clean
flutter pub get
pushd android
flutter build apk
./gradlew app:assembleAndroidTest
./gradlew app:assembleDebug -Ptarget=integration_test/integration_test.dart
popd

Note:

If you have your integration testing code in a file other thanintegration_test.dart, then in./gradlew app:assembleDebug -Ptarget=integration_test/integration_test.dartin this command after-Ptartget=provide your integration test file path

Once the command is run successfully you will get two apk files (inside the build folder of your project directory) at build/app/outputs/apk/debug/app-debug.apk and build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk paths respectively.

Let’s Setup Firebase Testlab

• If you already have a firebase project, you can use that or you can create a new project.
  [ Note: Running tests on Firebase Test Lab doesn’t require integrating Firebase functionalities within your Flutter app. You only need a Firebase project to get started. Test Lab operates independently of whether you use other Firebase services. ]
• Once your project setup is done, inside the firebase console go to the All Products section on the left panel and then select Test Lab from the options.

• If you open the test lab for the first time you may see something like this (Otherwise you can skip this step):
  

• now here we have to run a Robo test first. Click on the Browse button and select thebuild/app/outputs/apk/debug/app-debug.apkfile from the file picker and start the robo test. You can wait till the test is finished, or you can also cancel the robo test.

Run Instrumentation Test

• Now if you open the Test Lab you will see the previous test(s). Click on “Run a test” button and select “Instrumentation”. Then you will be asked to upload the two apk files that we have generated earlier.

• Click on the Browse button for the App APK or AAB and select the build/app/outputs/apk/debug/app-debug.apk file and then click on the Browse button for the Test APK and select the build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk file.When the files are uploaded successfully, click on Continue.
• If you haven’t selected any devices, you will be asked to select the devices you want to run your tests on. Click on Customize and select the devices and then Confirm the selection.

• Click on “Start 1 test”.
  

The test will be started. If the test is fails for some reason you will see the status of the test as failed. Otherwise if the test is successful, you will see the test details and report as below:

• Click on “Full run video” and you will be able see the options to view Test Issues, Logs, Videos, Performance metrics, Test Cases.

Congratulations!!! you have run your Flutter Integration Test successfully on Firebase Test Lab

Summary

This blog post teaches you how to run integration tests on your Flutter app using Firebase Test Lab. Integration testing helps catch bugs between different parts of your app. Firebase Test Lab offers a wider range of devices, automatic testing, and a controlled network environment compared to local testing.

In the Second Part of this blog post, we will learn how to automate this whole process using GitHub Actions.

If you found this blog helpful, let me know in the comments below! Your feedback is always appreciated.

Want to learn more about integration testing in Flutter? This blog post just scratches the surface. Feel free to ask questions or suggest future topics in the comments!

To know more about topics like this, you can follow me on LinkedIn & X.

Thank You

Happy Coding