Get started with Flutter tests and GitHub workflows

How many times have you broken a part of your app while adding a new feature or fixing a bug?
I’m pretty sure we all have been there, just a small feature could take days of fixing and manual testing, which is pretty inefficient and a waste of developers time and effort.

Writing tests is an important part of the development cycle, it’s not something you do after you finish writing all your code, because by then you wouldn’t really find time for it.
Furthermore, even if you’re a beginner, you shouldn’t postpone learning tests until you gain enough experience, you should start as early as you can.

In this blog, we walk through a getting started guide to writing tests for Flutter and Dart apps, then make these tests run automatically on every push and Pull Request using GitHub workflows.

What you will learn:

1. Why is it important to write automated tests?
2. Types of tests in Flutter.
3. Test packages.
4. Write a simple test.
5. Setup GitHub Actions to run tests on every push and PR.

Automated Tests

There are lots of types of tests, but the one we’re interested in are automated tests, which is a code we write to test our app code.

One might wonder “why would I spend extra time and effort on writing a code that’s not part of production code?”, and to answer this question,
let’s see the benefits we get from writing tests:

1. Make sure that we haven’t broke a part of our app/package while adding a new feature or fixing a bug.
2. Automation can eliminate the manual effort spent on testing before each release.
3. Helps the collaborators or team members to work together more efficiently such that one PR is well-tested before getting merged which ensures
   you or team members or the outside collaborator haven’t broken existing code.
4. Focus on what matters instead of being frustrated by small bugs that would be caught by tests and wasting time finding them after each release.
5. To write tests, our code should be well-structured and testable, so when we code with tests in mind, we will definitely write a better code.
6. Deliver faster with more confidence and avoid lots of last-minute-changes headache.

Small example

To better understand automation in tests, let’s see an example.

double sum(List<double> numbers) {
  double total = 0.0;
  for(double number in numbers){
    total += number;
  }
  return total;
}

The sum function sums up a list of numbers, simple. Let’s add another function that does exactly the same but with division.

double division(List<double> numbers) {
  double total = 0.0;
  for(double number in numbers){
    total /= number;
  }
  return total;
}

Trying to print the division of [1, 1] will result in NaN. What’s wrong? We assume that such simple calculations wouldn’t result in errors, similar to small changes we make to fix
a small bug somewhere.

To fix this on a test driven way, let’s first write test cases before fixing the problem. Tests go under test folder in the root of any Dart or Flutter project.

test('1 by 1 result in 1', (){
  final result = division([1, 1]);

  expect(result, equals(1));
});

test('1 by zero result in infinity', (){
  final result = division([1, 0]);

  expect(result, equals(double.infinity));
});

Writing unit tests in Dart is fairly simple, package/test provides us with all necessary utilities.
In our example, we notice that the test case goes inside a callback that is passed to a top-level function called test, this function also
takes a first argument as a string describing the test case.

The next step is to write the actual test, in this case we use the same division function, pass a known input, and expect a known output.
expect is provided by the test package, it helps us match the actual result with the expected result, using a matcher such as equals().

Running the test will give the following output:

Expected: <1>
  Actual: <0.0>

package:test_api            expect
main.<fn>

✖ 1 by 1 result in 1

Expected: <Infinity>
  Actual: <NaN>

package:test_api            expect
main.<fn>

✖ 1 by zero result in infinity

The first test failed, we expected dividing 1 by 1 to result in 1, but the actual result was 0. The second also failed,
the expectation was Infinity, but it failed with NaN.

To break down why both failed, let’s focus on the very first line in division() function:

double total = 0.0

In the summation case this wouldn’t cause an issue, but if we start the division
with 0.0, then this will mess up the whole calculation:

1. total starts with 0.0.
2. on the first loop, 1.0 is divided by 0.0, which equals 0.0.
3. in the second loop, 0.0 is divided by 0.0, which results in NaN.

We can fix this by assigning the first number of the input list to total on the first loop.

double? division(List<double> numbers) {
  double? total;
  for(double number in numbers){
    if(total == null) {
      total = number;
      continue;
    }
    total /= number;
  }
  return total;
}

By running our tests again, we see that Dart is happy 😄.

✓ 1 by 1 result in 1
✓ 1 by zero result in infinity

Types of Tests in Flutter and Dart

There are 3 types of tests in Flutter applications:

1. Unit tests: test an individual unit such as a function, method or class.
2. Widget tests: test a single widget or UI element.
3. Integration tests: test the whole app or big parts of it.

To read more, please visit the official guide to testing in the Flutter documentation.

Test Packages

Writing tests in any framework or language requires some utilities and tools which is usually exposed via a testing toolkit or packages.
In Flutter’s world, there are 2 official packages for testing:

1. test
2. flutter_test

The first one is used when writing Dart only apps, it doesn’t include widget testing tools, therefore for Flutter apps, we use flutter_test which provides necessary
methods to mimic a user behavior such as pressing a button and builds the widget tree without running the app, in addition to all the methods in test.

Usually these packages alone aren’t enough, we still need to mock many other external dependencies. In most cases, creating a mock from the class we’re testing
is easily done using mockito package, which generates mocks for any class and helps run tests internally avoiding any call to external dependencies.

There are other packages for certain use cases. For instance, if you’re using http package to make http requests, it exposes a
testing API to mock all http requests and responses without making the actual request.

import 'package:http/testing.dart';

Write a Simple Test

For this example, we will use a sample app that shows a fun cat image based on the status code of the response to any given link.
Find the full code here.

To break it down, there’re 2 folders we will work in on this example:

• lib where all the application code lives.
• test where we write unit and widget tests for the application code.

If you navigate to lib, you’ll find in main.dart all the code for the sample app,
but in this blog we are interested in tests, so I advice you to go over the code before moving forward.

Unit testing http

To decide which cat image to show, we first need the status code for the link the user is trying to visit, to get it, we’re using the http package.

final _statusCode = await CatsAPI.instance.checkStatusCode(controller.text);

Let’s take a look into how the CatsAPI class looks like.

class CatsAPI {
  CatsAPI._();

  static CatsAPI instance = CatsAPI._();

  http.Client? _client;

  @visibleForTesting
  setClient(client) {
    _client = client;
  }

  Future<String> checkStatusCode(String link) async {
    try {
      final url = Uri.parse('https://' + link);
      final res =
          _client != null ? await _client!.post(url) : await http.get(url);
      return res.statusCode.toString();
    } catch (e) {
      log('$e');
      rethrow;
    }
  }
}

It has only one instance method, checkStatusCode() that takes a link and returns the status code. Additionally, we define a http.Client private property, that only has a setter
for testing purposes as the @visibleForTesting annotation indicates.

In the application flow, we won’t need to change the http Client used to make the real requests, default is sufficient, but for testing, we need to use a MockClient from http/testing,
because in a unit test, we don’t want to test the actual backend rather than testing our logic on different http results. Therefore, we use the setter to instantiate a CatsAPI instance
with the MockClient.

import 'package:flutter_test/flutter_test.dart';
import 'package:http/http.dart' as http;
import 'package:http/testing.dart' as http_test;

import 'package:fun_cat_responses/main.dart';

Future<http.Response> _mockResponse(http.Request request) async {
  switch (request.url.toString()) {
    case 'https://google.com':
      return http.Response('Success', 200);
    default:
      return http.Response('Error: not found', 404);
  }
}

void main() {
  setUp(() {
    CatsAPI.instance.setClient(http_test.MockClient(_mockResponse));
  });
}

Any test would start inside the main() method, where will have a number of methods to define the test flow. The first method we used is setUp(),
we can setup our initial instances and any other thing we need ready before prior to each unit test, for example, if something needs to be cleaned before each test,
it can be done here.

In our case, we set the http Client in the CatsAPI to be a http_test.MockClient, and we pass a callback. We can define all the cases and responses we want to test in this callback,
for instance, how would our logic act on a 404 response code?

Now everything is ready, the actual test for http will happen on 2 cases, success 200 and failure 404.

group('$CatsAPI', () {
  test('200 response', () async {
    final res = await CatsAPI.instance.checkStatusCode('google.com');

    expect(res, '200');
  });
  test('404 response', () async {
    final res = await CatsAPI.instance.checkStatusCode('none.com');

    expect(res, '404');
  });
});

First, we create a group(), a top-level function from Flutter’s test API, that can describe a group of related tests, in this case it’s describing that all tests inside it are
concerned with CatsAPI class.

Next, we test for 2 cases with 2 individual unit tests using test() function, calling checkStatusCode() and comparing the actual output with the status code we’re expecting.
Note that setUp() will be called before each test(), not group().

Widget testing

In widget testing, the idea is pretty similar to unit testing, except that we’re testing UI elements instead of pure logic.

For this example, we will test the Home widget. In the app, we expect a NetworkImage widget to show in the screen when the user
types in a link, this is how the test code will look like.

group('$Home', () {
  testWidgets('200 shows cat image', (WidgetTester tester) async {
    await mockNetworkImagesFor(() async {
      await tester.pumpWidget(MyApp());

      await tester.enterText(find.byType(TextField), 'google.com');

      await tester.pump();

      await tester.tap(find.byType(IconButton));
      await tester.pumpAndSettle();

      expect(find.byKey(ValueKey('cat')), findsOneWidget);
    });
  });
});

We start a new group() with the widget name, naming here is optional based on your own tests. Instead of test(), we see here testWidget(),
this method is available in flutter_test, and gives us a free WidgetTester to help mimic a user behavior, and build the widget tree,
all automatically without building the app.

Before we carry on the test steps, there’s an extra package used for the purpose of mocking Network images. As we already have seen in http testing,
we never actually made any real network call, the same will apply on images since in the app they are displayed by making a network call,
we need a way to mock that. Luckily there’s a community package called network_image_mock,
that provides a mock response code for network images in Flutter. To use it, we call mockNetworkImagesFor() async method, and inside it the test code goes.

Let’s break down the test steps:

1. Ask the WidgetTester to build our widget, in this case MyApp().
2. Type in the text field using enterText() method.
3. Once done typing, pump it into the tree.
4. Find a widget with type IconButton and tap on it.
5. Ask the tester to rebuild the widget, similar to calling setState().
6. We know that our image widget has a String key ‘cat’, so we expect to find it by using findOneWidget matcher.

Next steps

That’s it! there are yet much to discover in this code, like the types of matchers available, and what options we have to find widgets
other than type and key, but by reaching here, you learned about writing simple unit and widget tests.

As a next step, go through the testing codelab provided by Flutter.

GitHub Test Workflow

To make things even faster and easier, let’s automate running the test using GitHub Actions.
The goal is to get our test to run on every push and pull request.

Setup

To setup a workflow in GitHub, create a yaml file in the following path at the root of the repo: .github/workflows/test.yaml.
This path is fixed, if you change it, GitHub will not recognize the workflow.

Inside the file, let’s configure our test workflow.

1. Name the workflow:

name: analyze & test cat example

2.Setup triggers using on keyword:

on:
  pull_request:
  push:

3. Define the steps:

jobs:
  analyze:
    runs-on: macos-latest
    steps:
      - uses: actions/checkout@v2.3.4
      - uses: subosito/flutter-action@v1
        with:
          channel: 'stable'
      
      - name: "Analyze"
        run: flutter pub get && flutter analyze .

      - name: "Format"
        run: flutter format lib/** --set-exit-if-changed

      - name: "Run tests"
        run: flutter test -r expanded

We can have multiple jobs in one workflow, each running on a different virtual machine, for example you might want to have a job for testing
on macOS platform, and another on windows.

Each job runs a sequence of steps in order:

1. Checkout into the current version of the code.
2. Install Flutter with the stable channel.
3. Run pub get and analyze to check for any issues reported by the analyzer.
4. Run format to make sure the code is formatted correctly.
5. Run the tests.

Once you push this workflow to a Flutter repository, it will run the tests in your test folder against every new push or new opened pull request.

You can check the final run result from here.