Assertions in Dart and Flutter tests: an ultimate cheat sheet

Tests are essential for ensuring any software quality. Whether you are creating unit, widget, or integration tests for Flutter applications, the end goal of any test is asserting that the reality matches the expectations. Here is an ultimate cheat sheet for assertions in Dart and Flutter tests with many details explained!

Cheat sheet

The previous post covered a significant part of checks you can perform in Flutter and Dart tests and explained more than half of this cheat sheet in detail. Here we focus on the bottom half, starting with asynchronous matches.

Before you start, you can read all tests in the following Zapp.run project.

Asynchronous expect

The expectLater() function is just like expect(), but returns a Future that completes when the matcher has finished matching.

Future<void> expectLater(
    dynamic actual,       // actual value to be verified
    dynamic matcher, {    // characterises the expected result
    String? reason,       // added to the output in case of failure
    dynamic skip,         // true or a String with the reason to skip
}) {...}

While expectLater() can accept any matcher, it makes sense to pass children of AsyncMatcher class, which does asynchronous computation.

Future matchers

There are a few matchers to test Future execution results.

The completes matcher completes successfully when the Future completes successfully with any value.

test('expectLater: completes ✅', () async {
  final result = Future.value(0);
  await expectLater(result, completes);
});

The completion matcher accepts the matcher to verify the Future result:

test('expectLater: completion ✅', () async {
  final result = Future.value(0);
  await expectLater(result, completion(isZero));
});

And the throwsA matcher should be already familiar:

test('expectLater: throwsA ✅', () async {
  final result = Future.error(Exception());
  await expectLater(result, throwsA(isException));
});

Stream matchers

First, we’ll focus on testing streams with hardcoded values to see the variety of stream matchers. And then, we’ll have a word about testing streams when it’s too late to use the expect() function.

emits / neverEmits

The emits matcher checks that the Stream has emitted a value that satisfies a matcher, that emits has been accepted as a parameter. It may accept the expected value, another matcher that characterizes the expected value, or a predicate function:

test('expect: emits ✅', () {
  final stream = Stream.fromIterable([0]);
  expect(stream, emits(0));
  expect(stream, emits(isZero));
  expect(stream, emits((value) => value == 0));
  expect(stream, emits(predicate<int>((value) => value == 0)));
});

The neverEmits matcher performs the opposite check:

test('expect: neverEmits ✅', () {
  final stream = Stream.fromIterable([1]);
  expect(stream, neverEmits(0));
  expect(stream, neverEmits(isZero));
  expect(stream, neverEmits((value) => value == 0));
  expect(stream, neverEmits(predicate<int>((value) => value == 0)));
});

emitsInOrder / emitsInAnyOrder

These matchers ensure a stream has emitted multiple events.

In particular order:

test('expect: emitsInOrder ✅', () {
  final stream = Stream.fromIterable([0, 1]);
  expect(stream, emitsInOrder([isZero, 1]));
});

Or in no particular order:

test('expect: emitsInAnyOrder ✅', () {
  final stream = Stream.fromIterable([Result(0), Result(1)]);
  expect(stream, emitsInAnyOrder([hasValue(1), Result(0)]));
});

As you see, both accept an array, containing expected values or matchers.

emitsDone

The emitsDone matcher helps ensure a stream does not emit any more unexpected values:

test('expect: emitsDone ✅', () {
  final stream = Stream.empty();
  expect(stream, emitsDone);
});

test('expect: emitsDone ✅', () {
  final stream = Stream.value(0);
  expect(stream, emitsInOrder([0, emitsDone]));
});

emitsError

The emitsError matcher helps ensure a stream has emitted an error and accepts another matcher to verify the exact error:

test('expect: emitsError ✅', () {
  final stream = Stream.error(UnimplementedError());
  expect(stream, emitsError(isUnimplementedError));
});

Testing closed / drained streams

So far we tested streams that contained hardcoded values, which were emitted immediately inside the expect() function. But imagine, we have to test a stream that was already closed or a stream that has already emitted values we are interested in.

Let’s take a look at this class:

class StreamExample {
  final _streamController = StreamController<int>.broadcast();

  void doWork() {
    _streamController.add(0);
    _streamController.add(1);
  }

  Stream<int> get stream => _streamController.stream;
}

When doWork method is called, the stream should emit two values: 0 and 1. Here is a test that comes to mind for this behavior:

test('expect: drained stream ❌', () async {
  final streamExample = StreamExample();
  streamExample.doWork();
  expect(streamExample.stream, emitsInOrder([0, 1]));
});

Unfortunately, the expect() function is called too late, emitted values are already gone, and this test never completes. Instead, expect() or expectLater() should be used before doWork() call.

Unlike using expectLater() with Future matchers, where it is placed at the end of the test and is awaited, for testing StreamMatcher, it should be placed before performing the calls that affect stream values. This way, we can catch values as the stream emits them. In such a case, the expectLater() call should not be awaited, otherwise, the test will not be complete as well.

test('expectLater: drained stream ✅', () async {
  final streamExample = StreamExample();
  expectLater(streamExample.stream, emitsInOrder([0, 1]));
  streamExample.doWork();
});

Flutter widgets matchers

Flutter widget tests use the same expect() method to verify the actual widget tree content matches expectations. However, the list of matchers is unique to this task.

No / One / N widgets matchers

This group of matchers is the most commonly used in Flutter widget tests.

findsNothing ensures no widget in the widget tree matches the first parameter of the expect() method:

testWidgets('widget test: findsNothing ✅', (tester) async {
  final content = Text('1');
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  expect(find.text('0'), findsNothing);
});

The findsWidgets/findsOneWidget matchers ensure at least/exactly one widget is present in the tree:

testWidgets('widget test: findsWidgets ✅', (tester) async {
  final content = Column(children: [Text('1'), Text('1')]);
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  expect(find.text('1'), findsWidgets);
});

testWidgets('widget test: findsOneWidget ❌', (tester) async {
  final content = Column(children: [Text('1'), Text('1')]);
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  expect(find.text('1'), findsOneWidget);
});

In the example above, findsOneWidget matcher failed the test because the widget tree contains two widgets with the text '1'. The test output is:

Expected: exactly one matching node in the widget tree
  Actual: _TextFinder:<2 widgets with text "1" (ignoring offstage widgets): [
        Text("1", dependencies:[DefaultSelectionStyle, DefaultTextStyle, MediaQuery]), 
        Text("1", dependencies:[DefaultSelectionStyle, DefaultTextStyle, MediaQuery])]>
   Which: is too many

The findsAtLeastNWidgets/findsNWidgets matchers ensure at least/exactly N widgets are present in the widgets tree.

testWidgets('widget test: findsAtLeastNWidgets ✅', (tester) async {
  final content = Column(children: [Text('1'), Text('1')]);
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  expect(find.text('1'), findsAtLeastNWidgets(2));
});

testWidgets('widget test: findsNWidgets ✅', (tester) async {
  final content = Column(children: [Text('1'), Text('1')]);
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  expect(find.text('1'), findsNWidgets(2));
});

Color matcher

isSameColorAs helps verify properties of Color type of any widget:

testWidgets('widget test: isSameColorAs ✅', (tester) async {
  final content = Container(color: Colors.green);
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  expect(
    tester.widget<Container>(find.byType(Container)).color,
    isSameColorAs(Colors.green),
  );
});

testWidgets('widget test: isSameColorAs ✅', (tester) async {
  final content = Text('1', style: TextStyle(color: Colors.green));
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  expect(
    tester.widget<Text>(find.text('1')).style!.color,
    isSameColorAs(Colors.green),
  );
});

Parent matchers

isInCard/isNotInCard helps to verify that widget has at least one/no Card widget ancestor:

testWidgets('widget test: isInCard ✅', (tester) async {
  final content = Card(child: Text('1'));
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  expect(find.text('1'), isInCard);
});

testWidgets('widget test: isNotInCard ✅', (tester) async {
  final content = Text('1');
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  expect(find.text('1'), isNotInCard);
});

Error matchers

We all know that the Container widget cannot accept both color and decoration parameters because of the following assert in its constructor:

assert(color == null || decoration == null,
   'Cannot provide both a color and a decoration\n'
   'To provide both, use "decoration: BoxDecoration(color: color)".',
 ),

If you take a similar approach when implementing your widgets, the throwsAssertionError matcher can help with testing these asserts:

testWidgets('widget test: throwsAssertionError ✅', (tester) async {
  final builder = () => Container(color: Colors.red, decoration: BoxDecoration(color: Colors.red));
  expect(() => builder(), throwsAssertionError);
});

Because throwsAssertionError uses throwsA matcher under the hood, the same principle applies here: the function that is expected to throw should be called under the expect() method call.

The throwsFlutterError matcher verifies that a function throws FlutterError. Here is an example from the Flutter framework tests:

testWidgets('widget test: throwsFlutterError ✅', (tester) async {
  final testKey = GlobalKey<NavigatorState>();
  await tester.pumpWidget(SizedBox(key: testKey));
  expect(() => Navigator.of(testKey.currentContext!), throwsFlutterError);
});

Accessibility matchers

There is only one pair of accessibility matchers: meetsGuideline/doesNotMeetGuideline. These matchers are asynchronous, and thus should be used with the expectLater assertion function mentioned before.

They accept AccessibilityGuideline object, which represents the type of performed accessibility check. There are several predefined guidelines to check against:

androidTapTargetGuideline checks that tappable nodes have a minimum size of 48 by 48 pixels;
iOSTapTargetGuideline checks that tappable nodes have a minimum size of 44 by 44 pixels;
textContrastGuideline provides guidance for text contrast requirements specified by WCAG;
labeledTapTargetGuideline enforces that all nodes with a tap or long press action also have a label.

Your own guidelines can be created by inheriting AccessibilityGuideline class or creating your own instances of MinimumTapTargetGuideline, MinimumTextContrastGuideline, LabeledTapTargetGuideline.

Let’s take a look at this example:

testWidgets('meetsGuideline: iOSTapTargetGuideline ✅', (tester) async {
  final handle = tester.ensureSemantics();
  final content = SizedBox.square(
    dimension: 46.0,
    child: GestureDetector(onTap: () {}),
  );
  await tester.pumpWidget(MaterialApp(home: Center(child: content)));
  await expectLater(tester, meetsGuideline(iOSTapTargetGuideline));
  handle.dispose();
});

The GestureDetector will have a size of 46×46, which is just enough to satisfy the iOSTapTargetGuideline which requires 44×44 tap area. A similar test that uses androidTapTargetGuideline, which requires a 48×48 tap area, fails:

testWidgets('meetsGuideline: androidTapTargetGuideline ❌', (tester) async {
  final handle = tester.ensureSemantics();
  final content = SizedBox.square(
    dimension: 46.0,
    child: GestureDetector(onTap: () {}),
  );
  await tester.pumpWidget(MaterialApp(home: Center(child: content)));
  await expectLater(tester, meetsGuideline(androidTapTargetGuideline));
  handle.dispose();
});

with the following output:

Expected: Tappable objects should be at least Size(48.0, 48.0)
  Actual: <Instance of 'WidgetTester'>
   Which: SemanticsNode(Rect.fromLTRB(377.0, 277.0, 423.0, 323.0), actions: [tap]): 
     expected tap target size of at least Size(48.0, 48.0), but found Size(46.0, 46.0)

Let’s check an example using textContrastGuideline:

testWidgets('meetsGuideline: textContrastGuideline ✅', (tester) async {
  final handle = tester.ensureSemantics();
  final content = Container(
    color: Colors.white,
    child: Text(
      'Text contrast test', 
      style: TextStyle(color: Colors.black),
    ),
  );
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  await expectLater(tester, meetsGuideline(textContrastGuideline));
  handle.dispose();
});

Black text on white background has a good contrast ratio, so this test passes. However, small orange text on a white background is hard to read, and this test fails:

testWidgets('meetsGuideline: textContrastGuideline ❌', (tester) async {
  final handle = tester.ensureSemantics();
  final content = Container(
    color: Colors.white,
    child: Text(
      'Text contrast test', 
      style: TextStyle(color: Colors.deepOrange),
    ),
  );
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  await expectLater(tester, meetsGuideline(textContrastGuideline));
  handle.dispose();
});

with the following output:

Expected: Text contrast should follow WCAG guidelines
  Actual: <Instance of 'WidgetTester'>
   Which: SemanticsNode(Rect.fromLTRB(0.0, 0.0, 14.0, 14.0), label: "Text contrast test", textDirection: ltr):
     Expected contrast ratio of at least 4.5 but found 3.03 for a font size of 14.0.

As you see, the output mentions that the expected contrast ratio depends on the font size. In the example above, when no text style was provided in Text widget, nor in MaterialApp, the default text size of 14 was applied. Interestingly enough, if the text font is increased, the same test passes because larger texts are easier to read even when the contrast ratio is not perfect:

testWidgets('meetsGuideline: textContrastGuideline ✅', (tester) async {
  final handle = tester.ensureSemantics();
  final content = Container(
    color: Colors.white,
    child: Text(
      'Text contrast test', 
      style: TextStyle(
        fontSize: 20, 
        color: Colors.deepOrange,
      ),
    ),
  );
  await tester.pumpWidget(MaterialApp(home: Scaffold(body: content)));
  await expectLater(tester, meetsGuideline(textContrastGuideline));
  handle.dispose();
});

Golden matchers

Image file matcher

The matchesGoldenFile matcher allows validating that a Finder, Future<ui.Image>, or ui.Image matches the reference (golden) image file:

testWidgets('expectLater: matchesGoldenFile ✅', (tester) async {
  final widget = Container(
    width: 200, height: 200,
    padding: EdgeInsets.all(20),
    color: Colors.white,
    child: ColoredBox(color: Colors.blue),
  );
  await tester.pumpWidget(widget);
  await expectLater(find.byType(Container), matchesGoldenFile('golden_test.png'));
});

Image object matcher

The matchesReferenceImage matcher allows validating that a Finder, Future<ui.Image>, or ui.Image matches the reference ui.Image object:

import 'dart:ui' as ui;

testWidgets('expectLater: matchesReferenceImage ✅', (tester) async {
  final key = UniqueKey();
  final widget = Container(
      width: 200, height: 200,
      padding: EdgeInsets.all(20),
      child: ColoredBox(color: Colors.green),
    );
  final image = await createTestImage();
  await tester.pumpWidget(
    Center(
      child: RepaintBoundary(
        key: key,
        child: widget,
      ),
    ),
  );
  await expectLater(find.byKey(key), matchesReferenceImage(image));
});

Future<ui.Image> createTestImage() {
  final paint = ui.Paint()
    ..style = ui.PaintingStyle.fill
    ..color = Colors.green;
  final recorder = ui.PictureRecorder();
  final pictureCanvas = ui.Canvas(recorder);
  pictureCanvas.drawRect(Rect.fromLTWH(20, 20, 160, 160), paint);
  final picture = recorder.endRecording();
  return picture.toImage(200, 200);
}

For testing a Finder, as in the test above, it must match exactly one widget and the rendered image of the first RepaintBoundary ancestor of the widget is treated as the image for the widget.

Mock invocations and parameters

So far, we have used expect() function to compare the results of some operations with the expected values. There is another category of checks developers constantly perform: ensuring that some side effect was triggered.

Verify invocations

For that, the verify() function is used:

test('verify ❌', () {
  final mock = MockService();
  verify(() => mock.sideEffect(0));
});

This verify() function comes from the mocktail package. The mockito package provides similar functionality.

For context, here is the definition of MockService:

abstract class Service { 
  void sideEffect(int value, {Result? result});
}

class MockService extends Mock implements Service {}

The verify() function ensures that provided method was called at least once. The test above fails because the sideEffect() method was never called, with the following output:

No matching calls (actually, no calls at all).

To ensure certain method was called exactly N times, use the called() method of verify() function result:

test('verify ❌', () {
  final mock = MockService();
  verify(() => mock.sideEffect(0)).called(1);
});

To ensure certain method was never called, use the verifyNever() function:

test('verifyNever ✅', () {
  final mock = MockService();
  verifyNever(() => mock.sideEffect(0));
});

Verify parameters

In the example above, the sideEffect() method accepts two parameters. In case their values do not matter, use the any() object that can represent literally any value:

test('verify: any ✅', () {
  final mock = MockService()..sideEffect(0, result: Result(0));
  verify(() => mock.sideEffect(any(), result: any(named: 'result')));
});

In case of invocation parameters matter, real values can be used instead of any():

test('verify: any ✅', () {
  final mock = MockService()..sideEffect(0, result: Result(0));
  verify(() => mock.sideEffect(0, result: Result(0)));
});

Similar to equals matcher, verify will compare parameters using the equality operator. Thus, for the above test to pass, the Result class has to have the operator == overridden, as was shown before. Otherwise, the test fails with the following output:

No matching calls. 
All calls: MockService.sideEffect(0, {result: Result{value: 0}})

In case you are only interested in checking those parameters met some condition, any() call can accept one of the matchers we looked so closely at before as that parameter:

test('verify: any that ✅', () {
  final mock = MockService()..sideEffect(0, result: Result(0));
  verify(() => mock.sideEffect(
      any(that: isZero),
      result: any(named: 'result', that: allOf(isResult, hasValue(0))),
    ));
});

This post concludes the topic of assertions available in Dart and Flutter tests.

Follow Anna on social media to get notifications about her latest work!

Stay tuned for more updates and exciting news that we will share in the future. Follow us on Invertase Twitter, Linkedin, and Youtube, and subscribe to our monthly newsletter to stay up-to-date. You may also join our Discord to have an instant conversation with us.

Related posts