Testing

Software testing¶

From Software testing:

Software testing is the act of checking whether software satisfies expectations.

Why testing?¶

• Coding without testing is dangerous.

  

• To make sure I conform with the specs, and/or define correct specs.

  

• To avoid regression:

  • when there is a refactor

  • when there is a critical code evolution

  • when it crashes, to select where to look for the pb

What do we test?¶

• Unit tests

• Functional tests

Unit tests¶

Test that functions conform with the specifications.

def add(a, b):
    """ add a and b and return a+b"""
    return a + b


def test_add():
    assert add(1, 2) == 3
    ...

Functional tests¶

Check that the code works when assembling different functions, i.e. the functions can work together!

Test-driven development¶

From Test-driven development:

Test-driven development (TDD) is a way of writing code that involves writing an automated unit-level test case that fails, then writing just enough code to make the test pass, then refactoring both the test code and the production code, then repeating with another new test case.

Pytest and coverage¶

Pytest¶

Pytest is a software testing framework that helps you write and run readable and scalable tests. It is not part of the standard library so it needs to be installed

Tests are functions whose names starts with test_ written in files starting test_*.py. They check for behaviors of code. They check that code continues to work after modifications.

They can be executed by calling pytest. Alternatively, one can run pytest test_xxx.py to only run this test file.

There are several useful options (see pytest -h) but here is a selection of the most useful ones:

  -v, --verbose         Increase verbosity
  -s                    Shortcut for --capture=no

  -x, --exitfirst       Exit instantly on first error or failed test

  --lf, --last-failed   Rerun only the tests that failed at the last run (or all
                        if none failed)
  --ff, --failed-first  Run all tests, but run the last failures first.

Test coverage and the Coverage package¶

The notion of test coverage is useful. It is important to know which code is at least executed during testing. The coverage is the percentage of lines run by the test suite.

One can measure the coverage with the package coverage and the Pytest plugin pytest-cov (pip install pytest coverage pytest-cov).

If your code is in a directory src and your test files in a directory tests, pytest --cov=src tests will run your tests, measure the coverage and produce a short report. You can then produce a html visualisation of these results by running coverage html.

Exercise¶

from my_mod import add_second_twice

def test_add_second_twice():
    """ test add second twice"""
    print("testing add second twice with int ", end="")
    assert add_second_twice(3, 5) == 13
    print("...OK")
    print("testing add second twice with strings ", end="")
    assert add_second_twice("aa", "bb") == "aabbbb"
    print("...OK")
    print("testing add second twice with list ", end="")
    assert add_second_twice([1,2], [3,4]) == [1, 2, 3, 4, 3, 4]
    print("...OK")
    print("test add second twice OK with int, string and list")

def add_second_twice(arg0, arg1):
    """Return the sum of the first argument with twice the second one.
        Arguments should be of type that support
        sum and product by an integer
        (e.g. numerical, string, list, ...)
        :param arg0: first argument
        :param arg1: second argument
        :return: arg0 + 2 * arg1
    """
    pass

def add_second_twice(arg0, arg1):
    """Return the sum of the first argument with twice the second one.
    Arguments should be of type that support sum and product by
    an integer (e.g. numerical, string, list, ...)
    :param arg0: first argument
    :param arg1: second argument
    :return: arg0 + 2 * arg1
    """
    result = arg0 + 2 * arg1
    print(f"arg0 + 2*arg1 = {arg0} + 2*{arg1} = {result}")
    return result


def test_add_second_twice():
    """test add second twice"""
    print("testing add second twice with int ", end="")
    assert add_second_twice(3, 5) == 13
    print("...OK")
    print("testing add second twice with strings ", end="")
    assert add_second_twice("aa", "bb") == "aabbbb"
    print("...OK")
    print("testing add second twice with list ", end="")
    assert add_second_twice([1, 2], [3, 4]) == [1, 2, 3, 4, 3, 4]
    print("...OK")
    print("test add second twice OK with int, string and list")


test_add_second_twice()

testing add second twice with int arg0 + 2*arg1 = 3 + 2*5 = 13
...OK
testing add second twice with strings arg0 + 2*arg1 = aa + 2*bb = aabbbb
...OK
testing add second twice with list arg0 + 2*arg1 = [1, 2] + 2*[3, 4] = [1, 2, 3, 4, 3, 4]
...OK
test add second twice OK with int, string and list