B17 - unittest framework

source: unittest python docs

G2 - unit test

• the unittest framework supports test automation, sharing of setup and shutdown code for tests, aggregation of tests into collections, and independence of the tests from the reporting framework

• test fixture

  • the preparation needed to perform tests, and any associated cleanup
  • eg: creating proxy databases, directories, etc

• test case

  • an individual unit of testing
  • checks for a specific response to a particular set of inputs
  • TestCase is a base class which may be used to create test cases

• test suite

  • a collection of test cases, test suites, or both
  • used to aggregate tests that should be run together

• test runner

  • a component which orchestrates the execution of tests
  • provides the outcome to the user

import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)

if __name__ == '__main__':
    unittest.main()

• a test case is created by subclassing

• conventionally, the individual testing methods have names starting with test

• the crux of each is to call assertEqual(), assertTrue() or assertFalse() instead of the assert statement

• this allows the accumulation of all the results to produce a report

• unittest.main() runs the test and provides a command line output:

...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

• -v option enables a higher level of verbosity

test_isupper (__main__.TestStringMethods.test_isupper) ... ok
test_split (__main__.TestStringMethods.test_split) ... ok
test_upper (__main__.TestStringMethods.test_upper) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

organisation

• setUp() is called every time a test is run
• if it raises an exception, the framework will consider the test to have suffered an error, and the test method will not be executed
• tearDown() tidies up after the test method is run
• if setUp() succeeded, tearDown() will be run regardless of the test methods succeeding or not
• such an environment is called a test fixture

import unittest

class WidgetTestCase(unittest.TestCase):
    def setUp(self):
        self.widget = Widget('The widget')

    def test_default_widget_size(self):
        self.assertEqual(self.widget.size(), (50,50),
                         'incorrect default size')

    def test_widget_resize(self):
        self.widget.resize(100,150)
        self.assertEqual(self.widget.size(), (100,150),
                         'wrong size after resize')
                         
	 def tearDown(self):
        self.widget.dispose()

reusing old test code

• FunctionTestCase is a subclass of TestCase that can be used to wrap a test function
• this is a quick and easy method, but not recommended

def testSomething():
    something = makeSomething()
    assert something.name is not None
    # ...
    
testcase = unittest.FunctionTestCase(testSomething,
	 setUp=makeSomethingDB,
	 tearDown=deleteSomethingDB)

skipping

• individual methods as well as classes can be skipped or marked as an expected failure

## unconditional skipping
@unittest.skip(reason) 

## conditional skipping: if true
@unittest.skipIf(condition, reason) 

## conditional skipping: unless true
@unittest.skipUnless(condition, reason)

## success if test fails in the funtion itself; failiture if it passes
@unittest.expectedFailure 

## execption raised to skip a test
*exception* unittest.SkipTest(reason)

• usage:

class MyTestCase(unittest.TestCase):

    @unittest.skip("demonstrating skipping")
    def test_nothing(self):
        self.fail("shouldn't happen")

    @unittest.skipIf(mylib.__version__ < (1, 3),  "not supported in this library version")
    def test_format(self):
        # Tests that work for only a certain version of the library.
        pass

    @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
    def test_windows_support(self):
        # windows specific testing code
        pass

    def test_maybe_skipped(self):
        if not external_resource_available():
            self.skipTest("external resource not available")
        # test code that depends on the external resource
        pass

test_format (__main__.MyTestCase.test_format) ... skipped 'not supported in this library version'
test_nothing (__main__.MyTestCase.test_nothing) ... skipped 'demonstrating skipping'
test_maybe_skipped (__main__.MyTestCase.test_maybe_skipped) ... skipped 'external resource not available'
test_windows_support (__main__.MyTestCase.test_windows_support) ... skipped 'requires Windows'

----------------------------------------------------------------------
Ran 4 tests in 0.005s

OK (skipped=4)

@unittest.skip("showing class skipping")
class MySkippedTestCase(unittest.TestCase):
    def test_not_run(self):
        pass
TestCase.set

• expected failures use the expectedFailure decorator:

class ExpectedFailureTestCase(unittest.TestCase):
    @unittest.expectedFailure
    def test_fail(self):
        self.assertEqual(1, 0, "broken")

grouping

class unittest.TestSuite(tests=())

• behaves similarly to TestCase, but do not implement a test
• aggregate tests into groups that should be run together
• methods:
  • addTest(test)
  • addTests(tests)