Python 201: An Intro to mock

/ Cross-Platform, Python, Python 3, Testing / By / / Python, Testing

The unittest module now includes a mock submodule as of Python 3.3. It will allow you to replace portions of the system that you are testing with mock objects as well as make assertions about how they were used. A mock object is used for simulating system resources that aren’t available in your test environment. In other words, you will find times when you want to test some part of your code in isolation from the rest of it or you will need to test some code in isolation from outside services.

Note that if you have a version of Python prior to Python 3, you can download the Mock library and get the same functionality.

Let’s think about why you might want to use mock. One good example is if your application is tied to some kind of third party service, such as Twitter or Facebook. If your application’s test suite goes out and retweets a bunch of items or “likes” a bunch of posts every time its run, then that is probably undesirable behavior since it will be doing that every time the test is run. Another example might be if you had designed a tool for making updates to your database tables easier. Each time the test runs, it will do some updates on the same records every time and could wipe out valuable data.

Instead of doing any of those things, you can use unittest’s mock. It will allow you to mock and stub out those kinds of side-effects so you don’t have to worry about them. Instead of interacting with the third party resources, you will be running your test against a dummy API that matches those resources. The piece that you care about the most is that your application is calling the functions it’s supposed to. You probably don’t care as much if the API itself actually executes. Of course, there are times when you will want to do an end-to-end test that does actually execute the API, but those tests don’t need mocks!

Simple Examples

The Python mock class can mimic any other Python class. This allows you to examine what methods were called on your mocked class and even what parameters were passed to them. Let’s start by looking at a couple of simple examples that demonstrate how to use the mock module:

>>> from unittest.mock import Mock
>>> my_mock = Mock()
>>> my_mock.__str__ = Mock(return_value='Mocking')
>>> str(my_mock)
'Mocking'

In this example, we import Mock class from the unittest.mock module. Then we create an instance of the Mock class. Finally we set our mock object’s __str__ method, which is the magic method that controls what happens if you call Python’s str function on an object. In this case, we just return the string “Mocking”, which is what you see when we actually execute the str() function at the end.

The mock module also supports five asserts. Let’s take a look at how at a couple of those in action:

>>> from unittest.mock import Mock
>>> class TestClass():
...     pass
... 
>>> cls = TestClass()
>>> cls.method = Mock(return_value='mocking is fun')
>>> cls.method(1, 2, 3)
'mocking is fun'
>>> cls.method.assert_called_once_with(1, 2, 3)
>>> cls.method(1, 2, 3)
'mocking is fun'
>>> cls.method.assert_called_once_with(1, 2, 3)
Traceback (most recent call last):
  Python Shell, prompt 9, line 1
  File "/usr/local/lib/python3.5/unittest/mock.py", line 802, in assert_called_once_with
    raise AssertionError(msg)
builtins.AssertionError: Expected 'mock' to be called once. Called 2 times.
>>> cls.other_method = Mock(return_value='Something else')
>>> cls.other_method.assert_not_called()
>>>

First off, we do our import and create an empty class. Then we create an instance of the class and add a method that returns a string using the Mock class. Then we call the method with three integers are arguments. As you will note, this returned the string that we set earlier as the return value. Now we can test out an assert! So we call the **assert_called_once_with** assert which will assert if we call our method two or more times with the same arguments. The first time we call the assert, it passes. So then we call the method again with the same methods and run the assert a second time to see what happens.

As you can see, we got an AssertionError. To round out the example, we go ahead and create a second method that we don’t call at all and then assert that it wasn’t called via the assert_not_called assert.

Side Effects

You can also create side effects of mock objects via the side_effect argument. A side effect is something that happens when you run your function. For example, some videogames have integration into social media. When you score a certain number of points, win a trophy, complete a level or some other predetermined goal, it will record it AND also post about it to Twitter, Facebook or whatever it is integrated with. Another side effect to running a function is that it might be tied to closely with your user interface and cause it to redraw unnecessarily.

Since we know about these kinds of side effect up front, we can mock them in our code. Let’s look at a simple example:

from unittest.mock import Mock


def my_side_effect():
    print('Updating database!')
    
def main():
    mock = Mock(side_effect=my_side_effect)
    mock()
    
if __name__ == '__main__':
    main()

Here we create a function that pretends to update a database. Then in our main function, we create a mock object and give it a side effect. Finally we call our mock object. If you do this, you should see a message printed to stdout about the database being updated.

The Python documentation also points out that you can make side effect raise an exception if you want to. One fairly common reason to want to raise an exception if you called it incorrectly. An example might be that you didn’t pass in enough arguments. You could also create a mock that raises a Deprecation warning.

Autospeccing

The mock module also supports the concept of auto-speccing. The autospec allows you to create mock objects that contain the same attributes and methods of the objects that you are replacing with your mock. They will even have the same call signature as the real object! You can create an autospec with the create_autospec function or by passing in the autospec argument to the mock library’s patch decorator, but we will postpone looking at patch until the next section.

For now, let’s look at an easy-to-understand example of the autospec:

>>> from unittest.mock import create_autospec
>>> def add(a, b):
...     return a + b
... 
>>> mocked_func = create_autospec(add, return_value=10)
>>> mocked_func(1, 2)
10
>>> mocked_func(1, 2, 3)
Traceback (most recent call last):
  Python Shell, prompt 5, line 1
  File "", line 2, in add
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 181, in checksig
    sig.bind(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py", line 2921, in bind
    return args[0]._bind(args[1:], kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/inspect.py", line 2842, in _bind
    raise TypeError('too many positional arguments') from None
builtins.TypeError: too many positional arguments

In this example, we import the create_autospec function and then create a simple adding function. Next we use create_autospec() by passing it our add function and setting its return value to 10. As long as you pass this new mocked version of add with two arguments, it will always return 10. However, if you call it with the incorrect number of arguments, you will receive an exception.

The patch

The mock module has a neat little function called patch that can be used as a function decorator, a class decorator or even a context manager. This will allow you to easily create mock classes or objects in a module that you want to test as it will be replaced by a mock.

Let’s start out by creating a simple function for reading web pages. We will call it webreader.py. Here’s the code:

import urllib.request


def read_webpage(url):
    response = urllib.request.urlopen(url)
    return response.read()

This code is pretty self-explanatory. All it does is take a URL, opens the page, reads the HTML and returns it. Now in our test environment we don’t want to get bogged down reading data from websites especially is our application happens to be a web crawler that downloads gigabytes worth of data every day. Instead, we want to create a mocked version of Python’s urllib so that we can call our function above without actually downloading anything.

Let’s create a file named mock_webreader.py and save it in the same location as the code above. Then put the following code into it:

import webreader

from unittest.mock import patch


@patch('urllib.request.urlopen')
def dummy_reader(mock_obj):
    result = webreader.read_webpage('https://www.google.com/')
    mock_obj.assert_called_with('https://www.google.com/')
    print(result)
    
if __name__ == '__main__':
    dummy_reader()

Here we just import our previously created module and the patch function from the mock module. Then we create a decorator that patches urllib.request.urlopen. Inside the function, we call our webreader module’s read_webpage function with Google’s URL and print the result. If you run this code, you will see that instead of getting HTML for our result, we get a MagicMock object instead. This demonstrates the power of patch. We can now prevent the downloading of data while still calling the original function correctly.

The documentation points out that you can stack path decorators just as you can with regular decorators. So if you have a really complex function that accesses databases or writes file or pretty much anything else, you can add multiple patches to prevent side effects from happening.

Wrapping Up

The mock module is quite useful and very powerful. It also takes some time to learn how to use properly and effectively. There are lots of examples in the Python documentation although they are all simple examples with dummy classes. I think you will find this module useful for creating robust tests that can run quickly without having unintentional side effects.

Related Reading