10. Testing

10.1. Unit tests

We encourage you to write automatic tests for your plugins. For unit testing, we recommend using pytest, and its mock module. These modules are already pre-installed in the pipenv environment if you followed the Getting started guide. Write the tests by usual means, and use the pluginv tool to run them.

pluginv run-tests

Refer to the pytest official documentation on how to write tests.

PyTest documentation

PyTest Mock project

To ease the testing, we provide fake services, with which you can fake out the services provided by SPS, and test your plugin without a real SPS.

• 10.1.1. Fake services for testing
  • 10.1.1.1. CredentialStore data injection
    • 10.1.1.1.1. Example usage with pytest and monkeypatch
  • 10.1.1.2. LDAPServer data injection
    • 10.1.1.2.1. Example usage with pytest and monkeypatch
  • 10.1.1.3. UserList data injection
    • 10.1.1.3.1. Example usage with pytest and monkeypatch

10.2. Integration tests

If your plugin uses external services (2FA for example), it is beneficial to write integration tests. For this, we recommend using the vcrpy module (pre-installed in the pipenv environment), with which you can record the communication (HTTP requests and responses) between your plugin and the external service, and later vcrpy can replay it without needing to access the service. Using this, your tests will be fast, deterministic, and can be run offline. Of course this test won’t detect if the external service has had incompatible changes, so you may want to re-record the communication regularly.

You don’t need to use the vcr module directly, the pluginv tool handles it automatically. All you need to do is to use either the urllib or the requests library for HTTP communication. Then if you run the tests using the pluginv run-recording-tests command, the communication will be recorded automatically, and will be put into YAML files in a folder named cassettes. After this recording is done, every time you run the tests using the pluginv run-tests command, vcr will replay the recorded communication from the YAML files.

For example, put this code into a file, with a name starting with test_.

import requests

def test_vcr_example():
    response = requests.get('https://httpbin.org/')
    assert response.ok
    assert 'A simple HTTP Request & Response Service.' in str(response.content)

Then use pluginv to run it, and record the HTTP requests and responses

pluginv run-recording-tests

Notice how slow the test run was. You will also notice that a new cassettes directory will show up with a YAML file in it. After the recording, all you have to do is run the tests the usual way. This time it will run much faster, as vcr will replay the previously recorded communication, without sending requests to the real service.

pluginv run-tests

If your plugin uses keyboard interactive input, and you want to test that also, you can use the @pytest.mark.interactive decorator (to mark the tests) along with the interactive fixture (to ask for the input). For example:

@pytest.mark.interactive
def test_interactive_input(interactive):
    password = interactive.askforinput('Please provide your password')
    assert password == 'secret'

The benefit of this is that you can record the keyboard inputs and replay them using the method described above, so you don’t always need to enter the input interactively. However, if you want to run the tests and provide the inputs by hand, you can use the following subcommand. Note that in this case, the tests will communicate with the real services, the cassettes won’t be replayed.

pluginv run-interactive-tests