Running the tests

If you push a commit to a branch in the Dallinger organization on GitHub, or open a pull request from your own fork, Dallinger’s automated code tests will be run on GitHub.

Current build status: status

The tests include:

  • Making sure that a source distribution of the Python package can be created.

  • Running flake8 to make sure Python code conforms to the PEP 8 style guide.

  • Running the tests for the Python code using pytest and making sure they pass on Python 3.7, 3.8, and 3.9.

  • Making sure that code coverage for the Python code is above the desired threshold.

  • Making sure the docs build without error.

If you see ImportErrors related to demo packages, this most likely means you have not installed the dlgr.demos sub-package. See the Dallinger development installation instructions for details.

Amazon Mechanical Turk Integration Tests

You can also run all these tests locally, with some additional requirements:

  • The Amazon Web Services credentials set in .dallingerconfig must correspond to a valid MTurk Sandbox Requester account.

  • Some tests require access to an MTurk Sandbox Worker account, so you should create this account (probably using the same AWS account as above).

  • The Worker ID from the Worker account (visible on the dashboard) needs to be set in tests/, which should be created by making a copy of tests/ before setting the value. tests/ is excluded from version control, so your Id will not be pushed to a remote repository.



You can run all tests locally, simply by running:


To run just the fastest Python tests (it’s recommended to run these tests first):


To run include slower Python tests:

pytest --runslow

To run the Python tests excluding those that interact with Amazon Mechanical Turk, run:

pytest -m "not mturk"

To run all tests except those that require a MTurk Worker ID, run:

pytest -m "not mturkworker"

To run the complete, comprehensive suite of tests which interact Mechanical Turk, add the mturkfull option when running the tests:

pytest --mturkfull --runslow

To run the JavaScript tests, run:

npm run test --coverage


To run black:

black dallinger

To run flake8: