Running the tests
If you push a commit to a branch in the Dallinger organization on GitHub, or open a pull request from your own fork, Dallinger’s automated code tests will be run on GitHub.
The tests include:
Making sure that a source distribution of the Python package can be created.
Running flake8 to make sure Python code conforms to the PEP 8 style guide.
Running the tests for the Python code using pytest and making sure they pass on Python 3.7, 3.8, and 3.9.
Making sure that code coverage for the Python code is above the desired threshold.
Making sure the docs build without error.
If you see ImportErrors related to demo packages, this most likely means you
have not installed the
dlgr.demos sub-package. See the
Dallinger development installation instructions for details.
Amazon Mechanical Turk Integration Tests
You can also run all these tests locally, with some additional requirements:
The Amazon Web Services credentials set in .dallingerconfig must correspond to a valid MTurk Sandbox Requester account.
Some tests require access to an MTurk Sandbox Worker account, so you should create this account (probably using the same AWS account as above).
The Worker ID from the Worker account (visible on the dashboard) needs to be set in
tests/config.py, which should be created by making a copy of
tests/config.py.inbefore setting the value.
tests/config.pyis excluded from version control, so your Id will not be pushed to a remote repository.
You can run all tests locally, simply by running:
To run just the fastest Python tests (it’s recommended to run these tests first):
To run include slower Python tests:
To run the Python tests excluding those that interact with Amazon Mechanical Turk, run:
pytest -m "not mturk"
To run all tests except those that require a MTurk Worker ID, run:
pytest -m "not mturkworker"
To run the complete, comprehensive suite of tests which interact Mechanical Turk,
mturkfull option when running the tests:
pytest --mturkfull --runslow
npm run test --coverage
To run black:
To run flake8: