Testing arviz-stats#
How to run the test suite#
The recommended way to run the test suite is to do it via tox.
Tox manages the environment, its env variables and the command to run
to allow testing the library with different combinations of optional dependencies
from the same development env.
To run the full test suite run:
tox -e full
Tip
Whenever you need dependencies to be updated for tests to pass add the r flag to indicate
you want the virtual environment the job runs from to be recreated:
tox -e full -r
At the time of writing, this will run the test suite after installing arviz-stats[xarray,numba]
and with env vars to ensure no test is skipped due to missing dependencies.
If a dependency is missing the respective test will fail instead.
To run the parts of the test suite that can be run with only some of the optional dependencies installed there are the following two commands:
tox -e minimal # after installing arviz-stats
tox -e xarray # after installing arviz-stats[xarray]
In addition, there is also a nightlies job that will install available packages
from the scientific python nightly index:
tox -e nightlies
Note
Installing this env will generally take noticeably longer than the envs for the other jobs.
This is because the cache pip builds when you install the same package and version will
generally not work and some nightlies might also require installing from source instead
of using a wheel. Moreover, this env is also much more prone to needing the -r flag.
How to write tests#
Use importorskip helper function from
tests/helpers.py
for any import outside of the Python standard library plus NumPy and SciPy. For example:
import copy
import re
import numpy as np
from .helpers import importorskip
xr = importorskip("xarray")
#... in the code use xr.Dataset, xr.DataArray as usual
As importorskip will skip all tests in that file, tests should be divided into
files with tests of the array_stats interface always being on their own file
with no optional dependencies import and the rest in a second (or more) files.
About arviz-stats testing#
The recommended install is via arviz-stats[xarray] which provides functionality similar to
the functions that used to be within arviz/stats module. However, it is also possible to install
arviz-stats without the xarray or arviz-base dependencies in which case a minimal version of the
library that depends only on NumPy and SciPy is installed.
To take this into account, tests use the importorskip helper function from
tests/helpers.py
when importing optional dependencies like xarray, arviz-base or pandas so that tests are skipped
if a dependency is not available.
In addition, the env variable ARVIZ_REQUIRE_ALL_DEPS can be set to disable this behaviour
and ensure uninstalled dependencies raise an error.
When using tox -e full all optional dependencies are installed,
and ARVIZ_REQUIRE_ALL_DEPS is set to ensure all tests in the test suite run.
However, the other jobs only install some optional dependencies and don’t set the env variable
which ensures that the smaller installs are also viable and work as expected.
Consequently, when writing tests we need to take this into account and separate the testing of the array functions and their results from the xarray interface/api built on top of that.
At the time of writing, we run the full, xarray and minimal jobs for all python versions
and the nightlies job for one python version.
Which tests are executed is defined by the combination of
tox.ini and
.github/workflows/test.yml.