How can I repeat each test multiple times in a py.test run?

python pytest
Martin Del Vecchio picture Martin Del Vecchio · Feb 13, 2014 · Viewed 30k times · Source

I want to run each selected py.test item an arbitrary number of times, sequentially.
I don't see any standard py.test mechanism for doing this.

I attempted to do this in the pytest_collection_modifyitems() hook. I modified the list of items passed in, to specify each item more than once. The first execution of a test item works as expected, but that seems to cause some problems for my code.

Further, I would prefer to have a unique test item object for each run, as I use id (item) as a key in various reporting code. Unfortunately, I can't find any py.test code to duplicate a test item, copy.copy() doesn't work, and copy.deepcopy() gets an exception.

Can anybody suggest a strategy for executing a test multiple times?

Answer

Frank T picture Frank T · Feb 18, 2014

One possible strategy is parameterizing the test in question, but not explicitly using the parameter.

For example:

@pytest.mark.parametrize('execution_number', range(5))
def run_multiple_times(execution_number):
    assert True

The above test should run five times.

Check out the parametrization documentation: https://pytest.org/latest/parametrize.html

Related questions

PATH issue with pytest 'ImportError: No module named YadaYadaYada'

I used easy_install to install pytest on a mac and started writing tests for a project with a file structure likes so: repo/ repo/app.py repo/settings.py repo/models.py repo/tests/ repo/tests/test_app.py …

Read More
How to properly assert that an exception gets raised in pytest?

Code: # coding=utf-8 import pytest def whatever(): return 9/0 def test_whatever(): try: whatever() except ZeroDivisionError as exc: pytest.fail(exc, pytrace=True) Output: ================================ test session starts ================================= platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2 plugins: django, cov collected 1 items pytest_…

Read More
How can I see normal print output created during pytest run?

Sometimes I want to just insert some print statements in my code, and see what gets printed out when I exercise it. My usual way to "exercise" it is with existing pytest tests. But when I run these, I don't …

Read More