Using py.test with coverage doesn't include imports

Question

For Jedi we want to generate our test coverage. There is a related question in stackoverflow, but it didn't help.

We're using py.test as a test runner. However, we are unable to add the imports and other "imported" stuff to the report. For example __init__.py is always reported as being uncovered:

Name                           Stmts   Miss  Cover
--------------------------------------------------
jedi/__init__                      5      5     0%
[..]

Clearly this file is being imported and should therefore be reported as tested.

We start tests like this [*]:

py.test --cov jedi

As you can see we're using pytest-coverage.

So how is it possible to properly count coverage of files like __init__.py?

[*] We also tried starting test without --doctest-modules (removed from pytest.ini) and activate the coverage module earlier by py.test -p pytest_cov --cov jedi. Neither of them work.

I've offered a bounty. Please try to fix it within Jedi. It's publicly available.

Answer 1

@hynekcer gave me the right idea. But basically the easiest solution lies somewhere else:

Get rid of pytest-cov!

Use

coverage run --source jedi -m py.test
coverage report

instead!!! This way you're just running a coverage on your current py.test configuration, which works perfectly fine! It's also philosophically the right way to go: Make each program do one thing well - py.test runs tests and coverage checks the code coverage.

Now this might sound like a rant, but really. pytest-cov hasn't been working properly for a while now. Some tests were failing, just because we used it.

---

As of 2014, pytest-cov seems to have changed hands. py.test --cov jedi test seems to be a useful command again (look at the comments). However, you don't need to use it. But in combination with xdist it can speed up your coverage reports.