Python: Multiple packages in one repository or one package per repository?

python python-3.x github setuptools python-packaging
AstrOne picture AstrOne · Jan 19, 2019 · Viewed 7.8k times · Source

I have a big Python 3.7+ project and I am currently in the process of splitting it into multiple packages that can be installed separately. My initial thought was to have a single Git repository with multiple packages, each with its own setup.py. However, while doing some research on Google, I found people suggesting one repository per package: (e.g., Python - setuptools - working on two dependent packages (in a single repo?)). However, nobody provides a good explanation as to why they prefer such structure.

So, my question are the following:

  • What are the implications of having multiple packages (each with its own setup.py) on the same GitHub repo?
  • Am I going to face issues with such a setup?
  • Are the common Python tools (documentation generators, pypi packaging, etc) compatible with with such a setup?
  • Is there a good reason to prefer one setup over the other?
  • Please keep in mind that this is not an opinion-based question. I want to know if there are any technical issues or problems with any of the two approaches.

Also, I am aware (and please correct me if I am wrong) that setuptools now allow to install dependencies from GitHub repos, even if the GitHub URL of the setup.py is not at the root of the repository.

Answer

boriska picture boriska · Mar 26, 2019

I am researching the same issue myself. PyPa documentation recommends the layout described in 'native' subdirectory of: https://github.com/pypa/sample-namespace-packages

I find the single package structure described below, very useful, see the discussion around testing the 'installed' version. https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure I think this can be extended to multiple packages. Will post as I learn more.

Related questions

Convert bytes to a string

I'm using this code to get standard output from an external program: >>> from subprocess import * >>> command_stdout = Popen(['ls', '-l'], stdout=PIPE).communicate()[0] The communicate() method returns an array of bytes: >>> …

Read More
Best way to convert string to bytes in Python 3?

There appear to be two different ways to convert a string to bytes, as seen in the answers to TypeError: 'str' does not support the buffer interface Which of these methods would be better or more Pythonic? Or is it …

Read More
How to install pip with Python 3?

I want to install pip. It should support Python 3, but it requires setuptools, which is available only for Python 2. How can I install pip with Python 3?

Read More