My Python Development Environment:
My Python Development Environment, 2018 Edition
This is out of date. For a newer version, see My Python Development Environment, 2020 Edition
For years I’ve noodled around with various setups for a Python development environment, and never really found something I loved – until now.
My setup pieces together pyenv, pipenv, and pipsi. It’s probably a tad more complex that is ideal for most Python users, but for the things I need, it’s perfect.
I do have somewhat specific (maybe unusual?) requirements:
- I need to develop against multiple Python versions, including 2.7, various Python 3 versions (3.5 and 3.6, mostly), and PyPy.
- I work on many projects simultaneously, each with different sets of dependencies, so some sort of virtual environment or isolation is critical.
- I use multiple OSes: macOS at work, and Linux (well, Linux-ish - actually it’s WSL) at home.
- I want to avoid using the System-provided Python. On macOS it’s too outdated. On Linux, the system Python is used by the OS itself, so if you hose your Python you can hose your system.
- I use a bunch Python-based CLI stuff, like youtube-dl, awscli, doc2dash, etc. I want to be able to install and use them without fussing around with activating environments, but I also don’t want their dependencies to clutter up a global installation.
- I deploy stuff mostly to Heroku (personal) and cloud.gov (work). I expect this not to change: I’m spoiled, and never want to manage my own infrastructure again.
- Although Docker meets all these requirements, I don’t really like using it. I find it slow, frustrating, and overkill for my purposes.
Why? I need to run multiple Python versions, isolated from the
system Python. pyenv makes it easy to
install, manage, and switch between those multiple Pythons. As a bonus,
pipenv integrates with
pyenv and will automatically install
missing Python versions if they’re required by a
On my Mac, I installed pyenv from
brew install pyenv). On Linux, I used the Github
installation technique documented in the installation
instructions, which was
easy and went smoothly.
Then, I installed some Python versions:
pyenv install 3.6.4 pyenv install 3.5.4 pyenv install 2.7.14 pyenv install pypy3.5-5.10.0
And made sure my default Python was set to the latest and greatest:
pyenv global 3.6.4
Why? pipsi lets me install Python-based CLI stuff (like youtube-dl, awscli, doc2dash, etc.) without those projects’ dependencies messing up my global Python.
Normally, installing pipsi is easy:
curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python
However, at the time I’m writing this (late February 2018), this didn’t work for me; I needed to do some tweaking to get it to work. I followed the workarounds documented in issues #124 and #125. Hopefully this’ll be fixed shortly.
I didn’t do this, but it might be a good idea to install
pipsi using the system Python
pyenv local system before running the installer). The
downside is that this will use a potentially old-ass Python, but the
upside is that you won’t break your
pipsi install if you delete a
pyenv version (like upon an upgrade).
Why? pipenv handles dependency- and virtual-environment-management in a way that’s very intuitive (to me), and fits perfectly with my desired workflow.
The documentation covers a few different ways to install pipenv. Because I’m already using pipsi, I chose the pipsi-based installation:
pipsi install pew pipsi install pipenv
What it looks like in use
With this all together, all my use-cases are handled simply:
To start new projects, I just make a directory and type
pipenv install ... to start installing my dependencies. Pipenv creates a
me, and manages it, and I’m up and running.
To work on existing projects, I clone a repository and either run
pipenv install (for projects that already have a
pipenv install -r requirements.txt (which as a side-effect
automatically converts a the requirements file to a
If I need to switch Python versions, I run
pyenv local <version> in my
project directory. I can also add:
[requires] python_version = "<version>"
pipenv will enforce that version requirement.
When I want to install CLI stuff, I use pipsi:
pipsi install awscli pipsi install doc2dash # ... etc
When it comes time to deploy, both Heroku and
cloud.gov will read and understand my
If I need to deploy to something that doesn’t do
installs, I create a
requirements.txt by running
pipenv lock --requirements.