UP | HOME
Impaktor

Impaktor

Inject Emacs intravenously

On using Python & pip-tools for optimal virtual env
Published on Sep 12, 2020 by Impaktor.

Introduction

A quick write up on how I set up a project myproject to use a virtual environment. This is needed if I return to it months later, when the python packages of that day might be incompatible with the version I used at creation.

Setting up a virtual environment

(Note: use python -m venv rather than python -m virtualenv)

Get help!

python -m venv --help

Say where to put the installation, and to use a prompt: “MYPROJ”

python -m venv .myvirt --prompt MYPROJ

Note, if we want a different version of python, we need to install python36 with our system package manager (e.g. AWS use 3.6):

python3.6 -m venv .myvirt36 --prompt MYPROJ

(E.g. tensorflow versions < 2. are not available in python3.8)

To enter into the virtual environment, from the project folder:

source .myvirt/bin/activate
pip install pip-tools

List of packages I want in a file requirements.in

numpy
torch
nltk
gensim
beautifulsoup4==4.6.0
allennlp

From the package list (requirements.in), compile list requirements.txt of specific version and their dependencies

pip-compile requirements.in

Install/uninstall packages such that our environment looks like specified (e.g. if I’ve pip-installed a package from command line, instead of putting it in requirements.in/txt-file it will be removed):

pip-sync requirements.txt

To quit/exit virtual env

deactivate

or, if that doesn’t work:

source deactivate

Make venv work with Emacs

Example to make Emacs python-mode use the new virtual env, eval:

(setq python-shell-interpreter (concat (getenv "HOME") "/myproject/.myvirt/bin/python")
      python-shell-interpreter-args "-i")

or better add file local variables e.g. to the bottom of the python file:

# Local Variables:
# python-shell-interpreter:  "~/myproject/.myvirt/bin/python"
# python-shell-interpreter-args: "-i"
# End:

Problem to solve: flycheck lint-checker still uses global python, which gives a complaint for packages that are installed in the virtual env but not the global, complaining the import statments are wrong.

General pip commands

Just a for myself to remember, general pip-commands:

List all installed packages visible

pip freeze

Install package globally, download it again

pip install <package> --no-cache

Uninstall existing package

pip uninstall <package>

Update locally/user installed package (user specific)

pip install --user <package> -U

Install package from a git repo:

pip install git+https://github.com/user/repository.git@branch

or faster:

pip install https://github.com/user/repository/archive/branch.zip

See which versions are available (if pip > 9.0, else ...==anything)

pip install <package>==