A quick write up on how I set up a project myproject to use a virtual environment. This is needed if I return to it months later, when the python packages of that day might be incompatible with the version I used at creation.

(Note: use python -m venv rather than python -m virtualenv)

Get help!

python -m venv --help

Say where to put the installation, and to use a prompt: “MYPROJ”

python -m venv .myvirt --prompt MYPROJ

Note, if we want a different version of python, we need to install python36 with our system package manager (e.g. AWS use 3.6):

python3.6 -m venv .myvirt36 --prompt MYPROJ

(E.g. tensorflow versions < 2. are not available in python3.8)

To enter into the virtual environment, from the project folder:

source .myvirt/bin/activate
pip install pip-tools

List of packages I want in a file requirements.in

numpy
torch
nltk
gensim
beautifulsoup4==4.6.0
allennlp

From the package list (requirements.in), compile list requirements.txt of specific version and their dependencies

pip-compile requirements.in

Install/uninstall packages such that our environment looks like specified (e.g. if I’ve pip-installed a package from command line, instead of putting it in requirements.in/txt-file it will be removed):

pip-sync requirements.txt

To quit/exit virtual env

deactivate

or, if that doesn’t work:

source deactivate

Example to make Emacs python-mode use the new virtual env, eval:

(setq python-shell-interpreter (concat (getenv "HOME") "/myproject/.myvirt/bin/python")
      python-shell-interpreter-args "-i")

or better add file local variables e.g. to the bottom of the python file:

# Local Variables:
# python-shell-interpreter:  "~/myproject/.myvirt/bin/python"
# python-shell-interpreter-args: "-i"
# End:

Problem to solve: flycheck lint-checker still uses global python, which gives a complaint for packages that are installed in the virtual env but not the global, complaining the import statments are wrong.

Just a for myself to remember, general pip-commands:

List all installed packages visible

pip freeze

Install package globally, download it again

pip install <package> --no-cache

Uninstall existing package

pip uninstall <package>

Update locally/user installed package (user specific)

pip install --user <package> -U

Install package from a git repo:

pip install git+https://github.com/user/repository.git@branch

or faster:

pip install https://github.com/user/repository/archive/branch.zip

See which versions are available (if pip > 9.0, else ...==anything)

pip install <package>==

Install pyenv and virtualenv (uses python -m venv by default):

curl https://pyenv.run | bash

This installs:

• pyenv: The actual pyenv application
• pyenv-virtualenv: Plugin for pyenv and virtual environments
• pyenv-update: Plugin for updating pyenv
• pyenv-doctor: Plugin to verify that pyenv and build dependencies are installed
• pyenv-which-ext: Plugin to automatically lookup system commands

Usage example,

1. Tell pyenv to install a new python version

pyenv install 3.9.2

1. create a virtual environment associated with that python version

pyenv virtualenv 3.9.2 toca-model

1. associate a virtual environment with current folder (and all sub folders)

pyenv local toca-model

now have a file .python-version in the folder specifying the virtual environment

It’s still possible to activate/deactivate a virtual environment manually (not sure why one would want that)

pyenv activate <name>
pyenv deactivate

A virtual environment can be removed by:

pyenv uninstall <name>

which removes the associated folder(s?) ./pyenv/versions/ and ~./pyenv/versions/{pythonversion}/envs

There are several upsides with

• Can offer different versions of installed python, which venv doesn’t do. List possible versions available to install

  pyenv install -l
  

  Special system version picks from the system python from $PATH variable.

  Install/uninstall python version (tab completion works fine):

  pyenv install <versions>
  pyenv uninstall <versions>
  

  List installed python version and virtual environments:

  pyenv versions
  

  Show current version / virtual environment

  pyenv version
  

  Show path to current version:

  pyenv which <version>
  

• It seamlessly switches virtual environment as you cd between directories, no need to manually turn them on/off as with python -m venv. (It loads the version set in .python-version, first found recursively up in directory structure).

  Activate in folder (writes .python-version in current folder):

  pyenv local <version>
  

  set global version (writes ~/.pyenv/version). Will be overwritten by PYENV_VERSION or local .python-version.

  pyenv global
  

  Set/unset shell specific version, (by setting PYENV_VERSION)

  pyenv shell <version>
  pyenv shell --unset
  

  (needs eval "$(pyenv init -)" in your .zshrc or .bashrc)

• Supported by (one of) Emacs’ python IDE: elpy

python --version

It works by editing $PATH, to have the right version of python, depending on the project you’re currently in. It provides several commands

pyenv install -l

pyenv uninstall <version>

https://github.com/pyenv/pyenv-virtualenv

creates virtual envs virtualenv

pyenv virtualenv <version> mytest

list virtual environments

pyenv virtualenvs

Q: pyenv-virtualenv only for anaconda and virtualenv - not the same as python -m venv

Q: installed through package manager

Q: add to zsh needed? (below if using from gitclone repo):

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n  eval "$(pyenv init -)"\nfi' >> ~/.zshrc

Q: when: “Installs shims for all Python binaries known to pyenv (i.e., ~/.pyenv/versions/*/bin/*). Run this command after you install a new version of Python, or install a package that provides binaries.”

eval "$(pyenv init -)"

In a active virtual environment, you can create a new ipython kernel that tells ipython which environment to use:

pip install ipykernel
python -m ipykernel install --user --name my_kernel

(Best name the kernel as same as the virtual environment).

Then start jupyter:

jupyter notebook

And in kernel menu select my_kernel.

List kernels

jupyter kernelspec list

Remove a kernel

jupyter kernelspec uninstall my_kernel

org

jupyter kernelspec remove old_kernel