Virtual Python on macOS

This document describes how to use a virtual-Python setup to layer packages of your choice on top of Python on macOS systems.

You might also want to install the extra MacPorts python libraries via the Self Service application, instructions can be found here

Or you can install Anaconda Python for your user account, instructions can be found here

Initial set up of Python 3.5 and up

Create a directory in /Data/username/, replace username with the name of the account you are using:

cd /Data
mkdir -p username
cd /Data/username

Now create the virtual environment:

cd /Data/username
python3 -m venv --system-site-packages myVirtualenv-3
source /Data/username/myVirtualenv-3/bin/activate.csh (users of csh)
source /Data/username/myVirtualenv-3/bin/activate (users of bash)
python3 -m pip install --upgrade pip setuptools wheel

Setup your aliases to point to the virtual environment you have just created

To simplify the startup process for the next time, you can add these aliases to your shell startup files, for use in subsequent shell sessions.

  • .cshrc: the middle of the line sbould read "-init 'source" (leave only one space).

    alias vpython3-init \
        'source /Data/username/myVirtualenv-3/bin/activate.csh'

  • .bashrc: the middle of the line should read "-init='. /Data" (leave only the equals sign).

    alias vpython3-init=\
        '. /Data/usernme/myVirtualenv-3/bin/activate'

Once the alias is set open a new shell and type the following to setup the python virtual environment.

vpython3-init

Now check that the shell prompt has changed to reflect the change:

which python
which python3

.... you'll see a Python instance named which is inside your virtual environment.

Installing and using Python packages

To install the PyRAF package into the virtual environment, activate your virtual Python and then:

python3 -m pip install pyraf

You can then verify that your Python package has been installed by starting Python, and saying at its prompt:

import pyraf
pyraf.__file__ # (a pathname inside your virtual environment)
pyraf.__version__ # ('2.1.14' at the time of writing)

.... or (in PyRAF's case) by saying at the shell prompt:

which pyraf

.... and verifying that its pathname is within your virtual-Python directory.

Third-party Python scripts

A script intended to be invoked by name at the shell prompt may well have the "flash-bang" line at the beginning, to tell it which (in this case) Python interpreter to use, eg one of:

#!/usr/bin/python # (WRONG under macOS: system Python)
#!/opt/local/bin/python3 # (WRONG under Linux: macOS-specific)

This is Bad News: it hard-wires use of one particular Python interpreter (Apple's, or MacPorts's, respectively) into the program, which is likely to break portability. In third-party Python scripts or examples, you may even see:

#!/usr/local/bin/python # (WRONG)

.... or some other variant which assumes you've installed some version of Python manually.

To properly generalise such scripts, edit the first line to instead say:

#!/usr/bin/env python3

This will cause the script to use the first appropriate Python executable on $PATH, which (when it's active) will be the one in your virtual environment.

Leaving the virtual Python environment

If you wish (for whatever reason) to revert to using the system Python, exit from Python if necessary, then say (at the shell prompt):

deactivate

Note that the shell prompt has been changed back, as saying:

which python # (should show the system Python)
which python3 # (should complain or return nothing)

.... will confirm.

Cloning a virtual Python environment

Virtual Pythons have absolute pathnames stitched in, so they can't be moved or copied as-is; but they can be recreated without excessive effort. The first part is done inside the virtual Python in question using:

python3 -m pip freeze -l | tee ~/requirements-3.txt

To add that set of packages into another virtual Python setup elsewhere, deactivate the old virtual Python, activate the new one (creating it if necesary), then saying in it:

python3 -m pip install --upgrade -r ~/requirements-3.txt

The requirements file documents your required set of extra packages in a near-executable form. If you're keeping all your own Python programs under version control, it would be wise to add this file to your saved set as a record of any support libraries you may have found necessary, and to keep it up-to-date. Please bear in mind that the underlying system packages, which aren't mentioned here, may well differ between your desktop, your laptop, and whatever Linux servers you're using. If in doubt, use separate names for them (eg requirements-desktop.txt, requirements-glamdring.txt etc)

Categories: Apple | Mac | python | virtual Python