Skip to main content

Using conda environment

First, to use conda environments you have to load miniconda module.
For example, from login node:

$ module avail miniconda3

In this tutorial, miniconda3-22.11.1/gcc-13.1.0 will be used.

$ module load miniconda3-22.11.1/gcc-13.1.0

Now we can use conda commands, the commands we will see in this tutorial are:

  1. create to create an environment
  2. activate to enter an environment
  3. deactivate to exit an environment
  4. install to install packages in environment
  5. env to manage environments
Help

All these commands and sub-commands take the -h or --help so you can see all their optional and positional arguments.

Create conda environment​

To create an environment:

$ conda create -n <env_name> [<packages_list>]

Where:

  • -n give a name to this env
  • [<packages_list>] optional preinstalled packages

Example:

$ conda create -n meso_lab

We can specify the python version to use:

$ conda create -n meso_lab python=3.9

We can specify a package to install after environment creation operation:

$ conda create -n meso_lab pandas

This will create an environment with package pandas installed.

Activate an environment​

To be able to install packages in the environment and so use them, you simply have to activate the environment using this command:

$ conda activate <env_name>

For example:

[login@host ~]$ conda activate meso_lab
(meso_lab) [login@host ~]$

So you can easily see which environment you are in or whether you are in an environment or not.

Deactivate an environment​

To deactivate an environment, simply execute:

$ conda deactivate

Install packages in environment​

Install packages with conda​

As seen before, you can directly create an environment with some packages.

But you can also install packages in environment, don't forget to activate it before ! (if it is not activated, you can however specify in which environment to install the packages with -n argument)

$ conda install [-n <env_name>] <packages_list>

For example:

[login@host ~]$ conda activate meso_lab
(meso_lab) [login@host ~]$ conda install numpy

To view installed packages:

(meso_lab) [login@host ~]$ conda list

Install packages with pip​

  • Some packages are not available for conda or they are outdated, the workaround is to install them via pip
  • pip MUST BE installed via conda otherwise an error will occur.
  • Install pip inside conda env:
(meso_lab) [login@host]$ conda install pip future

To be sure you have pip and future in your environment you can list packages with conda list.

(meso_lab) [login@host ~]$ conda list
# packages in environment at /users/login/.conda/envs/meso_lab:
#
#
<deleted lines>
 
pip                       21.2.4           py39h06a4308_0  
 
<deleted lines>

The command to install packages with pip is the same as with conda, for example:

(meso_lab) [login@host ~]$ pip install numpy

Manage environments​

Export​

Finally, when your environment is ready, you may want to share it with others or reuse it.

You can export an environment using conda env export command, you can export the current environment if activated, or another with -n argument.

By default, the output is directly printed to the terminal, but it is possible to export to a file using -f argument.

For example:

(meso_lab) [login@host ~]$ cd WORK
(meso_lab) [login@host WORK]$ conda env export -f environment.yml

Create from file​

To create an environment using this file, assuming the environment does not exist:

[login@host WORK]$ conda env create -f environment.yml

Update from file​

If the environment already exists, you can also update it with this file.

The environment.yml file contains the name of the exported environment, so if you want to install the same packages in another existing environment, you have to specify it with -n option.

[login@host WORK]$ conda env update -n test_terminal -f environment.yml

List​

You can list your environments using conda env list command, so no need to remember all the names:

[login@host ~]$ conda env list
# conda environments:
#
base                     /soft/spack/opt/spack/linux-rocky8-x86_64/gcc-13.1.0/miniconda3-22.11.1-pjlkuddflzsudqp4nwjdg4wpbgt6jdev
meso_lab                 /users/login/.conda/envs/meso_lab

Remove​

To avoid having too many environments, you can remove one with conda env remove command specifying which environment you want to remove with -n option.

[login@host ~]$ conda env remove -n test_terminal

Remove all packages in environment /users/login/.conda/envs/test_terminal:

[login@host ~]$

A practical example​

  • Create conda env with python 3.9 version
  • Install pandas package
  • Run python program within slurm
$ conda create -n py_post python=3.9
$ conda install -n py_post pandas

Add instructions to Slurm script

module load miniconda3-22.11.1
conda activate py_post
python myPandaProgram.py