---
jupytext:
  formats: md:myst
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
kernelspec:
  display_name: Python 3
  language: python
  name: python3
---

# The standard library

The [Python standard library](https://docs.python.org/3/library/) (see also
[this tuto](https://docs.python.org/3/tutorial/stdlib.html)) is a quite impressive set of
packages useful for many many things. These packages are included natively in Python.
They are very robust. Here is a small list:

- [math](doc-py-stdlib:math) - Mathematical functions
- [sys](doc-py-stdlib:sys) - System-specific parameters and functions (a lot about the
  Python system)
- [copy](doc-py-stdlib:copy) - Shallow and deep copy operations
- [os](doc-py-stdlib:os) - Miscellaneous operating system interfaces
- [shutil](doc-py-stdlib:shutil) - High-level file operations
- [pdb](doc-py-stdlib:pdb) - activate the python debugger
- [subprocess](doc-py-stdlib:subprocess) - Subprocesses with accessible I/O streams
- [datetime](doc-py-stdlib:datetime) - Basic date and time types
- [pickle](doc-py-stdlib:pickle) - Python object serialization
- [re](doc-py-stdlib:re) - Regular expressions
- [argparse](doc-py-stdlib:argparse) - Parser for command-line options, arguments and
  sub-commands
- [unittest](doc-py-stdlib:unittest) - Unit testing framework
- [logging](doc-py-stdlib:logging) - Event logging system
- [platform](doc-py-stdlib:platform) - Access to underlying platform’s identifying data
- [threading](doc-py-stdlib:threading) - Higher-level threading interface
- [multiprocessing](doc-py-stdlib:multiprocessing) - Process-based “threading” interface

In the following we present few very common packages. For some packages we also mention
very popular third-party projects which are interesting alternatives.

## Presentation of few very common packages

### math - Mathematical functions

For example to use $\pi$ in an environment where Numpy might not be installed:

```{code-cell}
import math

print(type(math))
```

```{code-cell}
from math import cos

print("pi is approximately equal to     ", math.pi)
print("cos(pi) is approximately equal to", cos(math.pi))
```

### pprint - Pretty-print lists, tuples, & dictionaries recursively

```{code-cell}
from pprint import pprint
```

### sys - System-specific parameters and functions (a lot about the Python system)

If you want to know where Python looks for module during the import statements, you can
do

```{code-cell}
import sys

pprint(sys.path)
```

### os: Miscellaneous operating system interfaces

os is a very important module.

```{code-cell}
import os

os.getcwd()
```

The [`os.path` module](doc-py-stdlib:os.path) is the historical module to work with paths
towards files and directories.

### pathlib: Object-oriented filesystem paths

A modern (Python 3) and nicer method to manipulate file paths.

```{code-cell}
from pathlib import Path
```

```{code-cell}
path_tmp = Path("/tmp/tmp_dir_example_python")
print(path_tmp.exists())
path_tmp.mkdir(exist_ok=True)

for i in range(4):
    (path_tmp / f"tmp{i}.xml").touch()
    (path_tmp / f"tmp{i}.txt").touch()

pprint(sorted(path_tmp.glob("*.txt")))
```

```{note}
Code using `libpath.Path` is often nicer than the equivalent using `os.path`
and `glob`.
```

### shutil - High-level file operations

Copy of files and directories can be done with shutil, in particular with
`shutil.copytree`.

`shutil.rmtree(path_tmp, ignore_errors=True)`

### pdb: useful to debug code

On a script:

1. import pdb
2. write pdb.set_trace() to set up a breakpoint
3. run the script

At execution time, the script will stop at the first line containing pdb.set_trace() and
gives the user access to the interpreter.

Remarks:

- even nicer: `ipdb` (but not part of the standard library).

- even nicer: `breakpoint()` built-in function in Python 3.7.

:::{admonition} Third party alternative
We have to mention [ipdb](pypi-project:ipdb).
:::

### subprocess

subprocess is very important since it is the simple way to launch other programs and bash
commands from Python. For example, in order to run bash (and not sh) commands, you can do

```{code-cell}
import subprocess


def call_bash(commands):
    return subprocess.call(["/bin/bash", "-c", commands])


ret = call_bash(
    """
echo The files in this directory are:; ls
"""
)
if ret == 0:
    print("command succeed")
else:
    print(f"command failed with return code {ret}")
```

### argparse - Parser for command-line options, arguments and sub-commands

argparse is the right tool to develop a command line script with options and help.
Example from the tutorial at https://docs.python.org/3/howto/argparse.html :

```python3
# File prog.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("echo", help="echo the string you use here")
args = parser.parse_args()
print(args.echo)
```

#### Usage :

```bash
$ python3 prog.py
usage: prog.py [-h] echo
prog.py: error: the following arguments are required: echo
$ python3 prog.py --help
usage: prog.py [-h] echo

positional arguments:
  echo        echo the string you use here

optional arguments:
  -h, --help  show this help message and exit
$ python3 prog.py foo
foo
```

:::{admonition} Third party alternative
We have to mention [click](pypi-project:click) and [typer](pypi-project:typer).
:::

### logging - Event logging system

logging allows the programmer to print (or not) different levels of messages.

```{code-cell}
import logging

log_level = logging.INFO  # to get information messages
# log_level = logging.WARNING  # no information messages
logging.basicConfig(
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s", level=log_level
)

thing = "beer"
logging.info('Would you like to have a "%s"?', thing)
```

:::{todo} Check standard library
- Small examples from (or just mention) `sys`, `os/shutils`, `copy`, `pathlib`/`glob`
  `argparse`, `math`, `re`, `dataclasses`, `itertools`, `turtle`, `functools`,
  `unittest`, `collections`, `datetime`
- https://docs.python.org/3/tutorial/stdlib.html
- Mention good alternatives outside of the standard library (pytest, click,
  requests/urllib3, typing-extensions, python-dateutil/pendulum, fsspec, certifi, idna,
  ...)
:::