docs: Document python bindings

Signed-off-by: Adrian Moreno <[email protected]>

RELEASE.md

@@ -6,7 +6,8 @@
    1. Add new authors to the authors file if needed, by running
       `./tools/authors.sh` and committing the changes.
    1. Update the version in `Cargo.toml`.
-   1. Update the version name in `src/main.rs`.
+   1. Update the version name in `retis/src/main.rs`.
+   1. Update the version name in `retis-events/pyproject.toml`.
    1. Run `cargo publish --dry-run` to check for any issue.
    1. Open a PR and get it merged. This must have runtime tests enabled!
    1. Tag the right commit in the `vx.y.z` form and push it.
@@ -22,6 +23,8 @@
       1. `$ buildah push quay.io/retis/retis:x.y.z`
       1. Manually tag on the web UI the image pushed with `latest`, if
          applicable.
+   1. Build and upload the python bindings.
+      1. `podman run --rm --env MATURIN_PYPI_TOKEN=$(cat ~/.my_pypi_token) -v $(pwd):/io:z ghcr.io/pyo3/maturin publish -m retis-events/Cargo.toml -F python-lib`  
 1. Release on [crates.io](https://crates.io): `cargo publish`.
 1. Write and publish a release notes in the GitHub interface. This must be done
    once the rpm and the image successfully built to allow pushing last minute

docs/CONTRIBUTING.md

@@ -122,6 +122,20 @@ flavor coding style for the BPF parts.
    latest `main`. This can happen at any time, e.g. when other pull-requests are
    being merged.
 
+### Python bindings development
+
+When changing python bindings, it can be useful to quickly build and install the
+current code into a virtual environment for testing. An easy way to do it is by
+using [maturin](https://github.com/PyO3/maturin):
+
+```
+$ python -m venv venv && source venv/bin/activate # jump into a virtual env (required)
+$ pip install maturin
+$ maturin develop -m retis-events/Cargo.toml -F python-lib
+$ python
+>>> import retis
+```
+
 ### Documentation preview
 
 HTTP documentation is automatically generated for releases and the

docs/index.md

@@ -121,6 +121,10 @@ $ PAGER=more retis sort
 $ NOPAGER=1 retis sort
 ```
 
+In addition to built-in post-processing commands, it is possible to use the
+python bindings to implement custom processing. See the
+[python bindings documentation](python.md).
+
 ## Profiles and customization
 
 Retis has the concept of profiles, which are a predefined set of cli arguments

docs/python.md

@@ -0,0 +1,56 @@
+# Python bindings
+
+Besides the basic post-processing commands provided by `retis` (e.g: `sort`),
+python bindings exist to enable writing custom post-processing scripts.
+
+These bindings can be used in two different ways: the built-in python shell and
+the external python library.
+
+## Overview
+
+Python bindings currently provide 3 basic python classes that allow inspecting
+retis events:
+
+- **Event**: Python representation of a retis event. It provides helpers to
+access the event's sections and data within those sections.
+- **EventSeries**: Python representation of a series of sorted events resulting
+from the execution of `retis sort -o`. It implements the iterator protocol to
+access the events.
+- **EventReader**: Class capable of reading a file created by retis and iterate
+over the events or series it contains. It implements the iterator protocol to
+access the events.
+
+More details can be found in the `retis_events` crate documentation.
+
+## Builtin shell
+
+The builtin shell, although basic, it enables quick event inspection.
+Once you drop into the shell, a global variable called `events` is available.
+It is of type `EventReader`.
+
+``` python
+$ retis python
+>>> for event in events:
+>>>     if "skb" in event and "tcp" in event["skb"].raw():
+>>>         print("TCP event with dport: {}".format(
+>>>             event["skb"].raw()["tcp"]["dport"])
+```
+
+## Python library
+
+For more sophisticated programs, a python library is available in
+[pypi](https://pypi.org/retis). Unlike the builtin command, in this case the
+`EventReader` has to be created manually.
+
+```python
+from retis import EventReader
+
+import statistics
+
+reader = EventReader("sorted_events.json")
+
+events_per_series = [len(s) for s in reader]
+
+print("Number of series: {}".format(len(events_per_series)))
+print("Average events per series: {}".format(statistics.mean(events_per_series)))
+```

mkdocs.yml

@@ -14,6 +14,7 @@ nav:
   - Requirements: requirements.md
   - Contributing: CONTRIBUTING.md
   - External resources: resources.md
+  - Python bindings: python.md
   - Collectors:
       - skb: modules/skb.md
       - skb-drop: modules/skb_drop.md