Skip to content

Commit

Permalink
Document karabo-bridge-serve-run command
Browse files Browse the repository at this point in the history
  • Loading branch information
takluyver committed Nov 1, 2023
1 parent 73208fc commit 10f145a
Show file tree
Hide file tree
Showing 3 changed files with 84 additions and 6 deletions.
72 changes: 72 additions & 0 deletions docs/cli.rst
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,75 @@ Check the structure of an EuXFEL run or HDF5 file:
If it finds problems with the data, the program will produce a list of them and
exit with status 1. See :doc:`validation` for details of what it checks.

.. _cmd-serve-run:

``karabo-bridge-serve-run``
---------------------------

Stream data from a run, by proposal & run number, in the `Karabo bridge
<https://rtd.xfel.eu/docs/data-analysis-user-documentation/en/latest/online.html#streaming-from-karabo-bridge>`_
format. See :doc:`streaming` for more information.

.. code-block:: shell
# Proposal run
karabo-bridge-serve-run 700000 40 --port 4321 \
--include 'SPB_IRDA_JF4M/DET/JNGFR*:daqOutput' \
--include '*/MOTOR/*[*Position]'
.. program:: karabo-bridge-serve-run

.. option:: --port <port>

Either a numeric TCP port, e.g. ``4321``, or a ZMQ endpoint address, e.g.
``tcp://0.0.0.0:4321``. You will need to give the receiving code this port
number or address as well.

If no port is specified, the program will pick an unused port, and display
the endpoint address as it starts.

.. option:: --include <pattern>

Sources matching the pattern will be included in the streamed data.
You can also match keys by putting a key pattern in ``[]`` square brackets
at the end of the source pattern.

You must specify at least one ``--include`` pattern, and you can use the
option several times to expand the selection.

If data is flowing slower than you expect, see if you can use more specific
patterns to avoid sending unnecessary sources.

.. option:: --allow-partial

By default, trains are only sent if they contain all the data selected by
``--include``. This option also sends trains where some of that data is
missing.

.. option:: --append-detector-modules

If the file data contains multiple detector modules as separate sources,
i. e. for big area detectors (AGIPD, LPD and DSSC), append these into one
single source.

.. option:: --dummy-timestamps

Add mock timestamps if missing in the original meta-data.

These two options above - appended module sources and dummy timestamps - are
required if streamed data shall be provided to OnDA.

.. option:: -z <type>, --socket-type <type>

The ZMQ socket type to use, one of ``PUB``, ``PUSH`` or ``REP``.
Default: ``REP``.

.. option:: --use-infiniband

Use the infiniband network interface (``ib0``) if it's present.
This is ignored if ``--port`` is used with a full ZMQ endpoint address.


.. _cmd-serve-files:

``karabo-bridge-serve-files``
Expand All @@ -53,6 +122,9 @@ Stream data from files in the `Karabo bridge
<https://rtd.xfel.eu/docs/data-analysis-user-documentation/en/latest/online.html#streaming-from-karabo-bridge>`_
format. See :doc:`streaming` for more information.

For streaming data from a run directory, we recommend the newer
:ref:`cmd-serve-run` command in place of this.

.. code-block:: shell
karabo-bridge-serve-files /gpfs/exfel/exp/XMPL/201750/p700000/proc/r0005 4321
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
language = 'en'

# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
Expand Down
16 changes: 11 additions & 5 deletions docs/streaming.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,20 @@ socket. The ``extra_data`` Python package can stream data from files using the s
protocol. You can use this to test code which expects to receive data from
Karabo Bridge, or use the same code for analysing live data and stored data.

To stream the data from a file or run unmodified, use the command::
To stream data from a saved run, use the ``karabo-bridge-serve-run`` command:

karabo-bridge-serve-files /gpfs/exfel/exp/SPB/201830/p900022/raw/r0034 4545
.. code-block:: shell
The number (4545) must be an unused TCP port above 1024. It will bind to
this and stream the data to any connected clients.
# Proposal run
karabo-bridge-serve-run 700000 40 --port 4545 \
--include 'SPB_IRDA_JF4M/DET/JNGFR*:daqOutput' \
--include '*/MOTOR/*[*Position]'
The port number (4545 above) must be an unused TCP port above 1024.
Clients will need this port and the IP address of the sender to connect.
For clients running on the same node, use the IP address ``127.0.0.1``.
Command-line options are explained on the
:ref:`command reference <cmd-serve-files>` page.
:ref:`command reference <cmd-serve-run>` page.

.. note::

Expand Down

0 comments on commit 10f145a

Please sign in to comment.