From 10f145af4eb189779fd0740db260b32af208869a Mon Sep 17 00:00:00 2001 From: Thomas Kluyver Date: Wed, 1 Nov 2023 11:11:16 +0000 Subject: [PATCH] Document karabo-bridge-serve-run command --- docs/cli.rst | 72 ++++++++++++++++++++++++++++++++++++++++++++++ docs/conf.py | 2 +- docs/streaming.rst | 16 +++++++---- 3 files changed, 84 insertions(+), 6 deletions(-) diff --git a/docs/cli.rst b/docs/cli.rst index a62cc479..07fb38bf 100644 --- a/docs/cli.rst +++ b/docs/cli.rst @@ -44,6 +44,75 @@ Check the structure of an EuXFEL run or HDF5 file: If it finds problems with the data, the program will produce a list of them and exit with status 1. See :doc:`validation` for details of what it checks. +.. _cmd-serve-run: + +``karabo-bridge-serve-run`` +--------------------------- + +Stream data from a run, by proposal & run number, in the `Karabo bridge +`_ +format. See :doc:`streaming` for more information. + +.. code-block:: shell + + # Proposal run + karabo-bridge-serve-run 700000 40 --port 4321 \ + --include 'SPB_IRDA_JF4M/DET/JNGFR*:daqOutput' \ + --include '*/MOTOR/*[*Position]' + +.. program:: karabo-bridge-serve-run + +.. option:: --port + + Either a numeric TCP port, e.g. ``4321``, or a ZMQ endpoint address, e.g. + ``tcp://0.0.0.0:4321``. You will need to give the receiving code this port + number or address as well. + + If no port is specified, the program will pick an unused port, and display + the endpoint address as it starts. + +.. option:: --include + + Sources matching the pattern will be included in the streamed data. + You can also match keys by putting a key pattern in ``[]`` square brackets + at the end of the source pattern. + + You must specify at least one ``--include`` pattern, and you can use the + option several times to expand the selection. + + If data is flowing slower than you expect, see if you can use more specific + patterns to avoid sending unnecessary sources. + +.. option:: --allow-partial + + By default, trains are only sent if they contain all the data selected by + ``--include``. This option also sends trains where some of that data is + missing. + +.. option:: --append-detector-modules + + If the file data contains multiple detector modules as separate sources, + i. e. for big area detectors (AGIPD, LPD and DSSC), append these into one + single source. + +.. option:: --dummy-timestamps + + Add mock timestamps if missing in the original meta-data. + +These two options above - appended module sources and dummy timestamps - are +required if streamed data shall be provided to OnDA. + +.. option:: -z , --socket-type + + The ZMQ socket type to use, one of ``PUB``, ``PUSH`` or ``REP``. + Default: ``REP``. + +.. option:: --use-infiniband + + Use the infiniband network interface (``ib0``) if it's present. + This is ignored if ``--port`` is used with a full ZMQ endpoint address. + + .. _cmd-serve-files: ``karabo-bridge-serve-files`` @@ -53,6 +122,9 @@ Stream data from files in the `Karabo bridge `_ format. See :doc:`streaming` for more information. +For streaming data from a run directory, we recommend the newer +:ref:`cmd-serve-run` command in place of this. + .. code-block:: shell karabo-bridge-serve-files /gpfs/exfel/exp/XMPL/201750/p700000/proc/r0005 4321 diff --git a/docs/conf.py b/docs/conf.py index cb8f06c0..aead9214 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -76,7 +76,7 @@ # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. -language = None +language = 'en' # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: diff --git a/docs/streaming.rst b/docs/streaming.rst index cacfa3e6..f0aaea06 100644 --- a/docs/streaming.rst +++ b/docs/streaming.rst @@ -6,14 +6,20 @@ socket. The ``extra_data`` Python package can stream data from files using the s protocol. You can use this to test code which expects to receive data from Karabo Bridge, or use the same code for analysing live data and stored data. -To stream the data from a file or run unmodified, use the command:: +To stream data from a saved run, use the ``karabo-bridge-serve-run`` command: - karabo-bridge-serve-files /gpfs/exfel/exp/SPB/201830/p900022/raw/r0034 4545 +.. code-block:: shell -The number (4545) must be an unused TCP port above 1024. It will bind to -this and stream the data to any connected clients. + # Proposal run + karabo-bridge-serve-run 700000 40 --port 4545 \ + --include 'SPB_IRDA_JF4M/DET/JNGFR*:daqOutput' \ + --include '*/MOTOR/*[*Position]' + +The port number (4545 above) must be an unused TCP port above 1024. +Clients will need this port and the IP address of the sender to connect. +For clients running on the same node, use the IP address ``127.0.0.1``. Command-line options are explained on the -:ref:`command reference ` page. +:ref:`command reference ` page. .. note::