From 69b09d5e0edc6192a8aaf549ac570979a66bd350 Mon Sep 17 00:00:00 2001 From: Pavel Semyonov Date: Thu, 8 Feb 2024 11:15:19 +0700 Subject: [PATCH] Update tt configuration and env description to 2.0 (#4033) Resolves #3997 Resolves #3998 Resolves #3836 --- doc/book/admin/logs.rst | 56 +++--- .../tooling/tt_cli/configuration.rst | 182 ++++++++++++------ .../tooling/tt_cli/global_options.rst | 11 +- doc/reference/tooling/tt_cli/logrotate.rst | 9 +- 4 files changed, 171 insertions(+), 87 deletions(-) diff --git a/doc/book/admin/logs.rst b/doc/book/admin/logs.rst index 176948fa77..e12b48aaf8 100644 --- a/doc/book/admin/logs.rst +++ b/doc/book/admin/logs.rst @@ -7,7 +7,7 @@ Each Tarantool instance logs important events to its own log file ```, the log location is defined by the ``log_dir`` parameter in the :ref:`tt configuration `. By default, it's ``/var/log/tarantool`` in the ``tt`` :ref:`system mode `, -and the ``var/log/`` subdirectory of the ``tt`` working directory in the :ref:`local mode `. +and the ``var/log`` subdirectory of the ``tt`` working directory in the :ref:`local mode `. In the specified location, ``tt`` creates separate directories for each instance's logs. To check how logging works, write something to the log using the :ref:`log ` module: @@ -41,31 +41,35 @@ Then check the logs: Log rotation ------------ -When :ref:`logging to a file `, the system administrator must ensure logs are -rotated timely and do not take up all the available disk space. -To prevent log files from growing infinitely, ``tt`` automatically rotates instance -logs. The following ``tt`` configuration parameters define the log rotation: -``log_maxsize`` (in megabytes) and ``log_maxage`` (in days). When any of these -limits is reached, the log is rotated. -Additionally, there is the ``log_maxbackups`` parameter (the number of stored log -files for an instance), which enables automatic removal of old log files. - -.. code-block:: yaml - - # tt.yaml - tt: - app: - log_maxsize: 100 - log_maxage: 3 - log_maxbackups: 50 - # ... - -There is also the :ref:`tt logrotate ` command that performs log -rotation on demand. - -.. code-block:: bash - - tt logrotate my_app +When :ref:`logging to a file `, the system administrator must ensure +logs are rotated timely and do not take up all the available disk space. +The recommended way to prevent log files from growing infinitely is using an external +log rotation program, for example, ``logrotate``, which is pre-installed on most +mainstream Linux distributions. + +A Tarantool log rotation configuration for ``logrotate`` can look like this: + +.. code-block:: text + + /var/log/tarantool/*.log { + daily + size 512k + missingok + rotate 10 + compress + delaycompress + create 0640 tarantool adm + postrotate + tt -c logrotate + endscript + } + +In this configuration, :ref:`tt logrotate ` is called after each log +rotation to reopen the instance log files after they are moved by the ``logrotate`` +program. + +There is also the built-in function :ref:`log.rotate() `, which you +can call on an instance to reopen its log file after rotation. To learn about log rotation in the deprecated ``tarantoolctl`` utility, check its :ref:`documentation `. diff --git a/doc/reference/tooling/tt_cli/configuration.rst b/doc/reference/tooling/tt_cli/configuration.rst index 807c2718d2..ff76543822 100644 --- a/doc/reference/tooling/tt_cli/configuration.rst +++ b/doc/reference/tooling/tt_cli/configuration.rst @@ -14,36 +14,63 @@ execution is its configuration file. By default, the configuration file is called ``tt.yaml``. The location where ``tt`` searches for it depends on the :ref:`launch mode `. You can also pass the configuration file explicitly in the ``--cfg`` -:ref:`option `. +:ref:`global option `. -The ``tt`` configuration file is a YAML file with the following content: +The ``tt`` configuration file is a YAML file with the following structure: .. code-block:: yaml - tt: - modules: - directory: path/to/modules/dir - app: - instances_enabled: path/to/applications - run_dir: path/to/run_dir - log_dir: path/to/log_dir - bin_dir: path/to/bin_dir - inc_dir: path/to/inc_dir - wal_dir: path/to/wal_dir - vinyl_dir: path/to/vinyl_dir - memtx_dir: path/to/memtx_dir - log_maxsize: num (MB) - log_maxage: num (days) - log_maxbackups: num - restart_on_failure: bool - repo: - rocks: path/to/rocks - distfiles: path/to/install - ee: - credential_path: path/to/file - templates: - - path: path/to/app/templates1 - - path: path/to/app/templates2 + env: + instances_enabled: path/to/available/applications + bin_dir: path/to/bin_dir + inc_dir: path/to/inc_dir + restart_on_failure: bool + tarantoolctl_layout: bool + modules: + directory: path/to/modules/dir + app: + run_dir: path/to/run_dir + log_dir: path/to/log_dir + wal_dir: path/to/wal_dir + vinyl_dir: path/to/vinyl_dir + memtx_dir: path/to/memtx_dir + repo: + rocks: path/to/rocks + distfiles: path/to/install + ee: + credential_path: path/to/file + templates: + - path: path/to/app/templates1 + - path: path/to/app/templates2 + +.. note:: + + The ``tt`` configuration format and application layout have been changed in version + 2.0. Learn how to upgrade from earlier versions in :ref:`tt-config_migrating-from-1`. + +.. _tt-config_file_env: + +env section +~~~~~~~~~~~ + +.. note:: + + The paths specified in ``env.*`` parameters are relative to the current ``tt`` + environment's root. + +* ``instances_enabled`` -- the directory where :ref:`instances ` + are stored. Default: ``instances.enabled``. +* ``bin_dir`` -- the directory where binary files are stored. Default: ``bin``. +* ``inc_dir`` -- the base directory for storing header files. They will + be placed in the ``include`` subdirectory inside the specified directory. + Default: ``include``. +* ``restart_on_failure`` -- restart the instance on failure: ``true`` or ``false``. + Default: ``false``. +* ``tarantoolctl_layout`` -- use a layout compatible with the deprecated ``tarantoolctl`` + utility for artifact files: control sockets, ``.pid`` files, log files. + Default: ``false``. + +.. _tt-config_file_modules: modules section ~~~~~~~~~~~~~~~ @@ -56,15 +83,18 @@ modules section app section ~~~~~~~~~~~ -* ``instances_enabled`` -- the directory where :ref:`instances ` - are stored. +.. note:: + + The paths specified in ``app.*_dir`` parameters are relative to the application + location inside the ``instances.enabled`` directory specified in the ``env`` + configuration section. For example, the default location of the ``myapp`` + application's logs is ``instances.enabled/myapp/var/log``. + Inside this location, ``tt`` creates separate directories for each application + instance that runs in the current environment. + * ``run_dir``-- the directory for instance runtime artifacts, such as console sockets or PID files. Default: ``var/run``. * ``log_dir`` -- the directory where log files are stored. Default: ``var/log``. -* ``bin_dir`` -- the directory where binary files are stored. Default: ``bin``. -* ``inc_dir`` -- the base directory for storing header files. They will - be placed in the ``include`` subdirectory inside the specified directory. - Default: ``include``. * ``wal_dir`` -- the directory where write-ahead log (``.xlog``) files are stored. Default: ``var/lib``. * ``memtx_dir`` -- the directory where memtx stores snapshot (``.snap``) files. @@ -72,28 +102,6 @@ app section * ``vinyl_dir`` -- the directory where vinyl files or subdirectories are stored. Default: ``var/lib``. - .. note:: - - In all directories specified in ``*_dir`` parameters, ``tt`` creates a - directory for each application and instance directories inside it. - Names of these directories match the names of applications and instances. - -* ``log_maxsize`` -- the maximum size of the log file before it gets rotated, - in megabytes. Default: 100. -* ``log_maxage`` -- the maximum age of log files in days. The age of a log - file is defined by the timestamp encoded in its name. Default: not defined - (log files aren't deleted based on their age). - - .. note:: - - A day is defined as exactly 24 hours. It may not exactly correspond to - calendar days due to daylight savings, leap seconds, and other time adjustments. - -* ``log_maxbackups`` -- the maximum number of stored log files. - Default: not defined (log files aren't deleted based on their count). -* ``restart_on_failure`` -- restart the instance on failure: ``true`` or ``false``. - Default: ``false``. - .. _tt-config_file_repo: repo section @@ -131,7 +139,7 @@ configuration file. There are three launch modes: Default launch ~~~~~~~~~~~~~~ -**Argument**: none +**Global option**: none **Configuration file**: searched from the current directory to the root. Taken from ``/etc/tarantool`` if the file is not found. @@ -143,7 +151,7 @@ Taken from ``/etc/tarantool`` if the file is not found. System launch ~~~~~~~~~~~~~ -**Argument**: ``--system`` or ``-S`` +**Global option**: ``--system`` or ``-S`` **Configuration file**: Taken from ``/etc/tarantool``. @@ -154,10 +162,68 @@ System launch Local launch ~~~~~~~~~~~~ -**Argument**: ``--local=DIRECTORY`` or ``-L=DIRECTORY`` +**Global option**: ``--local=DIRECTORY`` or ``-L=DIRECTORY`` **Configuration file**: Searched from the specified directory to the root. Taken from ``/etc/tarantool`` if the file is not found. **Working directory**: The specified directory. If ``tarantool`` or ``tt`` -executable files are found in the working directory, they will be used. \ No newline at end of file +executable files are found in the working directory, they will be used. + +.. _tt-config_migrating-from-1: + +Migrating from tt 1.* to 2.0 or later +------------------------------------- + +The `tt` configuration and application layout were changed in version 2.0. +If you are using ``tt`` 1.*, complete the following steps to migrate to ``tt`` 2.0 or later: + +#. **Update the tt configuration file**. + In tt 2.0, the following changes were made to the configuration file: + + * The root section ``tt`` was removed. Its child sections -- ``app``, ``repo``, + ``modules``, and other -- have been moved to the top level. + * Environment configuration parameters were moved from the ``app`` section + to the new section ``env``. These parameters are ``instances.enabled``, + ``bin_dir``, ``inc_dir``, and ``restart_on_failure``. + * The paths in the ``app`` section are now relative to the app directory in ``instances.enabled`` + instead of the environment root. + + You can use :ref:`tt init ` to generate a configuration file with + the new structure and default parameter values. + +#. **Move application artifacts**. + With ``tt`` 1.*, application artifacts (logs, snapshots, pid, and other files) + were created in the ``var`` directory inside the *environment root*. Starting from + ``tt`` 2.0, these artifacts are created in the ``var`` directory inside the + *application directory*, which is ``instances.enabled/``. This is + how an application directory looks: + + .. code-block:: text + + instances.enabled/app/ + ├── init.lua + ├── instances.yml + └── var + ├── lib + │ ├── instance1 + │ └── instance2 + ├── log + │ ├── instance1 + │ └── instance2 + └── run + ├── instance1 + └── instance2 + + To continue using existing application artifacts after migration from ``tt`` 1.*: + + #. Create the ``var`` directory inside the application directory. + #. Create the ``lib``, ``log``, and ``run`` directories inside ``var``. + #. Move directories with instance artifacts from the old ``var`` directory + to the new ``var`` directories in applications' directories. + +#. **Move the files accessed from the application code**. + The working directory of instance processes was changed from the ``tt`` working + directory to the application directory inside ``instances.enabled``. If the + application accesses files using relative paths, move the files accordingly + or adjust the application code. \ No newline at end of file diff --git a/doc/reference/tooling/tt_cli/global_options.rst b/doc/reference/tooling/tt_cli/global_options.rst index 3d108033f3..2f099557af 100644 --- a/doc/reference/tooling/tt_cli/global_options.rst +++ b/doc/reference/tooling/tt_cli/global_options.rst @@ -3,9 +3,18 @@ Global options ============== +.. important:: + + Global options of ``tt`` must be passed before its commands and other options. + For example: + + .. code-block:: console + + $ tt --cfg tt-conf.yaml start app + ``tt`` has the following global options: -.. option:: -c=FILE, --cfg=FILE +.. option:: -c=FILE, --cfg=FILE, Path to the :ref:`configuration file `. diff --git a/doc/reference/tooling/tt_cli/logrotate.rst b/doc/reference/tooling/tt_cli/logrotate.rst index bfb7a68bf0..d4de9906ba 100644 --- a/doc/reference/tooling/tt_cli/logrotate.rst +++ b/doc/reference/tooling/tt_cli/logrotate.rst @@ -7,13 +7,18 @@ Rotating instance logs $ tt logrotate {INSTANCE | APPLICATION[:APP_INSTANCE]} -``tt logrotate`` rotates logs of a specified Tarantool instance. +``tt logrotate`` rotates logs of a Tarantool application or specific instances, +and the ``tt`` log. For example, you need to call this function to continue logging +after a log rotation program renames or moves instances' logs. Learn more about :ref:`rotating logs `. +Calling ``tt logrotate`` on an application has the same effect as executing the +built-in :ref:`log.rotate() ` function on all its instances. + Examples -------- -Rotate logs of the ``app`` instance: +Rotate logs of the ``app`` application's instances: .. code-block:: console