diff --git a/docs/NooBaaNonContainerized/Logging.md b/docs/NooBaaNonContainerized/Logging.md index 629de0a119..e1dbf48b66 100644 --- a/docs/NooBaaNonContainerized/Logging.md +++ b/docs/NooBaaNonContainerized/Logging.md @@ -19,10 +19,19 @@ This document provides an overview of the logging capabilities within NooBaa, de ## NooBaa Service Logs -NooBaa service logs are written directly to stderr and to syslog - -1. NooBaa writes the logs into stderr and to `/var/log/noobaa.log`. -2. The `journalctl` service watches stderr and writes it to `journal`. -3. A `syslog` service, like `rsyslog/syslog_ng` watches `journal` and writes it to `/var/log/messages/`. +NooBaa service logs are written directly to stderr, syslog, or both. +1. The `journalctl` service watches stderr and writes into `journal`. That internally updates the `/var/log/messages`. +2. Noobaa sends logs to a syslog service such as rsyslog or syslog_ng and writes the logs to `/var/log/noobaa.log` if enabled. We can not have more than one service enabled at a time. It might cause some issues. + +### Set log config + +Set `LOG_TO_STDERR_ENABLED` property to `true` to send Noobaa logs to stderr. Noobaa logs will be send to syslog when the property `LOG_TO_SYSLOG_ENABLED` set to `true`. + +Default configurations are: +``` +LOG_TO_STDERR_ENABLED = true; +LOG_TO_SYSLOG_ENABLED = false; +``` ### Journal logs 1. NooBaa service logs - @@ -42,7 +51,7 @@ NooBaa logs are configured using rsyslog and logrotate. NooBaa RPM will configur systemctl status rsyslog ``` -NooBaa logs are pushed to `var/log/noobaa.log` and the log is rotated and compressed daily. +NooBaa logs are pushed to `/var/log/noobaa.log` and the log is rotated and compressed daily. The following files contain NooBaa specific configuration for rsyslog and logrotate - 1. `etc/rsyslog.d/noobaa_syslog.conf` @@ -50,7 +59,7 @@ The following files contain NooBaa specific configuration for rsyslog and logrot Verify the rsyslog and logrotate rpm configuration is complete by checking the files above. -#### Logrotate +### Logrotate ##### Logrotate automatic rotation @@ -79,16 +88,58 @@ logrotate etc/logrotate.d/noobaa-logrotate Set `NOOBAA_LOG_LEVEL` property in config.json to control the amount of debugging information generated by the application. For more info about setting custom properties, see - [Non Containerized Config File Customizations](./ConfigFileCustomizations.md) -## NooBaa Logs format +### NooBaa Logs format NooBaa logs are formatted with ANSI color codes.
Use the `cat | less -R` command in order to display the contents of a log file with pagination and the ability to handle ANSI color codes. -### Set log config +## How to configure syslog-ng + +1. create syslog-ng noobaa specific config file `noobaa-syslog-ng.conf` under `etc/syslog-ng/conf.d/` +``` +vi etc/syslog-ng/conf.d/noobaa-syslog-ng.conf +``` +2. update the file with bellow configuration -Set `LOG_TO_STDERR_ENABLED` property to `true` to send Noobaa logs to stderr. Noobaa logs will be send to syslog when the property `LOG_TO_SYSLOG_ENABLED` set to `true`. +``` +destination d_noobaa_msg { file("/var/log/noobaa.log"); }; +destination d_noobaa_event { file("/var/log/noobaa_events.log"); }; -Default configurations are: +filter f_noobaa_msg { facility(local0); }; +filter f_noobaa_event { facility(local2); }; + +log { source(s_sys); filter(f_noobaa_msg); destination(d_noobaa_msg); }; +log { source(s_sys); filter(f_noobaa_event); destination(d_noobaa_event); }; ``` -LOG_TO_STDERR_ENABLED = true; -LOG_TO_SYSLOG_ENABLED = false; -``` \ No newline at end of file +We are creating two new destination one is `d_noobaa_msg`, point to log file `/var/log/noobaa.log` and another one is `d_noobaa_event` , point to log file `/var/log/noobaa_events.log` +Filter `f_noobaa_msg` and `f_noobaa_event` will filter out only noobaa logs and events log. Last created two source that will filter and log only noobaa normal logs and events to previously added destinations. + +3. Update existing filter `f_default` in default syslog-ng `etc/syslog-ng/syslog-ng.conf` to make sure noobaa logs and event are not send to `/var/log/message` log file. +``` +vi etc/syslog-ng/syslog-ng.conf +// update file for filter f_default +filter f_default { level(info..emerg) and + not (facility(mail) + or facility(local0) + or facility(local2) + or facility(authpriv) + or facility(cron)); }; +``` + Noobaa logs with facility local0 and local2 are excluded from message logs +4. restart syslog-ng +``` +systemctl restart syslog-ng +``` + +### Known issues + +1. missing logs due to systemd-journald log suppression + `Suppressed 6284127 messages from noobaa.service` + + solution : + * Edit /etc/systemd/journald.conf + * add below configuration + ``` + RateLimitInterval=0 + RateLimitBurst=0 + ``` + link : https://my.f5.com/manage/s/article/K70501143