From 82c784a05541b5adaa268f825f459fa95676ceb5 Mon Sep 17 00:00:00 2001 From: Romy <35330373+romayalon@users.noreply.github.com> Date: Tue, 14 Nov 2023 21:09:12 +0200 Subject: [PATCH] docs improvements Signed-off-by: Romy <35330373+romayalon@users.noreply.github.com> --- .../NonContainerizedNSFSDesign.md} | 0 ...NonContainerizedDeveloperCustomizations.md | 64 ++++++++++ .../standalone_with_db.md} | 0 docs/non_containerized_NSFS.md | 117 ++++++++++-------- 4 files changed, 129 insertions(+), 52 deletions(-) rename docs/{nsfs-standalone.md => design/NonContainerizedNSFSDesign.md} (100%) create mode 100644 docs/dev_guide/NonContainerizedDeveloperCustomizations.md rename docs/{standalone.md => dev_guide/standalone_with_db.md} (100%) diff --git a/docs/nsfs-standalone.md b/docs/design/NonContainerizedNSFSDesign.md similarity index 100% rename from docs/nsfs-standalone.md rename to docs/design/NonContainerizedNSFSDesign.md diff --git a/docs/dev_guide/NonContainerizedDeveloperCustomizations.md b/docs/dev_guide/NonContainerizedDeveloperCustomizations.md new file mode 100644 index 0000000000..b31cc33ffe --- /dev/null +++ b/docs/dev_guide/NonContainerizedDeveloperCustomizations.md @@ -0,0 +1,64 @@ +# Non Containerized NSFS - Developer customization (OPTIONAL) + +The following list consists of supported optional developer customization - + +## 1. Number of forks - + +**Description -** Adjust the number of forks in order to increase the S3 endpoints number. This will enable NooBaa to handle a higher scale of S3 requests concurrently. + +**Environemnt Variable -** ENDPOINT_FORKS + +**Steps -** +``` +1. Open the /etc/noobaa.conf.d/.env file. +2. Set the ENDPOINT_FORKS variable to the desired level. +Example: +ENDPOINT_FORKS=8 +``` + +## 2. Log debug level - + +**Description -** Set the debug level to control the amount of debugging information generated by the application. + +Supported log debug levels: +1. default - for default debugging +2. warn - for only warning/errors debugging. +3. nsfs - for more nsfs detailed debugging. +4. all - for all detailed debugging. + + +**Environemnt Variable -** NOOBAA_LOG_LEVEL + +**Steps -** +``` +1. Open the /etc/noobaa.conf.d/.env file. +2. Set the NOOBAA_LOG_LEVEL variable to the desired level. +Example: +NOOBAA_LOG_LEVEL=nsfs +``` + +## 3. Ports - + +**Description -** Adjust the ports used by the application for communication + +The following S3 endpoint ports can be customized: +1. ENDPOINT_PORT - S3 HTTP port +2. ENDPOINT_SSL_PORT - S3 HTTPS port +3. ENDPOINT_SSL_PORT_STS - STS HTTPS port +4. EP_METRICS_SERVER_PORT - prometheus metrics port + + +**Environemnt Variables -** ENDPOINT_PORT, ENDPOINT_SSL_PORT, ENDPOINT_SSL_PORT_STS, EP_METRICS_SERVER_PORT + +**Steps -** +``` +1. Open the /etc/noobaa.conf.d/.env file. +2. Set the environment variable. +Example: +ENDPOINT_PORT=5555 +``` + +## 4. Allow http - +**Description -** Enable or disable HTTP support for your application. + +// TODO: step should move from system.json to config.json diff --git a/docs/standalone.md b/docs/dev_guide/standalone_with_db.md similarity index 100% rename from docs/standalone.md rename to docs/dev_guide/standalone_with_db.md diff --git a/docs/non_containerized_NSFS.md b/docs/non_containerized_NSFS.md index e23e0a6c7a..6d79582386 100644 --- a/docs/non_containerized_NSFS.md +++ b/docs/non_containerized_NSFS.md @@ -42,36 +42,51 @@ yum install wget ``` 2. Download and install boost - ```sh +// RHEL8/centos:stream8 wget https://rpmfind.net/linux/centos/8-stream/AppStream/x86_64/os/Packages/boost-system-1.66.0-13.el8.x86_64.rpm wget https://rpmfind.net/linux/centos/8-stream/AppStream/x86_64/os/Packages/boost-thread-1.66.0-13.el8.x86_64.rpm rpm -i boost-system-1.66.0-13.el8.x86_64.rpm rpm -i boost-thread-1.66.0-13.el8.x86_64.rpm ``` +```sh +// RHEL9/centos:stream9 +wget https://rpmfind.net/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-system-1.75.0-8.el9.x86_64.rpm +wget https://rpmfind.net/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-thread-1.75.0-8.el9.x86_64.rpm +rpm -i boost-system-1.75.0-8.el9.x86_64.rpm +rpm -i boost-thread-1.75.0-8.el9.x86_64.rpm +``` + +### Download NooBaa RPM - +Nightly RPM builds of the upstream master branch can be downloaded from a public S3 bucket by running the following command - + +```sh +wget noobaa-core-{VERSION}-{DATE}.el9.x86_64.rpm // Replace the VERSION and DATE + +Example: +wget https://noobaa-core-rpms.s3.amazonaws.com/noobaa-core-5.15.0-20231106.el9.x86_64.rpm +``` + ### Install NooBaa RPM - -Download the RPM to the machine and install it by running the following command - +Install NooBaa RPM by running the following command - ```sh -rpm -i noobaa-core-5.14.0-1.el8.x86_64.rpm +rpm -i .rpm ``` After installing NooBaa RPM, it's expected to have noobaa-core source code under /usr/local/noobaa-core and an nsfs systemd example script under /etc/systemd/system/. ## Create configuration files - -**IMPORTANT NOTE** - It's not recommended to create the config_root under /tmp/ since the contents of /tmp/ can be deleted occasionaly, In the following instruction we use it just as an example. +**IMPORTANT NOTE** - It's not mendatory to create the config_root under /etc/noobaa.conf.d/. config_dir path can be set using an CONFIG_JS_NSFS_NC_DEFAULT_CONF_DIR. **1. Create buckets and accounts directories -** ```sh -mkdir -p /tmp/noobaa_config_root/ -mkdir -p /tmp/noobaa_config_root/buckets/ -mkdir -p /tmp/noobaa_config_root/accounts/ -``` - -**2. Create accounts and exported buckets configuration files -** - -Find instruction at - https://github.com/noobaa/noobaa-core/blob/master/docs/nsfs-standalone.md.
-**Note** - All required paths on the configuration files (bucket - path, account - new_buckets_path) must be absolute paths. +mkdir -p /etc/noobaa.conf.d/ +mkdir -p /etc/noobaa.conf.d/buckets/ +mkdir -p /etc/noobaa.conf.d/accounts/ +mkdir -p /etc/noobaa.conf.d/access_keys/ +``` **3. Create env file under the configuration directory -** @@ -97,6 +112,22 @@ The systemd script runs noobaa non containerized, and requires config_root in or systemctl start nsfs ``` +## Developer customization of the nsfs service (OPTIONAL) - +The following list consists of supported optional developer customization - +1. Number of forks +2. Log debug level +3. Ports +4. Allow http + +For more details see - [Non Containerized NSFS Developer Customization](https://github.com/noobaa/noobaa-core/blob/master/docs/dev_guide/NonContainerizedDeveloperCustomizations.md) + +## Create accounts and exported buckets configuration files - ## + +In order to create accounts and exported buckets see the management CLI instructions - [Bucket and Account Manage CLI](#bucket-and-account-manage-cli)
+Design of Accounts and buckets configuration entities - [NonContainerizedNSFS](https://github.com/noobaa/noobaa-core/blob/master/docs/design/NonContainerizedNSFSDesign.md).
+**Note** - All required paths on the configuration files (bucket - path, account - new_buckets_path) must be absolute paths. + + ## NSFS service logs - Run the following command in order to get the nsfs service logs - @@ -104,53 +135,17 @@ Run the following command in order to get the nsfs service logs - journalctl -u nsfs.service ``` -## Test -#### 1. Create a bucket directory - -```sh -mkdir -p /tmp/fs1/bucket1/ -``` - -#### 2. List the buckets - -```sh - curl http://localhost:6001 -``` -The following is an expected valid response - -```xml -123NooBaabucket12023-08-30T17:12:29.000Z -``` ## S3 Test -#### 1. Create account configuration file - - -```sh -account1_json=' -{ - "name": "account1", - "email": "account1@noobaa.io", - "has_login": "false", - "has_s3_access": "true", - "allow_bucket_creation": "true", - "access_keys": [{ - "access_key": "abc", - "secret_key": "123" - }], - "nsfs_account_config": { - "uid": 0, - "gid": 0, - "new_buckets_path": "/tmp/fs1/", - "nsfs_only": "true" - } -}' - -cat $account1_json >/tmp/noobaa_config_dir/accounts/abc.json -``` +#### 1. Create account - +see the management CLI instructions for creating an account - [Bucket and Account Manage CLI](#bucket-and-account-manage-cli) #### 2. Install aws cli - see https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html -#### 3. Create s3 alias for account1 - +#### 3. Create s3 alias for the new account (to be referenced as account1) - ```sh alias s3-account1='AWS_ACCESS_KEY_ID=abc AWS_SECRET_ACCESS_KEY=123 aws --endpoint https://localhost:6443 --no-verify-ssl s3' ``` @@ -221,12 +216,30 @@ Output - -## health +## Health script Health status of the NSFS can be fetched using the command line. ``` node usr/local/noobaa-core/src/cmd/health ``` + Valid example output of a health script run - + ``` + { + service_name: 'nsfs', + status: 'OK', + memory: '137.9M', + checks: { + service: { + service_status: 'active', + pid: '90743' + }, + endpoint: { + endpoint_response: 200 + } + } +} + ``` + ## Bucket and Account Manage CLI Users can create, update, delete, and list buckets and accounts using CLI. If the config directory is missing CLI will create one and also create accounts and buckets sub-directories in it and default config directory is `/etc/noobaa.conf.d`.