Merge pull request #7597 from romayalon/romy-docs-improvements

NSFS | NC | Docs improvements
noobaa · Nov 20, 2023 · d7b1d6a · d7b1d6a
2 parents c4f2ff5 + 82c784a
commit d7b1d6a
Show file tree

Hide file tree

Showing 4 changed files with 129 additions and 52 deletions.
diff --git a/docs/nsfs-standalone.md → docs/design/NonContainerizedNSFSDesign.md b/docs/nsfs-standalone.md → docs/design/NonContainerizedNSFSDesign.md
diff --git a/docs/dev_guide/NonContainerizedDeveloperCustomizations.md b/docs/dev_guide/NonContainerizedDeveloperCustomizations.md
@@ -0,0 +1,64 @@
+# Non Containerized NSFS - Developer customization (OPTIONAL)
+
+The following list consists of supported optional developer customization  -
+
+## 1. Number of forks - 
+
+**Description -** Adjust the number of forks in order to increase the S3 endpoints number. This will enable NooBaa  to handle a higher scale of S3 requests concurrently.
+
+**Environemnt Variable -** ENDPOINT_FORKS
+
+**Steps -**
+```
+1. Open the /etc/noobaa.conf.d/.env file.
+2. Set the ENDPOINT_FORKS variable to the desired level.
+Example:
+ENDPOINT_FORKS=8
+```
+
+## 2. Log debug level -
+
+**Description -** Set the debug level to control the amount of debugging information generated by the application.
+
+Supported log debug levels:
+1. default - for default debugging
+2. warn - for only warning/errors debugging. 
+3. nsfs - for more nsfs detailed debugging.
+4. all - for all detailed debugging.
+
+
+**Environemnt Variable -** NOOBAA_LOG_LEVEL
+
+**Steps -**
+```
+1. Open the /etc/noobaa.conf.d/.env file.
+2. Set the NOOBAA_LOG_LEVEL variable to the desired level.
+Example:
+NOOBAA_LOG_LEVEL=nsfs
+```
+
+## 3. Ports -
+
+**Description -** Adjust the ports used by the application for communication
+
+The following S3 endpoint ports can be customized:
+1. ENDPOINT_PORT - S3 HTTP port 
+2. ENDPOINT_SSL_PORT - S3 HTTPS port 
+3. ENDPOINT_SSL_PORT_STS - STS HTTPS port 
+4. EP_METRICS_SERVER_PORT - prometheus metrics port 
+
+
+**Environemnt Variables -** ENDPOINT_PORT, ENDPOINT_SSL_PORT, ENDPOINT_SSL_PORT_STS, EP_METRICS_SERVER_PORT
+
+**Steps -**
+```
+1. Open the /etc/noobaa.conf.d/.env file.
+2. Set the environment variable.
+Example:
+ENDPOINT_PORT=5555
+```
+
+## 4. Allow http -
+**Description -** Enable or disable HTTP support for your application. 
+
+// TODO: step should move from system.json to config.json
diff --git a/docs/standalone.md → docs/dev_guide/standalone_with_db.md b/docs/standalone.md → docs/dev_guide/standalone_with_db.md
diff --git a/docs/non_containerized_NSFS.md b/docs/non_containerized_NSFS.md
@@ -42,36 +42,51 @@ yum install wget
 ```
 2. Download and install boost -
 ```sh
+// RHEL8/centos:stream8
 wget https://rpmfind.net/linux/centos/8-stream/AppStream/x86_64/os/Packages/boost-system-1.66.0-13.el8.x86_64.rpm
 wget https://rpmfind.net/linux/centos/8-stream/AppStream/x86_64/os/Packages/boost-thread-1.66.0-13.el8.x86_64.rpm
 rpm -i boost-system-1.66.0-13.el8.x86_64.rpm
 rpm -i boost-thread-1.66.0-13.el8.x86_64.rpm
 ```
 
+```sh
+// RHEL9/centos:stream9
+wget https://rpmfind.net/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-system-1.75.0-8.el9.x86_64.rpm
+wget https://rpmfind.net/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-thread-1.75.0-8.el9.x86_64.rpm
+rpm -i boost-system-1.75.0-8.el9.x86_64.rpm
+rpm -i boost-thread-1.75.0-8.el9.x86_64.rpm
+```
+
+### Download NooBaa RPM - 
+Nightly RPM builds of the upstream master branch can be downloaded from a public S3 bucket by running the following command - 
+
+```sh
+wget  noobaa-core-{VERSION}-{DATE}.el9.x86_64.rpm // Replace the VERSION and DATE
+
+Example:
+wget https://noobaa-core-rpms.s3.amazonaws.com/noobaa-core-5.15.0-20231106.el9.x86_64.rpm
+```
+
 ### Install NooBaa RPM - 
-Download the RPM to the machine and install it by running the following command - 
+Install NooBaa RPM by running the following command - 
 
 ```sh
-rpm -i noobaa-core-5.14.0-1.el8.x86_64.rpm
+rpm -i <rpm_file_name>.rpm
 ```
 
 After installing NooBaa RPM, it's expected to have noobaa-core source code under /usr/local/noobaa-core and an nsfs systemd example script under /etc/systemd/system/.
 
 ## Create configuration files -
-**IMPORTANT NOTE** - It's not recommended to create the config_root under /tmp/ since the contents of /tmp/ can be deleted occasionaly, In the following instruction we use it just as an example. 
+**IMPORTANT NOTE** - It's not mendatory to create the config_root under /etc/noobaa.conf.d/. config_dir path can be set using an CONFIG_JS_NSFS_NC_DEFAULT_CONF_DIR.
 
 **1. Create buckets and accounts directories -**
 ```sh
-mkdir -p /tmp/noobaa_config_root/
-mkdir -p /tmp/noobaa_config_root/buckets/
-mkdir -p /tmp/noobaa_config_root/accounts/
-```
-
-**2. Create accounts and exported buckets configuration files -**
-
-Find instruction at - https://github.com/noobaa/noobaa-core/blob/master/docs/nsfs-standalone.md. <br />
-**Note** - All required paths on the configuration files (bucket - path, account - new_buckets_path) must be absolute paths.
+mkdir -p /etc/noobaa.conf.d/
+mkdir -p /etc/noobaa.conf.d/buckets/
+mkdir -p /etc/noobaa.conf.d/accounts/
+mkdir -p /etc/noobaa.conf.d/access_keys/
 
+```
 
 **3. Create env file under the configuration directory -**
 
@@ -97,60 +112,40 @@ The systemd script runs noobaa non containerized, and requires config_root in or
 systemctl start nsfs
 ```
 
+## Developer customization of the nsfs service (OPTIONAL) - 
+The following list consists of supported optional developer customization -
+1. Number of forks 
+2. Log debug level
+3. Ports
+4. Allow http
+
+For more details see - [Non Containerized NSFS Developer Customization](https://github.com/noobaa/noobaa-core/blob/master/docs/dev_guide/NonContainerizedDeveloperCustomizations.md)
+
+## Create accounts and exported buckets configuration files - ##
+
+In order to create accounts and exported buckets see the management CLI instructions - [Bucket and Account Manage CLI](#bucket-and-account-manage-cli) <br />
+Design of Accounts and buckets configuration entities - [NonContainerizedNSFS](https://github.com/noobaa/noobaa-core/blob/master/docs/design/NonContainerizedNSFSDesign.md). <br />
+**Note** - All required paths on the configuration files (bucket - path, account - new_buckets_path) must be absolute paths.
+
+
 ## NSFS service logs -
 Run the following command in order to get the nsfs service logs - 
 
 ```sh
 journalctl -u nsfs.service
 ```
 
-## Test 
-#### 1. Create a bucket directory -
-```sh
-mkdir -p /tmp/fs1/bucket1/
-```
-
-#### 2. List the buckets -
-```sh
- curl http://localhost:6001
-```
-The following is an expected valid response - 
-```xml
-<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>123</ID><DisplayName>NooBaa</DisplayName></Owner><Buckets><Bucket><Name>bucket1</Name><CreationDate>2023-08-30T17:12:29.000Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>
-```
 
 ## S3 Test
 
-#### 1. Create account configuration file -
-
-```sh
-account1_json='
-{
-    "name": "account1",
-    "email": "[email protected]",
-    "has_login": "false",
-    "has_s3_access": "true",
-    "allow_bucket_creation": "true",
-    "access_keys": [{
-        "access_key": "abc",
-        "secret_key": "123"
-    }],
-    "nsfs_account_config": {
-        "uid": 0,
-        "gid": 0,
-        "new_buckets_path": "/tmp/fs1/",
-        "nsfs_only": "true"
-    }
-}'
-
-cat $account1_json >/tmp/noobaa_config_dir/accounts/abc.json
-```
+#### 1. Create account -
+see the management CLI instructions for creating an account - [Bucket and Account Manage CLI](#bucket-and-account-manage-cli)
 
 #### 2. Install aws cli -
 see https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
 
 
-#### 3. Create s3 alias for account1 -
+#### 3. Create s3 alias for the new account (to be referenced as account1) -
 ```sh
 alias s3-account1='AWS_ACCESS_KEY_ID=abc AWS_SECRET_ACCESS_KEY=123 aws --endpoint https://localhost:6443 --no-verify-ssl s3'
 ```
@@ -221,12 +216,30 @@ Output -
 
 
 
-## health
+## Health script
 Health status of the NSFS can be fetched using the command line.
  ```
  node usr/local/noobaa-core/src/cmd/health
  ```
 
+ Valid example output of a health script run - 
+ ```
+ {
+  service_name: 'nsfs', 
+  status: 'OK', 
+  memory: '137.9M', 
+  checks: { 
+    service: { 
+      service_status: 'active', 
+      pid: '90743' 
+    }, 
+    endpoint: { 
+      endpoint_response: 200 
+    } 
+  } 
+}
+ ```
+
 ## Bucket and Account Manage CLI
 Users can create, update, delete, and list buckets and accounts using CLI. If the config directory is missing CLI will create one and also create accounts and buckets sub-directories in it and default config directory is `/etc/noobaa.conf.d`.