diff --git a/postgres/autobase-ha-cluster.md b/postgres/autobase-ha-cluster.md new file mode 100644 index 0000000..3d4f308 --- /dev/null +++ b/postgres/autobase-ha-cluster.md @@ -0,0 +1,364 @@ +Hi friends, my name is Nelson and this is dwyl.
+Today we're going to deploy a Postgres Database Cluster +on +[`Hetzner`] +using [**autobase**](https://autobase.tech). + +As always, detailed instructions +for how we do _everything_ are available on `GitHub`; +link in the description. 🔗 + +Along the way we will clarify the steps as possible. +But keep in mind it's _not possible_ to cover everything in a **7 minute video**. + +If you have questions, suggestions or just want to say hi, +**please comment on YouTube**; +thanks. + +With all that out of the way, lets dive in! + +## 1. Login to `Hetzner` Cloud + + +When you _first_ login to `Hetzner`, +you will see the message: + +"You don't have any servers yet." + +hetzner-no-servers + +Click the "**Add Server**" button to begin your quest! + +## 2. Create a New Server (VPS) + +Select all the default options, +add your `ssh` `public` key +and create your server. + +new-server-created + +## 3. `SSH` into the `Hetzner` Server + +Use your `Terminal` to login to the newly created `Hetzner`server, e.g: + +```sh +ssh root@116.202.31.52 +``` + +Once you have successfully connected via `ssh`, +a best practice we recommend is to run a quick update. + +### Update The Server + +Run the following command chain: + +```sh +sudo apt update -y && sudo apt full-upgrade -y && sudo apt autoremove -y && sudo apt clean -y && sudo apt autoclean -y +``` + +> Updates and installs usually take a couple of minutes. +> We speed installs up for brevity. + +With everything up-to-date, install the necessary dependencies. + + +## 4. Install Dependencies + +As per the `autobase` getting started guide: +[autobase.tech/docs#getting-started](https://autobase.tech/docs#getting-started) +run the following command to get the necessary dependencies: + +```sh +sudo apt update && sudo apt install -y python3-pip sshpass git +pip3 install ansible +``` + +### Install `Docker` + +The `Ubuntu` Server + +Follow the installation instructions in the **official `Docker` docs**: +https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository + +Run: + +```sh +# Add Docker's official GPG key: +sudo apt-get update +sudo apt-get install ca-certificates curl +sudo install -m 0755 -d /etc/apt/keyrings +sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc +sudo chmod a+r /etc/apt/keyrings/docker.asc + +# Add the repository to Apt sources: +echo \ + "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ + $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ + sudo tee /etc/apt/sources.list.d/docker.list > /dev/null +sudo apt-get update +``` + +Followed by: +```sh +sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin +``` + +Verify that the installation is successful by running the `hello-world` image: + +```sh +$ sudo docker run hello-world +``` + +With that confirmed working, +go back to the previous step and run the `autobase`command. + + + +## 5. Run `autobase` Console Boot Script + +Sample: + +```sh +docker run -d --name autobase-console \ + --publish 80:80 \ + --publish 8080:8080 \ + --env PG_CONSOLE_API_URL=http://localhost:8080/api/v1 \ + --env PG_CONSOLE_AUTHORIZATION_TOKEN=secret_token \ + --env PG_CONSOLE_DOCKER_IMAGE=autobase/automation:latest \ + --volume console_postgres:/var/lib/postgresql \ + --volume /var/run/docker.sock:/var/run/docker.sock \ + --volume /tmp/ansible:/tmp/ansible \ + --restart=unless-stopped \ + autobase/console:latest +``` + +You will nee to replace the `localhost` in the `PG_CONSOLE_API_URL` +with the IP (v4) address of your server +and `secret_token`in the `PG_CONSOLE_AUTHORIZATION_TOKEN` + ++ IP: 116.202.31.52 (yours will be different!) ++ Token: 5b0b6259-a7d4-4435-947d-0dff528912ba (create your own!) + +Actual: + +```sh +docker run -d --name autobase-console \ + --publish 80:80 \ + --publish 8080:8080 \ + --env PG_CONSOLE_API_URL=http://116.202.31.52:8080/api/v1 \ + --env PG_CONSOLE_AUTHORIZATION_TOKEN=5b0b6259-a7d4-4435-947d-0dff528912ba \ + --env PG_CONSOLE_DOCKER_IMAGE=autobase/automation:latest \ + --volume console_postgres:/var/lib/postgresql \ + --volume /var/run/docker.sock:/var/run/docker.sock \ + --volume /tmp/ansible:/tmp/ansible \ + --restart=unless-stopped \ + autobase/console:latest +``` + + +Confirm it worked with the `docker ps` command. You should see something similar to the following: + +```sh +CONTAINER ID   IMAGE                     COMMAND                  CREATED              STATUS              PORTS                                                                                    NAMES + +9740dfd66c42   autobase/console:latest   "/usr/bin/supervisor…"   About a minute ago   Up About a minute   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 5432/tcp   autobase-console +``` + + +## 6. Login To `autobase`Console Web UI + +Visit the IP Address of your server in you web browser e.g: +http://116.202.31.52/ + +You should see a login screen: + +autobase-console-login + +Copy-paste the Token you defined in step 5 above. + +When you first login you should see that there are **No Postgres Clusters**: + +autobase-no-clusters + +## 7. Create Postgres Cluster + +Click the "**CREATE CLUSTER**" button: + +create-cluster-button + +Select `hetzner`and the datacenter region you prefer, in our case Europe: + +create-cluster-europe + +The default disk storage is **`100Gb`**; + +cluster-disk-storage + +this is _way_ too much for most simple projects. +lower it to **`10Gb`** for each instance to instantly save **50%** of the cost! +(you're welcome!) + +disk-storage-10gb + +> **Note**: all values for `DISK`storage, `RAM`, and `CPU`can easily be scaled later. + +Finally, you'll need to add your `public` SSH key. + +### Copy Your SSH Public Key + +```sh +cat ~/.ssh/id_ed25519.pub | pbcopy +``` + +Paste it into the `SSH public key*`field: + +add-ssh-key + +Then scroll down and click the "**CREATE CLUSTER**" button. + +You will see a modal window appear prompting you to input a `Hetzner`API Key: + +hetzner-api-key-modal + +### 8. Generate an API token + +Follow the instructions in the official `Hetzner` docs: +https://docs.hetzner.com/cloud/api/getting-started/generating-api-token/ + +In the `Hetzner`console, navigate to **Security** > **API tokens**. +You should see the message: +"**You haven't generated an API token yet.**" + +hetzner-api-tokens + +Click on the "**Generate API token**" button: + +generate-api-token + +That will open _another_ modal window, input the description for your key, +e.g: +"postgres-cluster-api-key" +and select "**Read and Write**": + +generate-api-token-modal + +Finally, click on the "**Generate API token**" button. +You should see a confirmation message: + +token-created + +Click to reveal the token you created: + +copy-token + +Copy the token to your clipboard, e.g: + +```sh +zH2qdgCeogrKjVKgV7sngMRxCfewgSdDARUBr8yqcjuHhGzlNdY72H13Sjh1il2D +``` + +Paste it into the Cluster creation window: + +paste-token-in-auto-window + +_Optionally_ save the API Key to the console and then +click "**CREATE CLUSTER**": + +create-cluster + +created: + +cluster-created + +Cluster details: + +cluster-details + +The `Postgres` cluster _appears_ to be deployed, +but how do we _know_ that it worked? + +## 9. Test The Cluster! 👩‍🔬 + +First: _connect_ to the **primary** `Postgres` instance. +In our case this is: `10.0.1.4` + +postgres-primary + +Sample: + +```sh +export PGPASSWORD='password'; +psql -h 127.0.0.1 -p 5432 -U postgres -d postgres +``` + +Get the **Password** and **Port** from the **Connection info** panel: + +postgres-connection-info + +Actual: + +```sh +export PGPASSWORD='9Djw2LNRMWwaDS1F9TlxeXiGj4dV3zNk'; +psql -h 88.99.81.115 -p 5432 -U postgres -d postgres +``` + +```sh +psql -h 10.0.1.4 -p 6432 -U postgres -d postgres +``` + +```sh +psql -h 10.0.1.4 -p 6432 -U postgres -d postgres -c "select version()" +``` + +Got the following error: + +```sh +Command 'psql' not found, but can be installed with: + +apt install postgresql-client-common +``` + +This is a barebones `Ubuntu` instance, remember, so it's not surprising that it doesn't have `psql` installed. So follow the instruction and install it: + +```sh +sudo apt install postgresql-client-common +``` + +The output is: + +But when trying to run `psql` again, we still get an error: + +```sh +Error: You must install at least one postgresql-client- package +``` + + + +## Outro: + +Given that this is a technical guide for an evolving system, +it may need to be enhanced/extended or updated in future, +that will be done on GitHub; +_everyone_ is welcome to and _encouraged_ to contribute! +Again, link in the description. + +Thanks for watching/listening. +If you found it useful and want to see more, +please subscribe. + + +## Privacy Disclaimer + +By the time you read/watch this,  +all of the sensitive data such as passwords, IP addresses,  +public keys and auth tokens will have been updated. +This avoids anyone getting ideas about accessing backend systems. + +We publish our notes and videos on how we do things +so that we can be as transparent as possible. +We have a strong security & privacy focus for all our systems +so all private backend systems like databases are always locked down. + +As always, if you have a security question or concern,  +Please contact us responsibly. +