Skip to content

Commit

Permalink
re-write backend overview
Browse files Browse the repository at this point in the history
Signed-off-by: Anisur Rahman <[email protected]>
  • Loading branch information
anisurrahman75 committed Mar 6, 2024
1 parent fe1d00d commit 2f84a12
Show file tree
Hide file tree
Showing 5 changed files with 152 additions and 154 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
20 changes: 13 additions & 7 deletions docs/guides/backends/overview/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,19 @@ section_menu_id: guides

KubeStash supports various backends for use as a BackupStorage. It can be a cloud storage like GCS bucket, AWS S3, Azure Blob Storage etc. or a Kubernetes persistent volume like [HostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath), [PersistentVolumeClaim](https://kubernetes.io/docs/concepts/storage/volumes/#persistentvolumeclaim), [NFS](https://kubernetes.io/docs/concepts/storage/volumes/#nfs) etc.

The following diagram shows how kubestash backup container accesses and backs up data into a backend.
The following diagram shows how kubestash operator and backup container accesses and backs up data into a backend.

<figure align="center">
<img alt="KubeStash Backend Overview" src="images/kubestash_backend_overview.svg">
<figcaption align="center">Fig: KubeStash Backend Overview</figcaption>
</figure>

You need to create a [BackupStorage]() object that contains backend information along with a Secret object containing the corresponding backend credentials required for accessing the backend.
The Backend process works in the following steps:

When you create the BackupStorage object, the Kubestash operator reads backend information from this object and retrieves access credentials from the associated Secret and initializes the BackupStorage.

Below, a screenshot that shows initialization of a BackupStorage in a GCS bucket named `kubestash-qa`:
- At first user creates a [BackupStorage](/docs/concepts/crds/backupstorage/index.md) object that contains the backend information along with a `Secret` object containing the corresponding backend credentials required for accessing the backend.
- KubeStash operator watches for `BackupStorage` custom resources and `Secrets`. When it finds a `BackupStorage` object, it establishes a connection with the storage by uploading the `metadata.yaml` file into the target storage.

Below, a screenshot that shows initialization of a `BackupStorage` in a GCS bucket named `kubestash-qa`:

<figure align="center">
  <img alt="BackupStorage initialization in GCS Backend" src="./images/backupstorage-initialize.png">
Expand All @@ -42,8 +42,11 @@ Below, a screenshot that shows initialization of a BackupStorage in a GCS bucket
Here, `kubestash-qa` serves as the bucket name, and the presence of `metadata.yaml` indicates the successful initialization of the BackupStorage.

## Repository
Once the BackupStoarge is initialize and in ready phase then the next steps is creating [BackupConfiguration](). When you create a BackupConfiguration, then KubeStash operator retrieves [Repository]() information from it and create Repository. This Repository object serve as a container for effectively managing and storing the backups data.

- Once the `BackupStorage` is initialized and in ready phase then the next steps is creating [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md).
- User creates a `BackupConfiguration` custom resource. The `BackupConfiguration` object specifies the `Repository` pointing to a `BackupStorage` that contains backend information, indicating where to upload backup data. Multiple `Repository` objects can refer to a single `BackupStorage`.
- Then KubeStash operator retrieves `Repository` information from it and create `Repository` object. This Repository object serves as a container for effectively managing and storing the backup data.
- KubeStash operator triggers a `Backup` job according to the schedule defined in the `BackupConfiguration`. This job accesses the `BackupStorage` object and `Secret` and uploads backup data into the storage.

Below, a screenshot the shows a `Repository` named `mongodb-backup` with backup data under a `BackupStorage` named `demo`:

Expand All @@ -54,4 +57,7 @@ Below, a screenshot the shows a `Repository` named `mongodb-backup` with backup


## Next Steps
- Learn how to configure `Kubernetes Volume` as backend from [here]().
- Learn how to configure `Kubernetes Volume` as backend from [here](/docs/guides/backends/local/index.md).
- Learn how to configure `AWS S3/Minio/Rook` backend from [here](/docs/guides/backends/s3/index.md).
- Learn how to configure `Google Cloud Storage (GCS)` backend from [here](/docs/guides/backends/gcs/index.md).
- Learn how to configure `Microsoft Azure Storage` backend from [here](/docs/guides/backends/azure/index.md).
52 changes: 37 additions & 15 deletions docs/guides/volumes/overview/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ section_menu_id: guides

# Stand-alone Volume Backup Overview

If you are using a volume that can be mounted in multiple workloads, aka `ReadWriteMany`/`RWX`, you might want to backup the volume independent of the workloads. KubeStash supports backup of stand-alone volumes. This guide will give you an overview of how stand-alone volume backup and restore process works in KubeStash.
If you are using a volume that can be mounted in multiple workloads, aka `ReadWriteMany`/`RWX`, you might want to take backup the volume independent of the workloads. KubeStash supports backup of stand-alone volumes. This guide will give you an overview of how stand-alone volume backup and restore process works in KubeStash.

## How Backup Works

Expand All @@ -27,39 +27,61 @@ The following diagram shows how KubeStash takes backup of a stand-alone volume.

The backup process consists of the following steps:

1. At first, a user creates a `Secret` with access credentials of the backend where the backed-up data will be stored.
1. At first, a user creates a `Secret`. This secret holds the credentials to access the backend where the backed up data will be stored.

2. Then, she creates a `BackupStorage` custom resource that specifies the backend information, along with the `Secret` containing the credentials needed to access the backend.

3. KubeStash operator watches for `BackupStorage` custom resources. When it finds a `BackupStorage` object, it initializes the `BackupStorage` by uploading the `metadata.yaml` file into the target storage.
4. Next, she creates a `BackupConfiguration` custom resource, targeting a standalone volume. The `BackupConfiguration` object specifies the `Repository` pointing to a `BackupStorage` that contains backend information, indicating where to upload backup data. It also defines the `Addon` information with a specified `Task` to be used for backing up the volume.

4. Then, she creates a `BackupConfiguration` custom resource which specifies the targeted a standalone volume. The `BackupConfiguration` object specifies the `Repository` pointing to a `BackupStorage` that contains backend information, indicating where to upload backup data. It also defines the `Addon` information with a specified `Task` to be used for backing up the volume.

5. KubeStash operator watches for `BackupConfiguration` custom resources.

6. Once the KubeStash operator finds a `BackupConfiguration` object, it creates `Repository` with the information specified in the BackupConfiguration.

7. KubeStash operator watches for `Repository` custom resources. When it finds the `Repository` object, it Initializes `Repository` by uploading `repository.yaml` file into the `spec.sessions[*].repositories[*].directory` path specified in `BackupConfiguration`.
8. Then, it creates a `CronJob` with the schedule specified in `BackupConfiguration` to trigger backup periodically.

8. Then, it creates a `CronJob` for each session with the schedule specified in `BackupConfiguration` to trigger backup periodically.

9. On the next scheduled slot, the `CronJob` triggers a backup by creating a `BackupSession` custom resource.

10. KubeStash operator watches for `BackupSession` custom resources.

11. When it finds a `BackupSession` object, it creates a `Snapshot` custom resource for each `Repository` specified in the `BackupConfiguration`.

12. Then it resolves the respective `Addon` and `Function` and prepares a backup `Job` definition.
13. Then, it mounts the targeted volume into the `Job` and creates it.
14. The Job takes backup of the targeted volume.
15. After the backup process is completed, the backup `Job` updates the `status.components[*]` field of the `Snapshot` resources with each target volume information. It also updates the `status.phase` field of the `BackupSession` to reflect backup completion.

13. Then, it mounts the targeted stand-alone volume into the `Job` and creates it.

14. The `Job` takes backup of the targeted stand-alone volume.

15. After the backup process is completed, the backup `Job` updates the `status.components[*]` field of the `Snapshot` object with target volume information. It also updates the `status.phase` field of the `BackupSession` to reflect backup completion.

## How Restore Works

The following diagram shows how Stash restores backed up data into a stand-alone volume. Open the image in a new tab to see the enlarged version.
The following diagram shows how KubeStash restores backed up data into a stand-alone volume. Open the image in a new tab to see the enlarged version.

<figure align="center">
  <img alt="Stand-alone Volume Restore Overview" src="images/volume_restore_overview.svg">
<figcaption align="center">Fig: Stand-alone Volume Restore Overview</figcaption>
</figure>

The restore process consists of the following steps:
1. At first, a user creates a `RestoreSession` custom resource, specifying the target volume where the backed-up data will be restored, the `Repository` object that points to a `BackupStorage` that holds backend information, and the target `Snapshot`, which will be restored. It also specifies the `Addon` info with `Task` to use to restore the volume.
2. KubeStash operator watches for `RestoreSession` custom resources.
3. Once it finds a `RestoreSession` object, it resolves the respective `Addon` and `Function` and prepares a restore Job definition.
4. Then, it mounts the targeted volume and creates a Restore `Job`
5. The restore `Job` restores the backed-up data into the targeted volume.
6. Finally, when the restore process is completed, the job updates the `status.phase` field of the `Restoresession` to reflect restore completion.

1. At first, the user creates a stand-alone volume where the data will be restored or the user can use the same stand-alone volume.

2. Then, she creates a `RestoreSession` custom resource, specifying the target stand-alone volume where the backed-up data will be restored, the `Repository` object that points to a `BackupStorage` that holds backend information, and the target `Snapshot`, which will be restored. It also specifies the `Addon` info with `Task` to use to restore the volume.

3. KubeStash operator watches for `RestoreSession` custom resources.

4. Once it finds a `RestoreSession` object, it resolves the respective `Addon` and `Function` and prepares a restore Job definition.

5. Then, it mounts the targeted volume and creates a Restore `Job`.

6. The restore `Job` restores the backed-up data into the targeted volume.

7. Finally, when the restore process is completed, the job updates the `status.phase` field of the `Restoresession` to reflect restore completion.

## Next Steps

- Learn how to backup and restore a stand-alone PVC from [here](/docs/guides/volumes/pvc/index.md).
- Learn how to take backup and restore a stand-alone PVC from [here](/docs/guides/volumes/pvc/index.md).
Loading

0 comments on commit 2f84a12

Please sign in to comment.