From c19ff8dd9324fe1411d0df7200e5256687ddd9ac Mon Sep 17 00:00:00 2001 From: "Md. Ishtiaq Islam" Date: Wed, 13 Dec 2023 19:31:31 +0600 Subject: [PATCH] Add kubestash docs Signed-off-by: Md. Ishtiaq Islam --- docs/CONTRIBUTING.md | 10 +- docs/README.md | 67 ++- docs/_index.md | 4 +- docs/acknowledgement.md | 4 +- docs/concepts/README.md | 35 +- docs/concepts/crds/addon/index.md | 202 +++++++ docs/concepts/crds/appbinding/index.md | 26 +- .../crds/backupbatch/backupbatch.yaml | 57 -- docs/concepts/crds/backupbatch/index.md | 187 ------- docs/concepts/crds/backupblueprint/index.md | 193 ++++--- .../crds/backupconfiguration/index.md | 522 ++++++++++-------- docs/concepts/crds/backupsession/index.md | 262 ++++----- docs/concepts/crds/backupstorage/index.md | 183 ++++++ docs/concepts/crds/function/index.md | 202 ++----- docs/concepts/crds/hooktemplate/index.md | 123 +++++ docs/concepts/crds/repository/index.md | 215 +++----- docs/concepts/crds/restorebatch/index.md | 204 ------- .../crds/restorebatch/restorebatch.yaml | 95 ---- docs/concepts/crds/restoresession/index.md | 496 ++++++++--------- docs/concepts/crds/retentionpolicy/index.md | 105 ++++ docs/concepts/crds/snapshot/index.md | 226 +++++--- docs/concepts/crds/task/index.md | 97 ---- docs/concepts/crds/task/task.yaml | 22 - .../_index.md | 0 .../images/stash_architecture.svg | 0 .../architecture/index.md | 0 .../overview/index.md | 4 +- .../images/enterprise_license_form.png | Bin .../install/{stash => kubestash}/index.md | 0 .../uninstall/{stash => kubestash}/index.md | 0 docs/support.md | 6 +- 31 files changed, 1691 insertions(+), 1856 deletions(-) create mode 100644 docs/concepts/crds/addon/index.md delete mode 100644 docs/concepts/crds/backupbatch/backupbatch.yaml delete mode 100644 docs/concepts/crds/backupbatch/index.md create mode 100644 docs/concepts/crds/backupstorage/index.md create mode 100644 docs/concepts/crds/hooktemplate/index.md delete mode 100644 docs/concepts/crds/restorebatch/index.md delete mode 100644 docs/concepts/crds/restorebatch/restorebatch.yaml create mode 100644 docs/concepts/crds/retentionpolicy/index.md delete mode 100644 docs/concepts/crds/task/index.md delete mode 100644 docs/concepts/crds/task/task.yaml rename docs/concepts/{what-is-stash => what-is-kubestash}/_index.md (100%) rename docs/concepts/{what-is-stash => what-is-kubestash}/architecture/images/stash_architecture.svg (100%) rename docs/concepts/{what-is-stash => what-is-kubestash}/architecture/index.md (100%) rename docs/concepts/{what-is-stash => what-is-kubestash}/overview/index.md (89%) rename docs/setup/install/{stash => kubestash}/images/enterprise_license_form.png (100%) rename docs/setup/install/{stash => kubestash}/index.md (100%) rename docs/setup/uninstall/{stash => kubestash}/index.md (100%) diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index a39336a..7efa1ee 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -1,9 +1,9 @@ --- -title: Contributing | Stash +title: Contributing | KubeStash description: Contributing menu: docs_{{ .version }}: - identifier: contributing-stash + identifier: contributing-kubestash name: Contributing parent: welcome weight: 1000 @@ -17,7 +17,7 @@ aliases: # Contribution Guidelines -Want to contribute to Stash? +Want to contribute to KubeStash? ## Getting Help @@ -25,8 +25,8 @@ To speak with us, please leave a message on [our website](https://appscode.com/c ## Bugs/Feature request -If you have found a bug with Stash or want to request for new features, please [file an issue](https://github.com/kubestash/project/issues/new). +If you have found a bug with KubeStash or want to request for new features, please [file an issue](https://github.com/kubestash/project/issues/new). ## Spread the word -If you have written blog post or tutorial on Stash, please share it with us on [Twitter](https://twitter.com/KubeStash). +If you have written blog post or tutorial on KubeStash, please share it with us on [Twitter](https://twitter.com/KubeStash). diff --git a/docs/README.md b/docs/README.md index f306199..6ea13d3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,9 +1,9 @@ --- -title: Welcome | Stash -description: Welcome to Stash +title: Welcome | KubeStash +description: Welcome to KubeStash menu: docs_{{ .version }}: - identifier: readme-stash + identifier: readme-kubestash name: Readme parent: welcome weight: -1 @@ -16,65 +16,62 @@ aliases: - /docs/{{ .version }}/README/ --- -# Stash +# KubeStash -[Stash](https://stash.run) by AppsCode is a cloud native data backup and recovery solution for Kubernetes workloads. If you are running production workloads in Kubernetes, you might want to take backup of your disks, databases etc. Traditional tools are too complex to set up and maintain in a dynamic compute environment like Kubernetes. Stash is a Kubernetes operator that uses [restic](https://github.com/restic/restic) or Kubernetes CSI Driver VolumeSnapshotter functionality to address these issues. Using Stash, you can backup Kubernetes volumes mounted in workloads, stand-alone volumes and databases. Users may even extend Stash via [addons](https://stash.run/docs/{{< param "info.version" >}}/guides/addons/overview/) for any custom workload. +[KubeStash](https://kubestash.com) by AppsCode is a cloud native data backup and recovery solution for Kubernetes workloads. If you are running production workloads in Kubernetes, you might want to take backup of your disks, databases etc. Traditional tools are too complex to set up and maintain in a dynamic compute environment like Kubernetes. KubeStash is a Kubernetes operator that uses [restic](https://github.com/restic/restic) or Kubernetes CSI Driver VolumeSnapshotter functionality to address these issues. Using KubeStash, you can backup Kubernetes volumes mounted in workloads, stand-alone volumes and databases. Users may even extend KubeStash via [addons](https://kubestash.com/docs/{{< param "info.version" >}}/guides/addons/overview/) for any custom workload. -Here, we are going to give you an overview of Stash documentation structure. +Here, we are going to give you an overview of KubeStash documentation structure. ## [Concepts](/docs/concepts/) -Concept explains some significant aspect of Stash. This is where you can learn about what Stash does and how it does it. +Concept explains some significant aspect of KubeStash. This is where you can learn about what KubeStash does and how it does it. -- [Stash Overview](/docs/concepts/what-is-stash/overview/index.md) Provides an introduction to Stash and gives an overview of the features it provides. -- [Stash Architecture](/docs/concepts/what-is-stash/architecture/index.md) Provides an overview of Stash architecture and it's core components. -- [Stash API](/docs/concepts/crds/repository/index.md) Introduces Stash CRDs. +- [KubeStash Overview](/docs/concepts/what-is-kubestash/overview/index.md) Provides an introduction to KubeStash and gives an overview of the features it provides. +- [KubeStash Architecture](/docs/concepts/what-is-kubestash/architecture/index.md) Provides an overview of KubeStash architecture, and it's core components. +- [KubeStash API](/docs/concepts/crds/backupstorage/index.md) Introduces KubeStash CRDs. ## [Setup](/docs/setup/) -Setup contains instruction for installing, uninstalling, and upgrading Stash. +Setup contains instruction for installing, uninstalling, and upgrading KubeStash. -- **Install Stash:** Provides installation instructions for Stash and its various components. - - [Stash](/docs/setup/install/stash/index.md): Provides installation instructions for Stash. - - [Stash kubectl Plugin](/docs/setup/install/kubectl-plugin/index.md): Provides installation instructions for Stash `kubectl` plugin. +- **Install KubeStash:** Provides installation instructions for KubeStash and its various components. + - [KubeStash](/docs/setup/install/kubestash/index.md): Provides installation instructions for KubeStash. + - [kubeStash kubectl Plugin](/docs/setup/install/kubectl-plugin/index.md): Provides installation instructions for KubeStash `kubectl` plugin. - [Troubleshooting](/docs/setup/install/troubleshooting/index.md): Provides troubleshooting guide for various installation problems. -- **Uninstall Stash:** Provides uninstallation instructions for Stash and its various components. - - [Stash](/docs/setup/uninstall/stash/index.md): Provides uninstallation instructions for Stash. - - [Stash kubectl Plugin](/docs/setup/uninstall/kubectl-plugin/index.md): Provides uninstallation instructions for Stash `kubectl` plugin. -- [Upgrade Stash](/docs/setup/upgrade/index.md): Provides instruction for updating Stash license and upgrading between various Stash versions. +- **Uninstall KubeStash:** Provides uninstallation instructions for KubeStash and its various components. + - [KubeStash](/docs/setup/uninstall/kubestash/index.md): Provides uninstallation instructions for KubeStash. + - [KubeStash kubectl Plugin](/docs/setup/uninstall/kubectl-plugin/index.md): Provides uninstallation instructions for KubeStash `kubectl` plugin. +- [Upgrade KubeStash](/docs/setup/upgrade/index.md): Provides instruction for updating KubeStash license and upgrading between various KubeStash versions. ## [Guides](/docs/guides/) -Guides show how to perform different operations with Stash. +Guides show how to perform different operations with KubeStash. - [Supported Backends](/docs/guides/backends/overview/index.md): Describes how to configure different storage for storing backed up data. -- [Workload Volume Backup](/docs/guides/workloads/overview/index.md): Shows how to use Stash to backup and restore volumes of a workload (i.e. `Deployment`, `StatefulSet`, `DaemonSet`, etc). -- [Stand-alone Volume Backup](/docs/guides/volumes/overview/index.md): Shows how to use Stash to backup and restore stand-alone volumes(i.e. `PersistentVolumeClaim`). -- [Batch Backup](/docs/guides/batch-backup/overview/index.md): Shows how to backup multiple co-related components using a single configuration known as `BackupBatch`. +- [Workload Volume Backup](/docs/guides/workloads/overview/index.md): Shows how to use KubeStash to backup and restore volumes of a workload (i.e. `Deployment`, `StatefulSet`, `DaemonSet`, etc). +- [Stand-alone Volume Backup](/docs/guides/volumes/overview/index.md): Shows how to use KubeStash to backup and restore stand-alone volumes(i.e. `PersistentVolumeClaim`). - [Auto Backup](/docs/guides/auto-backup/overview/index.md): Shows how to configure automatic backup of any stateful workload in your cluster. -- [Volume Snapshot](/docs/guides/volumesnapshot/overview/index.md): Shows how Stash takes snapshot of `PersistentVolumeClaim`s and restore them from snapshot using Kubernetes `VolumeSnapshot` API. +- [Volume Snapshot](/docs/guides/volumesnapshot/overview/index.md): Shows how KubeStash takes snapshot of `PersistentVolumeClaim`s and restore them from snapshot using Kubernetes `VolumeSnapshot` API. - **Different Use Cases:** -Shows different uses cases of Stash like instant backup, pause backup, cross-namespace backup and restore etc. +Shows different uses cases of KubeStash like instant backup, pause backup, cross-namespace backup and restore etc. - - [Instant Backup](/docs/guides/use-cases/instant-backup/index.md): Shows you how to take an instant backup in Stash. + - [Instant Backup](/docs/guides/use-cases/instant-backup/index.md): Shows you how to take an instant backup in KubeStash. - [Exclude/Include Files](/docs/guides/use-cases/exclude-include-files/index.md): Shows how to exclude/include subset of files during backup/restore process. - [Pause Backup](/docs/guides/use-cases/pause-backup/index.md): Shows how to pause a backup temporarily. - - [Clone Data Volumes](/docs/guides/use-cases/clone-pvc/index.md): Shows how to clone data volumes of a workload into a different namespace in a cluster using Stash. + - [Clone Data Volumes](/docs/guides/use-cases/clone-pvc/index.md): Shows how to clone data volumes of a workload into a different namespace in a cluster using KubeStash. - [File Ownership](/docs/guides/use-cases/ownership/index.md): Explains when and how ownership of the restored files can be changed. - - [Cross-Cluster Backup and Restore](/docs/guides/use-cases/cross-cluster-backup/index.md): Shows how to take backup and restore across clusters using Stash. - - [Customize Backup and Restore](/docs/guides/use-cases/customize-backup-restore/index.md): Shows how to customize backup and restore processes in Stash according to your needs. + - [Cross-Cluster Backup and Restore](/docs/guides/use-cases/cross-cluster-backup/index.md): Shows how to take backup and restore across clusters using KubeStash. + - [Customize Backup and Restore](/docs/guides/use-cases/customize-backup-restore/index.md): Shows how to customize backup and restore processes in KubeStash according to your needs. - **Managed Backup** -Shows how backup and restore can be managed in some specific scenerios. - - [Dedicated Backup Namespace](/docs/guides/managed-backup/dedicated-backup-namespace/index.md): Shows you how to manage backup and restore from a dedicated namespace for targets of different namespaces using Stash. - - [Dedicated Storage Namespace](/docs/guides/managed-backup/dedicated-storage-namespace/index.md): Shows you how to take backup and restore by keeping the storage resources (Repository and backend Secret) in a dedicated namespace using Stash. +Shows how backup and restore can be managed in some specific scenarios. + - [Dedicated Backup Namespace](/docs/guides/managed-backup/dedicated-backup-namespace/index.md): Shows you how to manage backup and restore from a dedicated namespace for targets of different namespaces using KubeStash. -- [Platforms](/docs/guides/platforms/eks-irsa/index.md): Shows how to use Stash to backup and restore volumes of a Kubernetes workload running in different platforms. -- [Monitoring](/docs/guides/monitoring/overview/index.md): Shows how Prometheus monitoring works with Stash, what metrics Stash exports, and how to enable monitoring. +- [Platforms](/docs/guides/platforms/eks-irsa/index.md): Shows how to use KubeStash to backup and restore volumes of a Kubernetes workload running in different platforms. - [Hooks](/docs/guides/hooks/overview/index.md): Shows how to execute different actions before/after the backup/restore process. -- [CLI](/docs/guides/cli/kubectl-plugin/index.md): Shows how to manage Stash objects quickly and easily using Stash `kubectl` plugin. +- [CLI](/docs/guides/cli/kubectl-plugin/index.md): Shows how to manage KubeStash objects quickly and easily using KubeStash `kubectl` plugin. - [Troubleshooting](/docs/guides/troubleshooting/how-to-troubleshoot/index.md): Gives an overview of how you can gather the necessary information to identify the issue that causes the backup/restore failure. -- [Security](/docs/guides/security/rbac/index.md): Describes different built-in cluster security support by Stash. +- [Security](/docs/guides/security/rbac/index.md): Describes different built-in cluster security support by KubeStash. We're always looking for help improving our documentation, so please don't hesitate to [file an issue](https://github.com/kubestash/project/issues/new) if you see some problem. diff --git a/docs/_index.md b/docs/_index.md index e10e72a..88c8976 100644 --- a/docs/_index.md +++ b/docs/_index.md @@ -1,6 +1,6 @@ --- -title: Docs | Stash -description: Stash Docs +title: Docs | KubeStash +description: KubeStash Docs menu: docs_{{ .version }}: identifier: welcome diff --git a/docs/acknowledgement.md b/docs/acknowledgement.md index 7705529..72a0ef0 100644 --- a/docs/acknowledgement.md +++ b/docs/acknowledgement.md @@ -1,9 +1,9 @@ --- -title: Acknowledgement | Stash +title: Acknowledgement | KubeStash description: Acknowledgement menu: docs_{{ .version }}: - identifier: acknowledgement-stash + identifier: acknowledgement-kubestash name: Acknowledgement parent: welcome weight: 1010 diff --git a/docs/concepts/README.md b/docs/concepts/README.md index 72fee63..32ea2da 100644 --- a/docs/concepts/README.md +++ b/docs/concepts/README.md @@ -1,5 +1,5 @@ --- -title: Concepts | Stash +title: Concepts | KubeStash menu: docs_{{ .version }}: identifier: concepts-readme @@ -16,23 +16,24 @@ aliases: # Concepts -Concepts help you to learn about the different parts of the Stash and the abstractions it uses. +Concepts help you to learn about the different parts of the KubeStash and the abstractions it uses. This concept section is divided into the following modules: -- What is Stash? - - [Overview](/docs/concepts/what-is-stash/overview/index.md) provides an introduction to Stash. It also give an overview of the features it provides. - - [Architecture](/docs/concepts/what-is-stash/architecture/index.md) provides a visual representation of Stash architecture. It also provides a brief overview of the components it uses. +## What is KubeStash? +- [Overview](/docs/concepts/what-is-kubestash/overview/index.md) provides an introduction to KubeStash. It also gives an overview of the features it provides. +- [Architecture](/docs/concepts/what-is-kubestash/architecture/index.md) provides a visual representation of KubeStash architecture. It also provides a brief overview of the components it uses. -- Declarative API - - [Repository](/docs/concepts/crds/repository/index.md) introduces the concept of `Repository` crd that holds backend information in a Kubernetes native way. - - [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) introduces the concept of `BackupConfiguration` crd that is used to configure backup for a target resource in a Kubernetes native way. - - [BackupBatch](/docs/concepts/crds/backupbatch/index.md) introduces the concept of `BackupBatch` crd that is used to setup backup of multiple co-related targets under single configuration. - - [BackupSession](/docs/concepts/crds/backupsession/index.md) introduces the concept of `BackupSession` crd that represents a backup run of a target resource for the respective `BackupConfiguration` or `BackupBatch` object. - - [RestoreSession](/docs/concepts/crds/restoresession/index.md) introduces the concept of `RestoreSession` crd that represents a restore run of a target resource. - - [RestoreBatch](/docs/concepts/crds/restorebatch/index.md) introduces the concept of `RestoreBatch` crd that allows restore of multiple targets that were backed up using `BackupBatch` under single configuration. - - [Function](/docs/concepts/crds/function/index.md) introduces the concept of `Function` crd that represents a step of a backup or restore process. - - [Task](/docs/concepts/crds/task/index.md) introduces the concept of `Task` crd which specifies an ordered collection of multiple `Function`s and their parameters that make up a complete backup or restore process. - - [BackupBlueprint](/docs/concepts/crds/backupblueprint/index.md) introduces the concept of `BackupBlueprint` crd that specifies a blueprint for `Repository` and `BackupConfiguration` object which provides an option to share backup configuration across similar targets. - - [AppBinding](/docs/concepts/crds/appbinding/index.md) introduces the concept of `AppBinding` crd which holds the information that are necessary to connect with an application like database. - - [Snapshot](/docs/concepts/crds/snapshot/index.md) introduces the concept of `Snapshot` object that represents backed up snapshots in a Kubernetes native way. +## Declarative API +- [BackupStorage](/docs/concepts/crds/backupstorage/index.md) introduces the concept of `BackupStorage` crd that holds the backend information in a Kubernetes native way where the backed up data of different applications will be stored. +- [Repository](/docs/concepts/crds/repository/index.md) introduces the concept of `Repository` crd that holds information about the targeted application that has been backed up. +- [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) introduces the concept of `BackupConfiguration` crd that is used to configure backup for a target application in a Kubernetes native way. +- [BackupSession](/docs/concepts/crds/backupsession/index.md) introduces the concept of `BackupSession` crd that represents a backup run of a target application for the respective `BackupConfiguration` object. +- [RestoreSession](/docs/concepts/crds/restoresession/index.md) introduces the concept of `RestoreSession` crd that represents a restore run of a target application. +- [HookTemplate](/docs/concepts/crds/hooktemplate/index.md) introduces the concept of `HookTemplate` crd that represents a template for some action that will be executed before or/and after backup/restore process. +- [Addon](/docs/concepts/crds/addon/index.md) introduces the concept of `Addon` crd which represents the backup and restore tasks that can be performed by a particular addon. +- [Function](/docs/concepts/crds/function/index.md) introduces the concept of `Function` crd that represents a step of a backup or restore process. +- [BackupBlueprint](/docs/concepts/crds/backupblueprint/index.md) introduces the concept of `BackupBlueprint` crd that represents a blueprint for `BackupConfiguration` object which provides an option to share backup configuration across similar targets. +- [Snapshot](/docs/concepts/crds/snapshot/index.md) introduces the concept of `Snapshot` crd that represents the state of a backup run to a particular `Repository`. +- [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) introduces the concept of `RetentionPolicy` crd that represents how the old `Snapshots` should be cleaned up. +- [AppBinding](/docs/concepts/crds/appbinding/index.md) introduces the concept of `AppBinding` crd which holds the information that are necessary to connect with an application like database. diff --git a/docs/concepts/crds/addon/index.md b/docs/concepts/crds/addon/index.md new file mode 100644 index 0000000..4752cd9 --- /dev/null +++ b/docs/concepts/crds/addon/index.md @@ -0,0 +1,202 @@ +--- +title: Addon Overview +menu: + docs_{{ .version }}: + identifier: addon-overview + name: Addon + parent: crds + weight: 30 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: concepts +--- + +> New to Stash? Please start [here](/docs/concepts/README.md). + +# Addon + +## What is Addon +An `Addon` is a Kubernetes `CustomResourceDefinition`(CRD) which specifies the backup and restore capabilities for a particular resource. For example, MySQL addon specifies the backup and restore capabilities of MySQL database where Postgres addon specifies backup and restore capabilities for PostgreSQL database. An Addon CR defines the backup and restore tasks that can be performed by this addon. + +When you install KubeStash, some `Addon`s will be pre-installed for supported targets like workloads, etc. However, you can create your own `Addon` to customize or extend the backup/restore process. + +## Addon CRD Specification +Like any official Kubernetes resource, an `Addon` has `TypeMeta`, `ObjectMeta` and `Spec` sections. However, unlike other Kubernetes resources, it does not have a `Status` section. + +A sample `Addon` object for a PostgreSQL database is shown below: +```yaml +apiVersion: addons.kubestash.com/v1alpha1 +kind: Addon +metadata: + labels: + app.kubernetes.io/instance: kubedb + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: kubedb-kubestash-catalog + app.kubernetes.io/version: v2023.12.11 + helm.sh/chart: kubedb-kubestash-catalog-v2023.12.11 + name: postgres-addon +spec: + backupTasks: + - driver: Restic + executor: Job + function: postgres-backup + name: LogicalBackup + parameters: + - name: args + required: false + usage: Arguments to be passed to the dump command. + - default: pg_dumpall + name: backupCmd + required: false + usage: Backup command to take a database dump (can only be pg_dumpall or pg_dump) + - default: postgres + name: user + required: false + usage: Specifies database user (not applicable for basic authentication) + - default: "true" + name: enableCache + required: false + usage: Enable or disable caching. Disabling caching may impact backup performance. + - default: /kubestash-tmp + name: scratchDir + required: false + usage: Directory for holding temporary files and restic cache. + singleton: true + volumeMounts: + - mountPath: /kubestash-tmp + name: kubestash-tmp-volume + volumeTemplate: + - name: kubestash-tmp-volume + source: + emptyDir: {} + usage: Holds temporary files and restic cache. + - driver: VolumeSnapshotter + executor: Job + function: postgres-csisnapshotter + name: VolumeSnapshot + parameters: + - name: volumeSnapshotClassName + required: false + usage: The VolumeSnapshotClassName to be used by volumeSnapshot + singleton: true + - driver: Restic + executor: Job + function: kubedbmanifest-backup + name: ManifestBackup + parameters: + - default: "true" + name: enableCache + required: false + usage: Enable or disable caching. Disabling caching may impact backup performance. + - default: /kubestash-tmp + name: scratchDir + required: false + usage: Directory for holding temporary files and restic cache. + singleton: true + volumeMounts: + - mountPath: /kubestash-tmp + name: kubestash-tmp-volume + volumeTemplate: + - name: kubestash-tmp-volume + source: + emptyDir: {} + usage: Holds temporary files and restic cache. + restoreTasks: + - driver: Restic + executor: Job + function: postgres-restore + name: LogicalBackupRestore + parameters: + - name: args + required: false + usage: Arguments to be passed to the dump command. + - default: postgres + name: user + required: false + usage: Specifies database user (not applicable for basic authentication) + - default: "true" + name: enableCache + required: false + usage: Enable or disable caching. Disabling caching may impact backup performance. + - default: /kubestash-tmp + name: scratchDir + required: false + usage: Directory for holding temporary files and restic cache. + singleton: true + volumeMounts: + - mountPath: /kubestash-tmp + name: kubestash-tmp-volume + volumeTemplate: + - name: kubestash-tmp-volume + source: + emptyDir: {} + usage: Holds temporary files and restic cache. + - driver: Restic + executor: Job + function: kubedbmanifest-restore + name: ManifestRestore + parameters: + - default: "true" + name: enableCache + required: false + usage: Enable or disable caching. Disabling caching may impact backup performance. + - default: /kubestash-tmp + name: scratchDir + required: false + usage: Directory for holding temporary files and restic cache. + singleton: true + volumeMounts: + - mountPath: /kubestash-tmp + name: kubestash-tmp-volume + volumeTemplate: + - name: kubestash-tmp-volume + source: + emptyDir: {} + usage: Holds temporary files and restic cache. +``` +This `Addon` has three backup tasks and two restore tasks for a PostgreSQL database. Each task refers to a `Function`. + +Here, we are going to describe the various sections of a `Addon` crd. + +## Addon `Spec` +A `Addon` object has the following fields in the `spec` section: +- **backupTasks** specifies a list of backup tasks that can be performed by the addon. +- **restoreTasks** specifies a list of restore tasks that can be performed by the addon. + +Each task contains the following fields: +- **name** specifies the name of the task. The name of a task should indicate what this task does. For example, a name LogicalBackup indicate that this task performs a logical backup of a database. +- **function** specifies the name of a `Function` CR that defines a container definition which will execute the backup/restore logic for a particular application. +- **driver** specifies the underlying tool that will be used to upload the data to the backend storage. Valid values are: + - **Restic** The underlying tool is [restic](https://restic.net/). + - **WalG** The underlying tool is [wal-g](https://github.com/wal-g/wal-g). + - **VolumeSnapshotter** The underlying method is creating [VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/). +- **executor** specifies the type of entity that will execute the task. For example, it can be Job(s), a sidecar container, an ephemeral container, or a Job that creates additional Jobs/Pods for executing the backup/restore logic. Valid values are: + - **Job** KubeStash will create Job(s) to execute the backup/restore task. + - **Sidecar** KubeStash will inject a sidecar container into the application to execute the backup/restore task. + - **EphemeralContainer** KubeStash will attach an ephemeral container to the respective Pods to execute the backup/restore task. + - **MultiLevelJob** KubeStash will create a Job that will create additional Jobs/Pods to execute the backup/restore task. +- **singleton** specifies whether this task will be executed on a single job or across multiple jobs. +- **parameters** defines a list of parameters that is used by the task to execute its logic. Parameter definition defines the parameter names, their usage, their requirements etc. + - **name** specifies the name of the parameter. + - **usage** specifies the usage of this parameter. + - **required** specify whether this parameter is required or not. + - **default** specifies a default value for the parameter +- **volumeTemplate** specifies a list of volume templates that is used by the respective backup/restore Job to execute its logic. You can overwrite these volume templates using `addonVolumes` field of `BackupConfiguration`. Each `volumeTemplate` contains: + - **name** specifies the name of the volume. + - **usage** specifies the usage of the volume. + - **source** specifies the source of this volume or a template for volume to use. +- **volumeMounts** specifies the mount path of the volumes specified in the `VolumeTemplate` section. These volumes will be mounted directly on the Job/Container created/injected by KubeStash operator. If the volume type is `VolumeClaimTemplate`, then KubeStash operator is responsible for creating the volume. +- **passThroughMounts** specifies a list of volume mount for the `VolumeTemplates` that should be mounted on second level Jobs/Pods created by the first level executor Job. If the volume needs to be mounted on both first level and second level Jobs/Pods, then specify the mount in both `VolumeMounts` and `PassThroughMounts` section. If the volume type is `VolumeClaimTemplate`, then the first level job is responsible for creating the volume. + +## Why Function and Addon? + +You might be wondering why we have introduced `Function` and `Addon` crd. We have designed `Function-Addon` model for the following reasons: + +- **Customizable:** `Function` and `Addon` enables you to customize backup/recovery process. For example, currently we use [mysqldump](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html) in `mysql-backup` task to backup MySQL database. You can build a custom `Function` using Percona's [xtrabackup](https://www.percona.com/software/mysql-database/percona-xtrabackup) tool instead of `mysqldump`. Then you can write an `Addon` or add a task in existing MySQL `Addon` with this custom `Function` and use it to backup your target MySQL database. +- **Extensibility:** You can easily backup the databases that are not officially supported by KubeStash. You just need to create `Function`s and an `Addon` for your desired database. +- **Re-usability:** `Function`s are self-sufficient and independent of KubeStash. So, you can reuse them in any application that uses `Function-Addon` model. + +## Next Steps + +- Learn how KubeStash backup databases using `Function-Addon` model from [here](/docs/guides/addons/overview/index.md). +- Learn how KubeStash backup stand-alone PVC using `Function-Addon` model from [here](/docs/guides/volumes/overview/index.md). \ No newline at end of file diff --git a/docs/concepts/crds/appbinding/index.md b/docs/concepts/crds/appbinding/index.md index 51b91a9..b99c556 100644 --- a/docs/concepts/crds/appbinding/index.md +++ b/docs/concepts/crds/appbinding/index.md @@ -5,19 +5,19 @@ menu: identifier: appbinding-overview name: AppBinding parent: crds - weight: 45 + weight: 60 product_name: kubestash menu_name: docs_{{ .version }} section_menu_id: concepts --- -> New to Stash? Please start [here](/docs/concepts/README.md). +> New to KubeStash? Please start [here](/docs/concepts/README.md). # AppBinding ## What is AppBinding -Stash needs to know how to connect with a target database and the credentials necessary to access it. This is done via an `AppBinding`. +KubeStash needs to know how to connect with a target database and the credentials necessary to access it. This is done via an `AppBinding`. An `AppBinding` is a Kubernetes `CustomResourceDefinition`(CRD) which points to an application using either its URL (usually for a non-Kubernetes resident service instance) or a Kubernetes service object (if self-hosted in a Kubernetes cluster), some optional parameters and a credential secret. To learn more about AppBinding and the problems it solves, please read this blog post: [The case for AppBinding](https://blog.byte.builders/post/the-case-for-appbinding). @@ -54,11 +54,11 @@ spec: version: "10.2" ``` -Here, we are going to describe the sections of an `AppBinding` crd that are relevant to Stash. +Here, we are going to describe the sections of an `AppBinding` crd that are relevant to KubeStash. ### AppBinding `Spec` -An `AppBinding` object has the following fields in the `spec` section that are relevant to Stash: +An `AppBinding` object has the following fields in the `spec` section that are relevant to KubeStash: #### spec.type @@ -68,8 +68,8 @@ This field follows the following format: `/`. The abov Here, the variables are parsed as follows: -| Variable | Usage | -| --------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| Variable | Usage | +|-----------------------|-----------------------------------------------------------------------------------------------------------------------------------| | `TARGET_APP_GROUP` | Represents the application group where the respective app belongs (i.e: `kubedb.com`). | | `TARGET_APP_RESOURCE` | Represents the resource under that application group that this appbinding represents (i.e: `postgres`). | | `TARGET_APP_TYPE` | Represents the complete type of the application. It's simply `TARGET_APP_GROUP/TARGET_APP_RESOURCE` (i.e: `kubedb.com/postgres`). | @@ -83,28 +83,28 @@ This secret must contain the following keys: PostgreSQL : | Key | Usage | -| ------------------- | --------------------------------------------------- | +|---------------------|-----------------------------------------------------| | `POSTGRES_USER` | Username of the target database. | | `POSTGRES_PASSWORD` | Password for the user specified by `POSTGRES_USER`. | MySQL : | Key | Usage | -| ---------- | ---------------------------------------------- | +|------------|------------------------------------------------| | `username` | Username of the target database. | | `password` | Password for the user specified by `username`. | MongoDB : | Key | Usage | -| ---------- | ---------------------------------------------- | +|------------|------------------------------------------------| | `username` | Username of the target database. | | `password` | Password for the user specified by `username`. | Elasticsearch: -| Key | Usage | -| ---------------- | ----------------------- | +| Key | Usage | +|------------------|-------------------------| | `ADMIN_USERNAME` | Admin username | | `ADMIN_PASSWORD` | Password for admin user | @@ -138,4 +138,4 @@ You can configure following fields in `spec.clientConfig` section: ## Next Steps -- Learn how to backup/restore databases using Stash from [here](/docs/guides/addons/overview/index.md). +- Learn how to backup/restore databases using KubeStash from [here](/docs/guides/addons/overview/index.md). diff --git a/docs/concepts/crds/backupbatch/backupbatch.yaml b/docs/concepts/crds/backupbatch/backupbatch.yaml deleted file mode 100644 index 405ec66..0000000 --- a/docs/concepts/crds/backupbatch/backupbatch.yaml +++ /dev/null @@ -1,57 +0,0 @@ -apiVersion: stash.appscode.com/v1beta1 -kind: BackupBatch -metadata: - name: deploy-backup-batch - namespace: demo -spec: - repository: - name: minio-repo - namespace: demo - schedule: "*/3 * * * *" - members: - - target: - alias: db - ref: - apiVersion: apps/v1 - kind: AppBinding - name: wordpress-mysql - task: - name: mysql-backup-8.0.14 - - target: - alias: app - ref: - apiVersion: apps/v1 - kind: Deployment - name: wordpress - volumeMounts: - - name: wordpress-persistent-storage - mountPath: /var/www/html - paths: - - /var/www/html - exclude: - - /var/www/html/my-file.html - - /var/www/html/*.json - executionOrder: Parallel - hooks: - preBackup: - exec: - command: - - /bin/sh - - -c - - echo "Sample PreBackup hook demo" - containerName: my-database-container - postBackup: - exec: - command: - - /bin/sh - - -c - - echo "Sample PostBackup hook demo" - containerName: my-database-container - retryConfig: - maxRetry: 3 - delay: 10m - timeOut: 1h30m - retentionPolicy: - name: 'keep-last-10' - keepLast: 10 - prune: true diff --git a/docs/concepts/crds/backupbatch/index.md b/docs/concepts/crds/backupbatch/index.md deleted file mode 100644 index 1334649..0000000 --- a/docs/concepts/crds/backupbatch/index.md +++ /dev/null @@ -1,187 +0,0 @@ ---- -title: BackupBatch Overview -menu: - docs_{{ .version }}: - identifier: backupbatch-overview - name: BackupBatch - parent: crds - weight: 15 -product_name: kubestash -menu_name: docs_{{ .version }} -section_menu_id: concepts ---- - -# BackupBatch - -## What is BackupBatch - -Sometimes, a single component may not meet the requirement for your application. For example, in order to deploy a WordPress, you will need a Deployment for the WordPress and another Deployment for a database to store its contents. Now, you may want to backup both of the deployment and database together as they are parts of a single application. - -A `BackupBatch` is a Kubernetes `CustomResourceDefinition`(CRD) which lets you configure backup for multiple co-related components(workload, database, etc.) together. - -## BackupBatch CRD Specification - -Like any official Kubernetes resource, a `BackupBatch` has `TypeMeta`, `ObjectMeta`, `Spec` and `Status` sections. - -A sample `BackupBatch` object to backup multiple co-related components is shown below: - -```yaml -apiVersion: stash.appscode.com/v1beta1 -kind: BackupBatch -metadata: - name: deploy-backup-batch - namespace: demo -spec: - repository: - name: minio-repo - namespace: demo - schedule: "*/3 * * * *" - members: - - target: - alias: db - ref: - apiVersion: apps/v1 - kind: AppBinding - name: wordpress-mysql - task: - name: mysql-backup-8.0.14 - - target: - alias: app - ref: - apiVersion: apps/v1 - kind: Deployment - name: wordpress - volumeMounts: - - name: wordpress-persistent-storage - mountPath: /var/www/html - paths: - - /var/www/html - exclude: - - /var/www/html/my-file.html - - /var/www/html/*.json - executionOrder: Parallel - hooks: - preBackup: - exec: - command: - - /bin/sh - - -c - - echo "Sample PreBackup hook demo" - containerName: my-database-container - postBackup: - exec: - command: - - /bin/sh - - -c - - echo "Sample PostBackup hook demo" - containerName: my-database-container - retryConfig: - maxRetry: 3 - delay: 10m - timeOut: 1h30m - retentionPolicy: - name: 'keep-last-10' - keepLast: 10 - prune: true -``` - -Here, we are going to describe the various sections of `BackupBatch` crd. - -### BackupBatch `Spec` - -A `BackupBatch` object has the following fields in the `spec` section. - -#### spec.driver - -`spec.driver` indicates the mechanism used to backup. Currently, Stash supports `Restic` and `VolumeSnapshotter` as drivers. The default value of this field is `Restic`. For more details, please see [here](/docs/concepts/crds/backupconfiguration/index.md#specdriver). - -#### spec.repository - -`spec.repository.name` indicates the `Repository` crd name that holds necessary backend information where the backed up data will be stored. - -#### spec.schedule - -`spec.schedule` is a [cron expression](https://en.wikipedia.org/wiki/Cron) that specifies the schedule of backup. Stash creates a Kubernetes [CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/) with this schedule. - -#### spec.executionOrder - -`spec.executionOrder` specifies whether Stash should backup the targets sequentially or parallelly. If `spec.executionOrder` is set to `Parallel`, Stash will start backup of all the targets simultaneously. If it is set to `Sequential`, Stash will not start backup of a target until all the previous members have completed their backup process. The default value of this field is `Parallel`. - -#### spec.members - -`spec.members` field specifies a list of targets to backup. Each member consists of the following fields: - -- **target :** Each member has a target specification. The target specification of a member is the same as the target specification of a `BackupConfiguration` explained [here](/docs/concepts/crds/backupconfiguration/index.md#spectarget). - -- **task :** `task` specifies the name and parameters of the [Task](/docs/concepts/crds/task/index.md) crd to use to backup the target. For more details, please see [here](/docs/concepts/crds/backupconfiguration/index.md#spectask). - -- **runtimeSettings :** `runtimeSettings` allows to configure runtime environment for the backup sidecar or job. You can specify runtime settings at both pod level and container level. For more details, please see [here](/docs/concepts/crds/backupconfiguration/index.md#specruntimesettings). - -- **tempDir :** Stash mounts an `emptyDir` for holding temporary files. It is also used for `caching` for faster backup performance. You can configure the `emptyDir` using `tempDir` section. You can also disable `caching` using this field. For more details, please see [here](/docs/concepts/crds/backupconfiguration/index.md#spectempdir). - -- **interimVolumeTemplate :** For some targets (i.e. some databases), Stash can't directly pipe the dumped data to the uploading process. In this case, it has to store the dumped data temporarily before uploading to the backend. `interimVolumeTemplate` specifies a PVC template for holding those data temporarily. Stash will create a PVC according to the template and use it to store the data temporarily. This PVC will be deleted according to the [backupHistoryLimit](#specbackuphistorylimit). For more details, please see [here](/docs/concepts/crds/backupconfiguration/index.md#specinterimvolumetemplate). - -- **hooks :** Each member has its own hook field which allows you to execute member-specific pre-backup or post-backup hooks. For more details about hooks, please visit [here](/docs/concepts/crds/backupconfiguration/index.md#spechooks). - -#### spec.hooks - -`spec.hooks` allows performing some global actions before and after the backup process of the members. You can send HTTP requests to a remote server via `httpGet` or `httpPost`. You can check whether a TCP port is open using `tcpSocket` hooks. You can also execute some commands using `exec` hook. - -- **spec.hooks.preBackup:** `spec.hooks.preBackup` hooks are executed on each backup session before taking backup of any of the members. -- **spec.hooks.postBackup:** `spec.hooks.postBackup` hooks are executed on each backup session after taking backup of all the members. - -For more details on how hooks work in Stash and how to configure different types of hook, please visit [here](/docs/guides/hooks/overview/index.md). - -#### spec.runtimeSettings - -`spec.runtimeSettings` This runtime settings is applicable for CronJob(used to create `BackupSession`) only. For more details, please see [here](/docs/concepts/crds/backupconfiguration/index.md#specruntimesettings). - -#### spec.backupHistoryLimit - -`spec.backupHistoryLimit` specifies the number of `BackupSession` and its associate resources (Job, PVC etc.) to keep for debugging purposes. The default value of this field is 1. Stash will clean up the old `BackupSession` and it's associate resources after each backup session according to `backupHistoryLimit`. - -#### spec.paused - -`spec.paused` can be used as `enable/disable` switch for backup. If it is set `true`, Stash will not take any backup of the target specified by this BackupBatch. - -#### spec.retentionPolicy - -`spec.retentionPolicy` specifies the policy to follow for cleaning old snapshots. For more details, please see [here](/docs/concepts/crds/backupconfiguration/index.md#specretentionpolicy). - - -#### spec.retryConfig - -`spec.retryConfig` specifies a retry logic for failed backup. For more details, please see [here](/docs/concepts/crds/backupconfiguration/index.md#specretryconfig). - -#### spec.timeOut - -`spec.timeOut` specifies the maximum duration of the backup. For more details, please see [here](/docs/concepts/crds/backupconfiguration/index.md#spectimeout). - -### BackupBatch `Status` - -A `BackupBatch` object has the following fields in the `status` section. - -- **observedGeneration :** The most recent generation observed by the `BackupBatch` controller. - -- **conditions :** The `status.conditions` shows current backup setup condition for this BackupBatch. The following conditions are set by the Stash operator: - -| Condition Type | Usage | -| -------------------- | -------------------------------------------------------------------- | -| `RepositoryFound` | Indicates whether the respective Repository object was found or not. | -| `BackendSecretFound` | Indicates whether the respective backend secret was found or not. | -| `CronJobCreated` | Indicates whether the backup triggering CronJob was created or not. | - -- **memberConditions :** Shows current backup setup condition of the members of a `BackupBatch`. Each entry has the following two fields: - - **target :** Points to the respective target whose condition is shown here. - - **conditions:** Shows the current backup setup condition of this member. - -The following conditions are set for the members of a `BackupBatch`. - -| Condition Type | Usage | -| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `BackupTargetFound` | Indicates whether the backup target was found or not. | -| `StashSidecarInjected` | Indicates whether `stash` sidecar was injected into the targeted workload or not. This condition is set only for the target that uses the sidecar model. | - -## Next Steps - -- Learn how to configure `BackupBatch` to backup data from [here](/docs/guides/batch-backup/overview/index.md). diff --git a/docs/concepts/crds/backupblueprint/index.md b/docs/concepts/crds/backupblueprint/index.md index 5122580..7f79d01 100644 --- a/docs/concepts/crds/backupblueprint/index.md +++ b/docs/concepts/crds/backupblueprint/index.md @@ -11,121 +11,134 @@ menu_name: docs_{{ .version }} section_menu_id: concepts --- -> New to Stash? Please start [here](/docs/concepts/README.md). +> New to KubeStash? Please start [here](/docs/concepts/README.md). # BackupBlueprint ## What is BackupBlueprint -Stash uses 1-1 mapping among `Repository`, `BackupConfiguration` and the target. So, whenever you want to backup a target(workload/PV/PVC/database), you have to create a `Repository` and `BackupConfiguration` object. This could become tiresome when you are trying to backup similar types of target and the `Repository` and `BackupConfiguration` has only slight difference. To mitigate this problem, Stash provides a way to specify a blueprint for these two objects via `BackupBlueprint` crd. +KubeStash uses 1-1 mapping among `BackupConfiguration` and the target. So, whenever you want to backup a target(workload/PV/PVC/database), you have to create a `BackupConfiguration` object. This could become tiresome when you are trying to backup similar types of target and the `BackupConfiguration` has only slight difference. To mitigate this problem, KubeStash provides a way to specify a blueprint for this object via `BackupBlueprint` crd. -A `BackupBlueprint` is a Kubernetes `CustomResourceDefinition`(CRD) which specifies a blueprint for `Repository` and `BackupConfiguration` in a Kubernetes native way. +A `BackupBlueprint` is a Kubernetes `CustomResourceDefinition`(CRD) which specifies a blueprint for `BackupConfiguration` in a Kubernetes native way. -You have to create only one `BackupBlueprint` for all similar types of workloads (i.e. Deployment, DaemonSet, StatefulSet etc.). Then, you just need to add some annotations in the target workload. Stash will automatically create respective `Repository`, `BackupConfiguration` object using the blueprint. In Stash parlance, we call this process as **auto backup**. +You have to create only one `BackupBlueprint` for all similar types of workloads (i.e. Deployment, DaemonSet, StatefulSet etc.). Then, you just need to add some annotations in the target workload. KubeStash will automatically create respective `BackupConfiguration` object using the blueprint. In KubeStash parlance, we call this process as **auto backup**. ## BackupBlueprint CRD Specification - Like any official Kubernetes resource, a `BackupBlueprint` has `TypeMeta`, `ObjectMeta` and `Spec` sections. However, unlike other Kubernetes resources, it does not have a `Status` section. -A sample `BackupBlueprint` object to auto backup the volumes of a Deployment is shown below, +A sample `BackupBlueprint` object to auto backup the volumes of workload is shown below, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupBlueprint metadata: - name: workload-backup-blueprint + name: sample-blueprint + namespace: demo spec: - # ============== Blueprint for Repository ========================== - backend: - gcs: - bucket: stash-backup - prefix: stash/${TARGET_NAMESPACE}/${TARGET_KIND}/${TARGET_NAME} - storageSecretName: gcs-secret - wipeOut: false - # backupNamespace: stash - # ============== Blueprint for BackupConfiguration ================= - schedule: "* * * * *" - backupHistoryLimit: 3 - timeOut: 30m - retryConfig: - maxRetry: 3 - delay: 10m - # task: # no task section is required for workload data backup - # name: workload-backup - runtimeSettings: - container: - securityContext: - runAsUser: 2000 - runAsGroup: 2000 - tempDir: - medium: "Memory" - sizeLimit: "1Gi" - disableCaching: false - retentionPolicy: - name: 'keep-last-5' - keepLast: 5 - prune: true + usagePolicy: + allowedNamespaces: + from: All + backupConfigurationTemplate: + deletionPolicy: OnDelete + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: backup-every-five-minutes + sessionHistoryLimit: 3 + scheduler: # CronJob specification + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: ${repoName} + backend: gcs-backend + directory: ${namespace}/${targetName} + encryptionSecret: + name: encry-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: LogicalBackup + params: + paths: ${paths} + retryConfig: + maxRetry: 2 + delay: 1m ``` -The sample `BackupBlueprint` that has been shown above can be used to backup Deployments, DaemonSets, StatefulSets, ReplicaSets and ReplicationControllers. You only have to add some annotations to these workloads. For more details on how auto backup works in Stash, please visit [here](/docs/guides/auto-backup/overview/index.md). +The sample `BackupBlueprint` that has been shown above can be used to backup Deployments, DaemonSets, StatefulSets. You only have to add some annotations to these workloads. For more details on how auto backup works in KubeStash, please visit [here](/docs/guides/auto-backup/overview/index.md). Here, we are going to describe the various sections of `BackupBlueprint` crd. ### BackupBlueprint `Spec` -We can divide BackupBlueprint's `.spec` section into two parts. One part specifies a blueprint for `Repository` object and other specifies a blueprint for `BackupConfiguration` object. - -#### Repository Blueprint - -You can configure `Repository` blueprint using `spec.backend` field and `spec.wipeOut` field. - -- **spec.backend :** `spec.backend` field is backend specification similar to [spec.backend](/docs/concepts/crds/repository/index.md#specbackend) field of a `Repository` crd. There is only one difference. You can now templatize `prefix` section (`subPath` for local volume) of the backend to store backed up data of different workloads at different directory. You can use the following variables to templatize `spec.backend` field: - -| Variable | Usage | -| --------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -| `TARGET_API_VERSION` | API version of the target | -| `TARGET_KIND` | Resource kind of the target | -| `TARGET_NAMESPACE` | Namespace of the target | -| `TARGET_NAME` | Name of the target | -| `TARGET_RESOURCE` | Plural form of the target kind. i.e. `deployments`, `statefulsets` etc. | - -The following variables are available only for database backup. - -| Variable | Usage | -| --------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -| `TARGET_APP_GROUP` | Represents the application group where the respective app belongs (i.e: `kubedb.com`). | -| `TARGET_APP_RESOURCE` | Represents the resource kind under that application group that the respective app works with (i.e: `postgres`). | -| `TARGET_APP_TYPE` | Represents the total types of the application. It's simply `TARGET_APP_GROUP/TARGET_APP_RESOURCE` (i.e: `kubedb.com/postgres`). | - - If you use the sample `BackupBlueprint` that has been shown above to backup a Deployment named `my-deploy` of `test` namespace, the backed up data will be stored in `stash/test/deployment/my-deploy` directory of the `stash-backup` bucket. Similarly, if you want to backup a StatefulSet with name `my-sts` of same namespace, the backed up data will be stored in `/stash/test/statefulset/my-sts` directory of the backend. - -- **spec.backend.\.storageSecretName:** specifies the name of the secret that holds the access credentials to the backend. - ->Note: `BackupBlueprint` is a non-namespaced crd. So, you can use a `BackupBlueprint` to backup targets in multiple namespaces. However, Storage Secret is a namespaced object. So, you have to manually create the secret in each namespace where you have a target for backup. - -- **spec.wipeOut :** `spec.wipeOut` indicates whether Stash should delete the respective backed up data from the backend if a user deletes a `Repository` crd. For more details, please visit [here](/docs/concepts/crds/repository/index.md#specwipeout). - -#### BackupConfiguration Blueprint - -You can provide a blueprint for the `BackupConfiguration` object that will be created for respective target using the following fields: - -- **spec.schedule :** `spec.schedule` is the schedule that will be used to create `BackupConfiguration` for respective target. For more details, please visit [here](/docs/concepts/crds/backupconfiguration/index.md#specschedule). - -- **spec.backupHistoryLimit :** `spec.backupHistoryLimit` specifies a limit for backup history to keep for debugging purposes. For more details, please visit [here](/docs/concepts/crds/backupconfiguration/index.md#specbackuphistroylimit). +A `BackupBlueprint` object has the following fields in the `spec` section. +- **usagePolicy** specifies a policy of how this `BackupBlueprint` will be used. For example, you can use `allowedNamespaces` policy to restrict the usage of this `BackupBlueprint` to particular namespaces. This field is optional. If you don't provide the usagePolicy, then it can be used only from the current namespace. -- **spec.backupNamespace :** `spec.backupNamespace` specifies the namespace where the backup resources (i.e. BackupConfiguration, BackupSession, Job, Repository etc.) will be created. - -- **spec.retryConfig :** `spec.retryConfig` specifies a retry logic for failed backup. For more details, please visit [here](/docs/concepts/crds/backupconfiguration/index.md#specretryconfig). - -- **spec.task :** `spec.task` specifies the name and the parameters of [Task](/docs/concepts/crds/task/index.md) to use to backup the target. You can template the name field with `TARGET_APP_VERSION` variable for database backup. Stash will replace this variable with respective database version. This will allow you to backup multiple database versions with the same `BackupBlueprint`. For more details, please check the following [guide](/docs/guides/auto-backup/database/index.md). - -- **spec.runtimeSettings :** `spec.runtimeSettings` allows to configure runtime environment for backup sidecar or job. For more details, please visit [here](/docs/concepts/crds/backupconfiguration/index.md#specruntimesettings). - -- **spec.tempDir :** `spec.tempDir` specifies the temporary volume setting that will be used to create respective `BackupConfiguration` object. For more details, please visit [here](/docs/concepts/crds/backupconfiguration/index.md#spectempdir). - -- **spec.interimVolumeTemplate :** `spec.interimVolumeTemplate` specifies a PVC template for holding data temporarily before uploading to the backend. For more details, please visit [here](/docs/concepts/crds/backupconfiguration/index.md#specinterimvolumetemplate). +Here is an example of `spec.usagePolicy` that limits referencing the `BackupBlueprint` only from the same namespace, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: Same +``` +Here is an example of `spec.usagePolicy` that allows referencing the `BackupBlueprint` from only `prod` and `staging` namespaces, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: Selector + selector: + matchExpressions: + - key: "kubernetes.io/metadata.name" + operator: In + values: ["prod","staging"] +``` +Here is an example of `spec.usagePolicy` that allows referencing the `BackupBlueprint` from all namespaces, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: All +``` -- **spec.retentionPolicy :** `spec.retentionPolicy` specifies the retention policies that will be used to create respective `BackupConfiguration` object. For more details, please visit [here](/docs/concepts/crds/backupconfiguration/index.md#specretentionpolicy). +- **backupConfigurationTemplate** specifies the `BackupConfiguration` that will be created by `BackupBlueprint`. It consists of the following fields: + - **namespace** specifies the namespace of the `BackupConfiguration`. The field is optional. If you don't provide the namespace, then `BackupConfiguration` will be created in the `BackupBlueprint` namespace. + - **backends** specifies a list of storage references where the backed up data will be stored. The respective `BackupStorage`s can be in a different namespace than the `BackupConfiguration`. However, it must be allowed by the `usagePolicy` of the `BackupStorage` to refer from this namespace. This field is optional, if you don't provide any backend here, KubeStash will use the default `BackupStorage` for the namespace. If a default `BackupStorage` does not exist in the same namespace, then KubeStash will look for a default `BackupStorage` in other namespaces that allows using it from the `BackupConfiguration` namespace. Each backend has the following fields: + +| Field | Usage | +|-------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `name` | specifies an identifier for this storage. This name must be **unique**. | +| `storageRef` | refers to the CR that holds the information of a storage. You can refer to the `BackupStorage` CR of a different namespace as long as it is allowed by the `usagePolicy` of the `BackupStorage`. | +| `retentionPolicy` | refers to a `RetentionPolicy` CRs which defines how to cleanup the old `Snapshots`. This field is optional, if you don't provide this field, KubeStash will use the default `RetentionPolicy` for the namespace. If there is no default `RetentionPolicy` for the namespace, then KubeStash will find a `RetentionPolicy` from other namespaces that is allowed to use from the current namespace. | + + - **sessions** specifies a list of session template for backup. You can use custom variables in your template then provide the variable value through annotations. Each session has the following fields: + - **name** specifies an identifier for this session. This name must be **unique**. + - **scheduler** specifies the configuration for backup triggering CronJob. To learn about the fields under `scheduler`, see [Scheduler Spec](#scheduler-spec). + - **hooks** specifies the backup hooks that should be executed before and/or after the backup. Hooks has two fields: + - **preBackup** specifies a list of hooks that will be executed before backup. To learn about the fields under `preBackup`, see [HookInfo](#hookinfo). + - **postBackup** specifies a list of hooks that will be executed after backup. To learn about the fields under `postBackup`, see [HookInfo](#hookinfo). + - **failurePolicy** specifies what to do if the backup fail. Valid values are: + - **Fail** KubeStash should mark the backup as failed if any component fail to complete its backup. This is the default behavior. + - **Retry** KubeStash will retry to backup the failed component according to the `retryConfig`. + - **retryConfig** specifies the behavior of retry in case of a backup failure. RetryConfig has the following fields: + - **maxRetry** specifies the maximum number of times Stash should retry the backup/restore process. By default, KubeStash will retry only 1 time. + - **delay** The amount of time to wait before next retry. If you don't specify this field, KubeStash will retry immediately. Format: 30s, 2m, 1h etc. + - **timeout** specifies the maximum duration of backup. BackupSession will be considered Failed if backup does not complete within this time limit. By default, KubeStash don't set any timeout for backup. + - **sessionHistoryLimit** specifies how many backup Jobs and associate resources KubeStash should keep for debugging purpose. The default value is 1. + - **addon** specifies addon configuration that will be used to backup the target. Addon has the following fields: + - **name** specifies the name of the addon that will be used for the backup purpose. + - **tasks** specifies a list of backup tasks and their configuration parameters. To learn about the fields under `task`, see [Task Reference](#task-reference). + - **containerRuntimeSettings** specifies runtime settings for the backup executor container. More information can be found [here](#container-level-runtime-settings). + - **jobTemplate** specifies runtime configurations for the backup Job. More information can be found [here](#podtemplate-spec). + - **deletionPolicy** specifies whether the `BackupConfiguration` will be deleted on `BackupBlueprint` deletion. This field is optional, if you don't provide deletionPolicy, then `BackupConfiguration` will not be deleted on `BackupBlueprint` deletion. The only valid value for this field is `OnDelete` which specifies the `BackupConfiguration` will be deleted on `BackupBlueprint` deletion. ## Next Steps diff --git a/docs/concepts/crds/backupconfiguration/index.md b/docs/concepts/crds/backupconfiguration/index.md index 9126926..5a26128 100644 --- a/docs/concepts/crds/backupconfiguration/index.md +++ b/docs/concepts/crds/backupconfiguration/index.md @@ -5,218 +5,265 @@ menu: identifier: backupconfiguration-overview name: BackupConfiguration parent: crds - weight: 10 + weight: 15 product_name: kubestash menu_name: docs_{{ .version }} section_menu_id: concepts --- -> New to Stash? Please start [here](/docs/concepts/README.md). +> New to KubeStash? Please start [here](/docs/concepts/README.md). # BackupConfiguration -## What is BackupConfiguration - -A `BackupConfiguration` is a Kubernetes `CustomResourceDefinition`(CRD) which specifies the backup target, parameters(schedule, retention policy etc.) and a `Repository` object that holds snapshot storage information in a Kubernetes native way. +## What is Snapshot +A `BackupConfiguration` is a Kubernetes `CustomResourceDefinition`(CRD) which specifies the backup target, the backends reference and the sessions that specifies when and how to take backup in a Kubernetes native way. You have to create a `BackupConfiguration` object for each backup target. A backup target can be a workload, database or a PV/PVC. ## BackupConfiguration CRD Specification +Like any official Kubernetes resource, a `BackupConfiguration` has `TypeMeta`, `ObjectMeta`, `Spec` and `Status` sections. -Like any official Kubernetes resource, a `BackupConfiguration` has `TypeMeta`, `ObjectMeta` and `Spec` sections. However, unlike other Kubernetes resources, it does not have a `Status` section. - -A sample `BackupConfiguration` object to backup the volumes of a Deployment is shown below: - +A sample `BackupConfiguration` object to backup the volumes of a StatefulSet is shown below: ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: demo-backup + name: sample-backup-sts namespace: demo spec: - driver: Restic - repository: - name: local-repo - namespace: demo - # task: - # name: workload-backup # task field is not required for workload data backup but it is necessary for database backup. - schedule: "* * * * *" # backup at every minutes - paused: false - backupHistoryLimit: 3 - timeOut: 2h + backends: + - name: gcs-backend + retentionPolicy: + name: demo-retention + namespace: demo + storageRef: + name: gcs-storage + namespace: demo + sessions: + - addon: + name: workload-addon + tasks: + - name: LogicalBackup + params: + exclude: /source/data/lost+found + paths: /source/data + targetVolumes: + volumeMounts: + - mountPath: /source/data + name: source-data + failurePolicy: Fail + name: demo-session + repositories: + - backend: gcs-backend + directory: /demo/data + encryptionSecret: + name: encry-secret + namespace: demo + name: gcs-demo-repo + retryConfig: + delay: 1m0s + maxRetry: 2 + scheduler: + jobTemplate: + backoffLimit: 1 + template: + controller: {} + metadata: {} + spec: + resources: {} + schedule: '*/5 * * * *' + sessionHistoryLimit: 1 target: - alias: app-data - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-demo - paths: - - /source/data - exclude: - - /source/data/not-important.txt - - /source/data/*.html - - /source/data/tmp/* - volumeMounts: - - name: source-data - mountPath: /source/data - hooks: - preBackup: - exec: - command: - - /bin/sh - - -c - - echo "Sample PreBackup hook demo" - containerName: my-app-container - postBackup: - executionPolicy: Always - exec: - command: - - /bin/sh - - -c - - echo "Sample PostBackup hook demo" - containerName: my-app-container - runtimeSettings: - container: - resources: - requests: - memory: 256M - limits: - memory: 256M - securityContext: - runAsUser: 2000 - runAsGroup: 2000 - nice: - adjustment: 5 - ionice: - class: 2 - classData: 4 - pod: - imagePullSecrets: - - name: my-private-registry-secret - serviceAccountName: my-backup-svc - tempDir: - medium: "Memory" - sizeLimit: "2Gi" - disableCaching: false - retryConfig: - maxRetry: 3 - delay: 10m - retentionPolicy: - name: 'keep-last-5' - keepLast: 5 - prune: true + apiGroup: apps + kind: StatefulSet + name: sample-sts + namespace: demo +status: + backends: + - name: gcs-backend + ready: true + retentionPolicy: + found: true + ref: + name: demo-retention + namespace: demo + storage: + phase: Ready + ref: + name: gcs-storage + namespace: demo + conditions: + - lastTransitionTime: "2023-12-14T08:38:01Z" + message: Validation has been passed successfully. + reason: ResourceValidationPassed + status: "True" + type: ValidationPassed + dependencies: + - found: true + kind: Addon + name: workload-addon + phase: Ready + repositories: + - name: gcs-demo-repo + phase: Ready + sessions: + - conditions: + - lastTransitionTime: "2023-12-14T08:38:01Z" + message: Scheduler has been ensured successfully. + reason: SchedulerEnsured + status: "True" + type: SchedulerEnsured + name: demo-session + targetFound: true ``` - Here, we are going to describe the various sections of `BackupConfiguration` crd. -### BackupConfiguration `Spec` - +## BackupConfiguration `Spec` A `BackupConfiguration` object has the following fields in the `spec` section. -#### spec.driver - -`spec.driver` indicates the mechanism used to backup a target. Currently, Stash supports `Restic` and `VolumeSnapshotter` as drivers. The default value of this field is `Restic`. - -| Driver | Usage | -| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `Restic` | Used to backup workload data, persistent volumes data and databases. It uses [restic](https://restic.net) to backup the target. | -| `VolumeSnapshotter` | Used to take snapshot of PersistentVolumeClaims of a targeted workload. It leverages Kubernetes [VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/) crd and CSI driver to snapshot the PVCs. | - -#### spec.target - -`spec.target` field indicates the target for backup runs. This field consists of the following sub-fields: - -- **spec.target.alias :** The alias is used as an identifer of the backed up data in the backend. This is particularly useful for `BackupBatch` where multiple targets are backed up into a single repository. - -- **spec.target.ref :** `spec.target.ref` refers to the target of backup. You have to specify `apiVersion`, `kind` and `name` of the target. Stash will use this information to inject a sidecar to the target or to create a backup job for it. - -- **spec.target.paths :** `spec.target.paths` specifies list of file paths to backup. - -- **spec.target.exclude :** Specifies a list of pattern for the files that should be ignored during backup. Stash will not backup the files that matches these patterns. - -- **spec.target.volumeMounts :** `spec.target.volumeMounts` are the list of volumes and their `mountPath`s that contain the target file paths. Stash will mount these volumes inside a sidecar container or a backup job. - -- **spec.target.snapshotClassName:** `spec.target.snapshotClassName` indicates the [VolumeSnapshotClass](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/) to use for volume snasphotting. Use this field only if `spec.driver` is set to `VolumeSnapshotter`. - -#### spec.repository - -`spec.repository` specifies the name and namespace of the Repository CR that holds the necessary backend information where the backed up data will be stored. - -- `spec.repository.name` specifies the name of the Repository CR. -- `spec.repository.namespace` specifies the namespace of the Repository. If you don't provide this field, Stash will look for the Repository CR in the same namespace as the BackupConfiguration. - -#### spec.schedule - -`spec.schedule` is a [cron expression](https://en.wikipedia.org/wiki/Cron) that specifies the schedule of backup. Stash creates a Kubernetes [CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/) with this schedule. - -#### spec.backupHistoryLimit - -`spec.backupHistoryLimit` specifies the number of `BackupSession` and its associate resources (Job, PVC etc.) to keep for debugging purposes. The default value of this field is 1. Stash will cleanup the old `BackupSession` and it's associate resources after each backup session according to `backupHistoryLimit`. Stash will always keep the last completed BackupSession when `backuphistorylimit>0`. It will keep the last completed BackupSession even if it exceeds the history limit. This will help to keep the backup history when a backup gets skipped due to another running backup. - -#### spec.timeOut - -`spec.timeOut` specifies the maximum amount of time to wait for the backup to complete. If the backup doesn't complete within this time limit, Stash will mark the respective BackupSession as `Failed`. You can specify the timeout in the following format: - -- Seconds `30s` -- Minutes `10m` -- Hours `1h` -- Combination of seconds, minutes, and hours `10m30s`, `1h30m` etc. - -Stash does not support providing days (`d`) in the `timeOut` field. Use the equivalent hours instead. - -#### spec.task - -`spec.task` specifies the name and parameters of the [Task](/docs/concepts/crds/task/index.md) crd to use to backup the target. - -- **spec.task.name:** `spec.task.name` indicates the name of the `Task` to use for this backup process. -- **spec.task.params:** `spec.task.params` is an array of custom parameters to use to configure the task. - -> `spec.task` section is not required for backing up workload data (i.e. Deployment, DaemonSet, StatefulSet etc.). However, it is necessary for backing up databases and stand-alone PVCs. - -#### spec.paused - -`spec.paused` can be used as `enable/disable` switch for backup. If it is set `true`, Stash will not take any backup of the target specified by this BackupConfiguration. - -#### spec.hooks - -`spec.hooks` allows performing some actions before and after the backup process. You can send HTTP requests to a remote server via `httpGet` or `httpPost` hooks. You can check whether a TCP socket is open using `tcpSocket` hook. You can also execute some commands into your application pod using `exec` hook. - -- **spec.hooks.preBackup:** `spec.hooks.preBackup` hooks are executed before the backup process. -- **spec.hooks.postBackup:** `spec.hooks.postBackup` hooks are executed after the backup process. Unlike the `preBackup` hook, `postBackup` hook has an extra field named `executionPolicy` which let you execute hook based on the backup status. Currently, it support the following values: - - `Always`: The hook will be executed after the backup process no matter the backup has failed or succeeded. This is the default behavior. - - `OnSuccess`: The hook will be executed after the backup process only if the backup has succeeded. - - `OnFailure`: The hook will be executed after the backup process only if the backup has failed. - - `OnFinalRetryFailure`: The hook will be executed after the backup process only if the backup has failed with no more retry attempts left. -For more details on how hooks work in Stash and how to configure different types of hook, please visit [here](/docs/guides/hooks/overview/index.md). - -#### spec.runtimeSettings - -`spec.runtimeSettings` allows to configure runtime environment for the backup sidecar or job. You can specify runtime settings at both pod level and container level. - -- **spec.runtimeSettings.container** - - `spec.runtimeSettings.container` is used to configure the backup sidecar/job at container level. You can configure the following container level parameters: - -| Field | Usage | -| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `resources` | Compute resources required by the sidecar container or backup job. To learn how to manage resources for containers, please visit [here](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/). | -| `livenessProbe` | Periodic probe of backup sidecar/job container's liveness. Container will be restarted if the probe fails. | -| `readinessProbe` | Periodic probe of backup sidecar/job container's readiness. Container will be removed from service endpoints if the probe fails. | -| `lifecycle` | Actions that the management system should take in response to container lifecycle events. | -| `securityContext` | Security options that backup sidecar/job's container should run with. For more details, please visit [here](https://kubernetes.io/docs/concepts/policy/security-context/). | -| `nice` | Set CPU scheduling priority for backup process. For more details about `nice`, please visit [here](https://www.askapache.com/optimize/optimize-nice-ionice/#nice). | -| `ionice` | Set I/O scheduling class and priority for backup process. For more details about `ionice`, please visit [here](https://www.askapache.com/optimize/optimize-nice-ionice/#ionice). | -| `env` | A list of the environment variables to set in the sidecar container or backup job's container. | -| `envFrom` | This allows to set environment variables to the container that will be created for this function from a Secret or ConfigMap. | - -- **spec.runtimeSettings.pod** - - `spec.runtimeSettings.pod` is used to configure backup job in pod level. You can configure the following pod level parameters, +**spec.target** + +`spec.target` refers to the target of backup. This field consists of `apiGroup`, `kind`, `name` and `namespace`. The backup target can be in a different namespace than the `BackupConfiguration`. + +**spec.backends** + +`spec.backends` specifies a list of storage references where the backed up data will be stored. The respective `BackupStorages` can be in a different namespace than the `BackupConfiguration`. +However, it must be allowed by the `usagePolicy` of the `BackupStorage` to refer from this namespace. This field is optional, if you don't provide any backend here, KubeStash will use the default `BackupStorage` +for the namespace. If a default `BackupStorage` does not exist in the same namespace, then KubeStash will look for a default `BackupStorage` in other namespaces that allows using it from the `BackupConfiguration` +namespace. Each backend has the following fields: + +| Field | Usage | +|-------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `name` | specifies an identifier for this storage. This name must be **unique**. | +| `storageRef` | refers to the CR that holds the information of a storage. You can refer to the `BackupStorage` CR of a different namespace as long as it is allowed by the `usagePolicy` of the `BackupStorage`. | +| `retentionPolicy` | refers to a `RetentionPolicy` CRs which defines how to cleanup the old `Snapshots`. This field is optional, if you don't provide this field, KubeStash will use the default `RetentionPolicy` for the namespace. If there is no default `RetentionPolicy` for the namespace, then KubeStash will find a `RetentionPolicy` from other namespaces that is allowed to use from the current namespace. | + +**spec.sessions** + +`spec.sessions` defines a list of session configuration that specifies when and how to take backup. Each session has the following fields: +- **name** specifies an identifier for this session. This name must be **unique**. +- **scheduler** specifies the configuration for backup triggering CronJob. To learn about the fields under `scheduler`, see [Scheduler Spec](#scheduler-spec). +- **hooks** specifies the backup hooks that should be executed before and/or after the backup. Hooks has two fields: + - **preBackup** specifies a list of hooks that will be executed before backup. To learn about the fields under `preBackup`, see [HookInfo](#hookinfo). + - **postBackup** specifies a list of hooks that will be executed after backup. To learn about the fields under `postBackup`, see [HookInfo](#hookinfo). +- **failurePolicy** specifies what to do if the backup fail. Valid values are: + - **Fail** KubeStash should mark the backup as failed if any component fail to complete its backup. This is the default behavior. + - **Retry** KubeStash will retry to backup the failed component according to the `retryConfig`. +- **retryConfig** specifies the behavior of retry in case of a backup failure. RetryConfig has the following fields: + - **maxRetry** specifies the maximum number of times Stash should retry the backup/restore process. By default, KubeStash will retry only 1 time. + - **delay** The amount of time to wait before next retry. If you don't specify this field, KubeStash will retry immediately. Format: 30s, 2m, 1h etc. +- **timeout** specifies the maximum duration of backup. BackupSession will be considered Failed if backup does not complete within this time limit. By default, KubeStash don't set any timeout for backup. +- **sessionHistoryLimit** specifies how many backup Jobs and associate resources KubeStash should keep for debugging purpose. The default value is 1. +- **addon** specifies addon configuration that will be used to backup the target. Addon has the following fields: + - **name** specifies the name of the addon that will be used for the backup purpose. + - **tasks** specifies a list of backup tasks and their configuration parameters. To learn about the fields under `task`, see [Task Reference](#task-reference). + - **containerRuntimeSettings** specifies runtime settings for the backup executor container. More information can be found [here](#container-level-runtime-settings). + - **jobTemplate** specifies runtime configurations for the backup Job. More information can be found [here](#podtemplate-spec). + +#### Scheduler Spec +Scheduler Spec specifies the configuration for the backup triggering CronJob for a session. `scheduler` has the following fields: +- **schedule** The schedule in Cron format, see [here](https://en.wikipedia.org/wiki/Cron) to learn more. +- **startingDeadlineSeconds** Optional deadline in seconds for starting the job if it misses scheduled time for any reason. Missed jobs executions will be counted as failed ones. +- **concurrencyPolicy** Specifies how to treat concurrent executions of a Job. Valid values are: + - **Allow** (default) allows CronJobs to run concurrently. + - **Forbid** forbids concurrent runs, skipping next run if previous run hasn't finished yet. + - **Replace** cancels currently running job and replaces it with a new one. +- **suspend** This flag tells the controller to suspend subsequent executions, it does not apply to already started executions. Defaults to false. +- **successfulJobsHistoryLimit** The number of successful finished jobs to retain. Value must be non-negative integer. Defaults to 3. +- **failedJobsHistoryLimit** The number of failed finished jobs to retain. Value must be non-negative integer. Defaults to 1. +- **jobTemplate** Specifies the job that will be created when executing a CronJob. JobTemplate has the following fields: + + | Field | Usage | + |---------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| + | `parallelism` | Specifies the maximum desired number of pods the job should run at any given time. The actual number of pods running in steady state will be less than this number when ((`.spec.completions` - `.status.successful)` < `.spec.parallelism`), i.e. when the work left to do is less than max parallelism. More info can be found [here](https://kubernetes.io/docs/concepts/workloads/controllers/job/#parallel-jobs) | + | `completions` | Specifies the desired number of successfully finished pods the job should be run with. Setting to nil means that the success of any pod signals the success of all pods, and allows parallelism to have any positive value. Setting to 1 means that parallelism is limited to 1 and the success of that pod signals the success of the job. More info [here](https://kubernetes.io/docs/concepts/workloads/controllers/job/#parallel-jobs) | + | `activeDeadlineSeconds` | Specifies the duration in seconds relative to the startTime that the job may be continuously active before the system tries to terminate it; value must be positive integer. If a Job is suspended (at creation or through an update), this timer will effectively be stopped and reset when the Job is resumed again. | + | `backoffLimit` | Specifies the number of retries before marking this job failed. Defaults to 6. | + | `template` | Describes the pod that will be created when executing a job. To know more about the fields in `template`, see [PodTemplate Spec]() | + | `ttlSecondsAfterFinished` | `ttlSecondsAfterFinished` limits the lifetime of a Job that has finished execution (either Complete or Failed). If this field is set, `ttlSecondsAfterFinished` after the Job finishes, it is eligible to be automatically deleted. When the Job is being deleted, its lifecycle guarantees (e.g. finalizers) will be honored. If this field is unset, the Job won't be automatically deleted. If this field is set to zero, the Job becomes eligible to be deleted immediately after it finishes. This field is alpha-level and is only honored by servers that enable the TTLAfterFinished feature. | + | `completionMode` | CompletionMode specifies how Pod completions are tracked. More Information can be found [here](https://kubernetes.io/docs/concepts/workloads/controllers/job/#completion-mode) | + | `suspend` | Suspend specifies whether the Job controller should create Pods or not. If a Job is created with suspend set to true, no Pods are created by the Job controller. If a Job is suspended after creation (i.e. the flag goes from false to true), the Job controller will delete all active Pods associated with this Job. Users must design their workload to gracefully handle this. Suspending a Job will reset the StartTime field of the Job, effectively resetting the ActiveDeadlineSeconds timer too. This is an alpha field and requires the SuspendJob feature gate to be enabled; otherwise this field may not be set to true. Defaults to false. | + +#### PodTemplate Spec +PodTemplate Spec describes the data a pod should have when created from a template. `template` has the following fields: +- **metadata** specifies standard object's metadata. It contains `labels` and `annotations` fields. +- **controller** specifies workload controller's metadata. It contains `labels` and `annotations` fields. +- **spec** specifies the desired behavior of the pod. It contains the following fields: + +| Field | Usage | +|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `volumes` | List of volumes that can be mounted by containers belonging to the pod. More info can be found [here](https://kubernetes.io/docs/concepts/storage/volumes) | +| `initContainers` | List of initialization containers belonging to the pod. More info can be found [here](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) | +| `terminationGracePeriodSeconds` | Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds. | +| `dnsPolicy` | Set DNS policy for the pod. Defaults to "ClusterFirst". Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`. DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy. To have DNS options set along with hostNetwork, you have to specify DNS policy explicitly to `ClusterFirstWithHostNet`. | +| `nodeSelector` | NodeSelector is a selector which must be true for the pod to fit on a node. Selector which must match a node's labels for the pod to be scheduled on that node. More info [here](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) | +| `serviceAccountName` | ServiceAccountName is the name of the ServiceAccount to use to run this pod. More info [here](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) | +| `hostNetwork` | Host networking requested for this pod. Use the host's network namespace. If this option is set, the ports that will be used must be specified. Default to false. | +| `hostPID` | Use the host's pid namespace. It is optional. Default to false. | +| `hostIPC` | Use the host's ipc namespace. It is optional. Default to false. | | +| `shareProcessNamespace` | Share a single process namespace between all of the containers in a pod. When this is set containers will be able to view and signal processes from other containers in the same pod, and the first process in each container will not be assigned PID 1. HostPID and ShareProcessNamespace cannot both be set. It is optional. Default to false. | +| `securityContext` | SecurityContext holds pod-level security attributes and common container settings. It is optional. Defaults to empty. See type description for default values of each field. | +| `imagePullSecrets` | ImagePullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images used by this PodSpec. If specified, these secrets will be passed to individual puller implementations for them to use. More info [here](https://kubernetes.io/docs/concepts/containers/images#specifying-imagepullsecrets-on-a-pod) | +| `affinity` | Affinity is a group of affinity scheduling rules. If specified, the pod's scheduling constraints. | +| `schedulerName` | If specified, the pod will be dispatched by specified scheduler. If not specified, the pod will be dispatched by default scheduler. | +| `tolerations` | If specified, the pod's tolerations. | +| `priorityClassName` | If specified, indicates the pod's priority. "system-node-critical" and "system-cluster-critical" are two special keywords which indicate the highest priorities with the former being the highest priority. Any other name must be defined by creating a PriorityClass object with that name. If not specified, the pod priority will be default or zero if there is no default. | +| `priority` | The priority value. Various system components use this field to find the priority of the pod. When Priority Admission Controller is enabled, it prevents users from setting this field. The admission controller populates this field from PriorityClassName. The higher the value, the higher the priority. | +| `dnsConfig` | Specifies the DNS parameters of a pod. Parameters specified here will be merged to the generated DNS configuration based on DNSPolicy. | +| `runtimeClassName` | RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run this pod. If no RuntimeClass resource matches the named class, the pod will not be run. If unset or empty, the "legacy" RuntimeClass will be used, which is an implicit class with an empty definition that uses the default runtime handler. More info [here](https://git.k8s.io/enhancements/keps/sig-node/585-runtime-class) | +| `enableServiceLinks` | EnableServiceLinks indicates whether information about services should be injected into pod's environment variables, matching the syntax of Docker links. It is optional. Defaults to true. | +| `topologySpreadConstraints` | TopologySpreadConstraints describes how a group of pods ought to spread across topology domains. Scheduler will schedule pods in a way which abides by the constraints. All topologySpreadConstraints are ANDed. | +| `args` | Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info [here](https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell) | +| `env` | List of environment variables to set in the container. Cannot be updated. | +| `resources` | Compute Resources required by the sidecar container. | +| `livenessProbe` | Periodic probe of container liveness. Container will be restarted if the probe fails. Controllers may set default LivenessProbe if no liveness probe is provided. To ignore defaulting, set the value to empty LivenessProbe "{}". Cannot be updated. More info [here](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes) | +| `readinessProbe` | Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. Controllers may set default ReadinessProbe if no readyness probe is provided. To ignore defaulting, set the value to empty ReadynessProbe "{}". More info [here](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes) | +| `lifecycle` | Actions that the management system should take in response to container lifecycle events. Cannot be updated. | +| `containerSecurityContext` | Security options the pod should run with. More info [here](https://kubernetes.io/docs/concepts/policy/security-context/) and [here](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) | | +| `volumeMounts` | Pod volumes to mount into the container's filesystem. Cannot be updated. | + +#### HookInfo +HookInfo specifies the information about the backup hooks. It contains the following fields: + +| Field | Usage | +|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `name` | specifies a name for the hook | +| `hookTemplate` | points to a HookTemplate CR that will be used to execute the hook. You can refer to a HookTemplate from other namespaces as long as your current namespace is allowed by the `usagePolicy` in the respective HookTemplate. | +| `params` | specifies parameters for the hook. You must provide the parameter in the HookTemplates desired structure. | +| `maxRetry` | MaxRetry specifies how many times Stash should retry the hook execution in case of failure. The default value of this field is 0 which means no retry. | +| `timeout` | Timeout specifies a duration in seconds that KubeStash should wait for the hook execution to be completed. If the hook execution does not finish within this time period, KubeStash will consider this hook execution as failure. Then, it will be re-tried according to MaxRetry policy. | +| `executionPolicy` | ExecutionPolicy specifies when to execute the hook. Valid values are:
  • **Always** KubeStash will execute this hook no matter the backup/restore failed. This is the default execution policy.
  • **OnSuccess** KubeStash will execute this hook only if the backup/restore has succeeded.
  • **OnFailure** KubeStash will execute this hook only if the backup/restore has failed.
| +| `variables` | specifies a list of variables and their sources that will be used to resolve the HookTemplate. | +| `volumes` | indicates the list of volumes of targeted application that should be mounted on the hook executor. Use this field only for `Function` type hook executor. | +| `volumeMounts` | specifies the mount for the volumes specified in `Volumes` section. Use this field only for `Function` type hook executor. | +| `runtimeSettings` | specifies runtime configurations for the hook executor Job. Use this field only for `Function` type hook executor. To know more about the fields in `runtimeSettings`, see [Runtime Settings](#runtime-settings) | + +#### Runtime Settings +Runtime Settings allows to configure runtime environment for the `Function` type hook executor job. You can specify runtime settings at both pod level and container level. + +##### Container Level Runtime Settings +`runtimeSettings.container` is used to configure the corresponding job at container level. You can configure the following container level parameters: + +| Field | Usage | +|-------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `resources` | Compute resources required by the corresponding job containers. To learn how to manage resources for containers, please visit [here](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/). | +| `livenessProbe` | Periodic probe of corresponding job container's liveness. Container will be restarted if the probe fails. | +| `readinessProbe` | Periodic probe of corresponding job container's readiness. Container will be removed from service endpoints if the probe fails. | +| `lifecycle` | Actions that the management system should take in response to container lifecycle events. | +| `securityContext` | Security options that corresponding job's container should run with. For more details, please visit [here](https://kubernetes.io/docs/concepts/policy/security-context/). | +| `nice` | Set CPU scheduling priority for current process. For more details about `nice`, please visit [here](https://www.askapache.com/optimize/optimize-nice-ionice/#nice). | +| `ionice` | Set I/O scheduling class and priority for current process. For more details about `ionice`, please visit [here](https://www.askapache.com/optimize/optimize-nice-ionice/#ionice). | +| `env` | A list of the environment variables to set in the corresponding job's container. | +| `envFrom` | This allows to set environment variables to the container that will be created for this function from a Secret or ConfigMap. | + +##### Pod Level Runtime Settings +`runtimeSettings.pod` is used to configure the corresponding job in pod level. You can configure the following pod level parameters: | Field | Usage | -| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `serviceAccountName` | Name of the `ServiceAccount` to use for the backup job. Stash sidecar will use the same `ServiceAccount` as the target workload. | -| `nodeSelector` | Selector which must be true for backup job pod to fit on a node. | +|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `podLabels` | The labels that will be attached with the respective Pod | +| `serviceAccountName` | Name of the `ServiceAccount` to use for the corresponding job. | +| `nodeSelector` | Selector which must be true for corresponding job pod to fit on a node. | | `automountServiceAccountToken` | Indicates whether a service account token should be automatically mounted into the backup pod. | | `nodeName` | `nodeName` is used to request to schedule backup job's pod onto a specific node. | | `securityContext` | Security options that backup job's pod should run with. For more details, please visit [here](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/). | @@ -230,65 +277,54 @@ For more details on how hooks work in Stash and how to configure different types | `runtimeClassName` | RuntimeClass is used for selecting the container runtime configuration. For more details, please visit [here](https://kubernetes.io/docs/concepts/containers/runtime-class/) | | `enableServiceLinks` | EnableServiceLinks indicates whether information about services should be injected into pod's environment variables. | -#### spec.tempDir - -Stash mounts an `emptyDir` for holding temporary files. It is also used for `caching` for faster backup performance. You can configure the `emptyDir` using `spec.tempDir` section. You can also disable `caching` using this field. The following fields are configurable in `spec.tempDir` section: - -- **spec.tempDir.medium :** Specifies the type of storage medium should back this directory. -- **spec.tempDir.sizeLimit :** Maximum limit of storage for this volume. -- **spec.tempDir.disableCaching :** Disable caching while backup. This may negatively impact backup performance. This is set to `false` by default. - -#### spec.interimVolumeTemplate - -For some targets (i.e. some databases), Stash can't directly pipe the dumped data to the uploading process. In this case, it has to store the dumped data temporarily before uploading to the backend. `spec.interimVolumeTemplate` specifies a PVC template for holding those data temporarily. Stash will create a PVC according to the template and use it to store the data temporarily. This PVC will be deleted according to the [spec.backupHistoryLimit](#specbackuphistorylimit). - ->Note that the usage of this field is different than `spec.tempDir` which is used for caching purpose. Stash has introduced this field because the `emptyDir` volume that is used for `spec.tempDir` does not play nice with large databases( i.e. 100Gi database). Also, it provides debugging capability as Stash keeps it until it hits the limit specified in `spec.backupHistoryLimit`. - -#### spec.retryConfig - -`spec.retryConfig` is an optional field the let users to specify a retry logic for failed backup. It has the following fields: - -- `spec.retryConfig.maxRetry` specifies the maximum number of times Stash should retry a failed backup. -- `spec.retryConfig.delay` specifies the amount of time to wait before retrying a failed backup. You can specify the delay in the following format: - - Seconds `30s` - - Minutes `10m` - - Hours `1h` - - Combination of seconds, minutes, and hours `10m30s`, `1h30m` etc. - - Stash does not support providing days (`d`) in the `delay` field. Use the equivalent hours instead. - -#### spec.retentionPolicy - -`spec.retentionPolicy` specifies the policy to follow for cleaning old snapshots. Following options are available to configure retention policy: - -| Policy | Value | `restic` forget command flag | Description | -| ------------- | ------- | ---------------------------- | -------------------------------------------------------------------------------------------------- | -| `name` | string | | Name of retention policy. You can provide any name. | -| `keepLast` | integer | --keep-last n | Never delete the **n** last (most recent) snapshots. | -| `keepHourly` | integer | --keep-hourly n | For the last **n** hours in which a snapshot was made, keep only the last snapshot for each hour. | -| `keepDaily` | integer | --keep-daily n | For the last **n** days which have one or more snapshots, only keep the last one for that day. | -| `keepWeekly` | integer | --keep-weekly n | For the last **n** weeks which have one or more snapshots, only keep the last one for that week. | -| `keepMonthly` | integer | --keep-monthly n | For the last **n** months which have one or more snapshots, only keep the last one for that month. | -| `keepYearly` | integer | --keep-yearly n | For the last **n** years which have one or more snapshots, only keep the last one for that year. | -| `keepTags` | array | --keep-tag | Keep all snapshots which have all tags specified by this option (can be specified multiple times). | -| `prune` | bool | --prune | If set `true`, Stash will cleanup unreferenced data from the backend. | -| `dryRun` | bool | --dry-run | Stash will not remove anything but print which snapshots would be removed. | - - -### BackupConfiguration `Status` +#### Task Reference +Task Reference specifies a task and its configuration parameters. A `task` contains the following fields: +- **name** indicates to the name of the task. +- **variables** specifies a list of variables and their sources that will be used to resolve the task. For more details, please visit [here](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/) +- **params** specifies parameters for the task. You must provide the parameter in the Addon desired structure. +- **targetVolumes** specifies which volumes from the target should be mounted in the backup job/container. It contains the following fields: + - **volumes** indicates the list of volumes of targeted application that should be mounted on the backup job. + - **volumeMounts** specifies the mount for the volumes specified in `Volumes` section. + - **volumeClaimTemplates** specifies a template for the PersistentVolumeClaims that will be created for each Pod in a StatefulSet. +- **addonVolumes** lets you overwrite the volume sources used in the VolumeTemplate section of Addon. Make sure that name of your volume matches with the name of the volume you want to overwrite. Each `addonVolume` contains the following fields: + - **name** specifies the name of the volume. + - **source** specifies the source of this volume or a template for volume to use. + +## BackupConfiguration `Status` A `BackupConfiguration` object has the following fields in the `status` section. - -- **observedGeneration :** The most recent generation observed by the `BackupConfiguration` controller. - -- **conditions :** The `spec.conditions` shows current backup setup condition for this BackupConfiguration. The following conditions are set by the Stash operator: - -| Condition Type | Usage | -| -------------------- | -------------------------------------------------------------------- | -| `RepositoryFound` | Indicates whether the respective Repository object was found or not. | -| `BackendSecretFound` | Indicates whether the respective backend secret was found or not. | -| `CronJobCreated` | Indicates whether the backup triggering CronJob was created or not. | -| `ValidationPassed` | Indicates whether the resource has passed validation checks or not. | +- **backends** specifies whether the backends exist or not. Each backend consists of the following fields: + - **name** indicates the backend name. + - **ready** indicates whether the respective `BackupStorage` is ready or not. + - **storage** indicates the status of the respective `BackupStorage`. It has the following fields: + - **ref** indicates to the `BackupStorage` object. + - **phase** indicates the current phase of the respective `BackupStorage` which can be `Ready` or `NotReady`. + - **reason** specifies the error messages found while checking the `BackupStorage` phase. + - **retentionPolicy** indicates the status of the respective `RetentionPolicy`. It has the following fields: + - **ref** indicates the `RetentionPolicy` object reference. + - **found** indicates whether the `RetentionPolicy` is Found or not. + - **reason** specifies the error messages found while checking the `RetentionPolicy`. +- **repositories** specifies whether the repositories have been successfully initialized or not. It consists of the following fields: + - **name** indicate the name of the `Repository`. + - **phase** indicates whether the respective Repository is ready or not. The value of this field can be `Ready` or `NotReady`. + - **reason** specifies the error messages found while ensuring the respective `Repository`. +- **dependencies** specifies whether the objects required by this `BackupConfiguration` exist or not. This field contains the resource reference and indicates whether the resource was found or not. +- **sessions** specifies status of the session specific resources. It consists of the following fields: + - **name** indicates the name of the session. + - **nextSchedule** specifies when the next backup will execute for this session. + - **conditions** specifies a list of conditions related to this session. The following condition is set by the KubeStash operator on each `.status.sessions`. + +| Condition Type | Usage | +|--------------------|----------------------------------------------------| +| `SchedulerEnsured` | indicates whether the Scheduler is ensured or not. | + +- **phase** represents the current state of the Backup Invoker. +- **targetFound** specifies whether the backup target exist or not. +- **conditions** represents list of conditions regarding this `BackupConfiguration`. The following condition is set by the KubeStash operator on a `BackupConfiguration`. + +| Condition Type | Usage | +|--------------------|-------------------------------------------------------------------| +| `ValidationPassed` | indicates the validation conditions of the CRD are passed or not. | ## Next Steps diff --git a/docs/concepts/crds/backupsession/index.md b/docs/concepts/crds/backupsession/index.md index bf555db..cdd688f 100644 --- a/docs/concepts/crds/backupsession/index.md +++ b/docs/concepts/crds/backupsession/index.md @@ -11,15 +11,15 @@ menu_name: docs_{{ .version }} section_menu_id: concepts --- -> New to Stash? Please start [here](/docs/concepts/README.md). +> New to KubeStash? Please start [here](/docs/concepts/README.md). # BackupSession ## What is BackupSession -A `BackupSession` is a Kubernetes `CustomResourceDefinition`(CRD) which represents a backup run of the respective target(s) referenced by a `BackupConfiguration`/`BackupBatch` in a Kubernetes native way. +A `BackupSession` is a Kubernetes `CustomResourceDefinition`(CRD) which represents a backup run of the respective target referenced by a `BackupConfiguration` in a Kubernetes native way. -Stash operator creates a Kubernetes `CronJob` according to the schedule defined in a `BackupConfiguration`/`BackupBatch`. On each backup schedule, this `CronJob` creates a `BackupSession` object. It points to the respective `BackupConfiguration`/`BackupBatch`. The controller that runs inside backup sidecar (in case of backup via jobs, it is stash operator itself) watches this `BackupSession` object and starts taking backup instantly. +KubeStash operator creates a Kubernetes `CronJob` according to the schedule defined in a `BackupConfiguration`. On each backup schedule, this `CronJob` creates a `BackupSession` object. It points to the respective `BackupConfiguration`. The `BackupSession` will trigger a backup executor (job, sidecar, multilevel jobs etc.) and take backup instantly. You can also create a `BackupSession` object manually to trigger backup at any time. @@ -27,99 +27,71 @@ You can also create a `BackupSession` object manually to trigger backup at any t Like any official Kubernetes resource, a `BackupSession` has `TypeMeta`, `ObjectMeta` and `Spec` , `Status` sections. -A sample `BackupSession` created for backing up a WordPress Application and it's components' is shown below, +A sample `BackupSession` created for backing up an Application and it's components' is shown below, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupSession metadata: - creationTimestamp: "2020-07-25T17:41:28Z" - labels: - app: stash - stash.appscode.com/invoker-name: wordpress-backup - stash.appscode.com/invoker-type: BackupBatch - name: wordpress-backup-1578458376 + creationTimestamp: "2023-12-14T08:40:01Z" + finalizers: + - kubestash.com/cleanup + generation: 1 + name: sample-backup-sts-demo-session-1702543201 namespace: demo + ownerReferences: + - apiVersion: core.kubestash.com/v1alpha1 + blockOwnerDeletion: true + controller: true + kind: BackupConfiguration + name: sample-backup-sts + uid: 83b71921-16be-4300-bc72-9eed5a94de58 + resourceVersion: "344530" + uid: 948cd21a-cc2d-4c4a-ba8f-d5045ffa32f9 spec: invoker: - apiGroup: stash.appscode.com - kind: BackupBatch - name: wordpress-backup + apiGroup: core.kubestash.com + kind: BackupConfiguration + name: sample-backup-sts retryLeft: 2 + session: demo-session status: conditions: - - lastTransitionTime: "2020-07-25T17:41:31Z" - message: Repository exist in the backend. - reason: BackendRepositoryFound + - lastTransitionTime: "2023-12-14T08:40:01Z" + message: Snapshots have been ensured successfully. + reason: SuccessfullyEnsuredSnapshots status: "True" - type: BackendRepositoryInitialized - - lastTransitionTime: "2020-07-25T17:41:48Z" - message: Successfully applied retention policy. - reason: SuccessfullyAppliedRetentionPolicy + type: SnapshotsEnsured + - lastTransitionTime: "2023-12-14T08:40:01Z" + message: Backup Executor has been ensured successfully. + reason: SuccessfullyEnsuredBackupExecutor status: "True" - type: RetentionPolicyApplied - - lastTransitionTime: "2020-07-25T17:41:50Z" - message: Repository integrity verification succeeded. - reason: SuccessfullyVerifiedRepositoryIntegrity + type: BackupExecutorEnsured + - lastTransitionTime: "2023-12-14T08:40:33Z" + reason: SuccessfullyCleanedSessionHistory status: "True" - type: RepositoryIntegrityVerified - - lastTransitionTime: "2020-07-25T17:41:50Z" - message: Successfully pushed repository metrics. - reason: SuccessfullyPushedRepositoryMetrics + type: SessionHistoryCleaned + - lastTransitionTime: "2023-12-14T08:40:33Z" + message: Metrics have been pushed successfully. + reason: SuccessfullyPushedMetrics status: "True" - type: RepositoryMetricsPushed + type: MetricsPushed phase: Succeeded - sessionDuration: 22.575920065s - sessionDeadline: "2020-07-25T17:46:28Z" - targets: - - phase: Succeeded - preBackupActions: - - InitializeBackendRepository + retentionPolicy: + - phase: Applied ref: - apiVersion: apps/v1 - kind: Deployment - name: wordpress - stats: - - duration: 831.018039ms - hostname: app - phase: Succeeded - snapshots: - - fileStats: - modifiedFiles: 0 - newFiles: 1 - totalFiles: 1 - unmodifiedFiles: 0 - name: b54ee4a0 - path: /var/www/html - processingTime: "0:00" - totalSize: 0 B - uploaded: 711 B - totalHosts: 1 - - phase: Succeeded - postBackupActions: - - ApplyRetentionPolicy - - VerifyRepositoryIntegrity - - SendRepositoryMetrics - ref: - apiVersion: appcatalog.appscode.com/v1alpha1 - kind: AppBinding - name: wordpress-mysql - stats: - - duration: 1.147010638s - hostname: db - phase: Succeeded - snapshots: - - fileStats: - modifiedFiles: 0 - newFiles: 1 - totalFiles: 1 - unmodifiedFiles: 0 - name: b30beb44 - path: dumpfile.sql - processingTime: "0:00" - totalSize: 0 B - uploaded: 3.408 MiB - totalHosts: 1 + name: demo-retention + namespace: demo + repository: gcs-demo-repo + snapshots: + - appRef: + apiGroup: apps + kind: StatefulSet + name: sample-sts + namespace: demo + name: gcs-demo-repo-sample-backup-sts-demo-session-1702543201 + phase: Succeeded + repository: gcs-demo-repo ``` Here, we are going to describe the various sections of a `BackupSession` object. @@ -128,17 +100,13 @@ Here, we are going to describe the various sections of a `BackupSession` object. #### metadata.name -`metadata.name` indicates the name of the `BackupSession`. This name is automatically generated by respective `CronJob` and it follows the following pattern: `-`. +`metadata.name` indicates the name of the `BackupSession`. This name is automatically generated by respective `CronJob` and it follows the following pattern: `--`. #### metadata.namespace -`metadata.namespace` indicates the name of the `BackupSession`. It is the same as the namespace of respective `BackupConfiguration`/`BackupBatch` object. - -#### metadata.labels +`metadata.namespace` indicates the name of the `BackupSession`. It is the same as the namespace of respective `BackupConfiguration` object. -`metadata.labels` holds respective `BackupConfiguration`/`BackupBatch` kind and name as a label. The stash backup sidecar container use this label to watch only the BackupSessions of that `BackupConfiguration`/`BackupBatch`. - ->If you create `BackupSession` manually to trigger a backup instantly, make sure that you have added `stash.appscode.com/invoker-type: ` and `stash.appscode.com/invoker-name: ` label to your `BackupSession`. Otherwise, it will not trigger backup for workloads (those resources that are backed up using sidecar). +>If you create `BackupSession` manually to trigger a backup instantly, make sure that you have provided a valid invoker at `spec.invoker` field to your `BackupSession`. ### BackupSession `Spec` @@ -148,110 +116,76 @@ A `BackupSession` object has the following fields in the `spec` section: `spec.invoker` specifies the `apiGroup`, `kind`, and `name` of the respective object which is responsible for invoking this backup session. +#### spec.session +`spec.session` specifies the name of the session that triggered this backup. + #### spec.retryLeft -`spec.retryLeft` specifies the number of retry attempt left for this backup session. +`spec.retryLeft` specifies the number of retry attempt left for this backup session. If this set to non-zero, KubeStash will create a new BackupSession if the current one fails. ### BackupSession `Status` -`.status` section of `BackupSession` shows stats and progress of backup process in this session.A backup sidecar container or job updates the respective fields under `.status` section after it completes its task. `.status` section consists of the following fields: +`.status` section of `BackupSession` shows stats and progress of backup process in this session. `.status` section consists of the following fields: #### status.phase `status.phase` indicates the overall phase of the backup process for this BackupSession. `status.phase` will be `Succeeded` only if the phase of all targets is `Succeeded`. If any of the targets fail to complete its backup, `status.phase` will be `Failed`. -#### status.sessionDuration +#### status.duration -`status.sessionDuration` indicates the total time taken to complete the backup of all targets in this session. +`status.duration` specifies the time required to complete the backup process. #### status.sessionDeadline -`status.sessionDeadline` indicates the the deadline of the backup process. `BackupSession` will be considered `Failed` if the backup does not complete within this deadline. - -#### status.retried - -`status.retried` is a boolean field which specifies whether this session was retried or not in case of failed backup. - -#### status.nextRetry - -`status.nextRetry` specifies the timestamp when this backup will be retried if it has failed. - -#### status.conditions - -`status.conditions` shows the conditions of different operations/steps of the backup process. The following conditions are set by the Stash operator on a BackupSession. +`status.sessionDeadline` indicates the deadline of the backup process. `BackupSession` will be considered `Failed` if the backup does not complete within this deadline. -| Condition Type | Usage | -| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -| `BackendRepositoryInitialized` | Indicates whether the backend repository was initialized or not. | -| `RetentionPolicyApplied` | Indicates whether the retention policies were applied or not. | -| `RepositoryIntegrityVerified` | Indicates whether the repository integrity check succeeded or not. | -| `RepositoryMetricsPushed` | Indicates whether the Repository metrics for this backup session were pushed or not. | -| `GlobalPreBackupHookSucceeded` | Indicates whether the global PreBackupHook was executed successfully or not. Only available during backup using BackupBatch. | -| `GlobalPostBackupHookSucceeded` | Indicates whether the global PostBackupHook was executed successfully or not. Only available during backup BackupBatch. | -| `DeadlineExceeded` | Indicates whether the session deadline was exceeded or not.| +#### status.snapshots +`status.snapshots` specifies the `Snapshot`s status. It contains the following fields: +- **name** indicates to the name of the `Snapshot`. +- **phase** indicate the phase of the `Snapshot`. +- **appRef** points to the application that is being backed up in this `Snapshot`. +- **repository** indicates the name of the Repository where the Snapshot is being stored. -#### status.targets +#### status.hooks +`status.hooks` represents the hook execution status. Each hook contains the following fileds: +- **preHooks** represents the pre-restore hook execution status. +- **postHooks** represents the post-restore hook execution status. -`status.targets` field contains an array of the status of the individual target for a backup run. Each target's status field consists of the following sub-fields: +Each `preHook` or `postHook` contains the following fields: +- **name** indicates the name of the hook whose status is being shown here. +- **phase** represents the hook execution phase. -- **totalHosts :** Not every pod or replica of a target is subject to backup. Thus, we refer those entities that are subject to backup as a host. `totalHosts` specifies the total number of hosts of the target that will be backed up for this BackupSession. For more details on how many hosts will be backed up for which types of workload, please visit [here](#hosts-of-a-backup-process). -- **preBackupActions :** Specifies a list of actions that the backup process should execute before taking backup. For example, the backend repository must be initialized by one of the targets before taking backup. Stash automatically assigned which target should execute this action. The `preBackupActions` should not be confused with `preBackup` hook. The hooks are meant to be configured by the users where the `preBackupActions` are meant to be configured by Stash itself. +#### status.retentionPolicy +`status.retentionPolicy` specifies whether the retention policies were properly applied on the repositories or not. Each one of them consists of the following fields: +- **ref** points to the `RetentionPolicy` CR that is being used to cleanup the old `Snapshot`s for this session. +- **repository** specifies the name of the Repository on which the RetentionPolicy has been applied. +- **phase** specifies the state of retention policy apply process. +- **error** represents the reason if the retention policy applying fail. -- **postBackupActions :** Similar to `preBackupActions`, it specifies a list of actions that a backup process should execute after taking the backup. For example, when all the targets complete their backup, one target must apply retention policy into the repository. Stash automatically selects which target should execute these `postBackupActions`. - -- **ref :** `ref` refers to the target whose backup stats has been presented by this array entry. - -- **phase :** `phase` indicates the backup phase of the target. `phase` will be `Succeeded` only if the phase of all hosts are `Succeeded`. If any of the hosts fail to complete its backup, `phase` will be `Failed`. - -- **stats :** `stats` section is an array of backup statistics about individual hosts of the target. Each host adds its statistics in this array after completing its backup process. -Each stats entry consists of the following fields: - - - **hostname:** `hostname` indicates the name of the host. - - **phase:** `phase` indicates the backup phase of this host. - - **duration:** `duration` indicates the total time taken to complete backup for this host. - - **snapshots:** Stash creates one snapshot for each targeted file paths specified in `spec.target.paths` field of `BackupConfiguration` object. The `snapshots` field holds statistics of each of these individual snapshots. Each snapshot statistics has the following fields: - - **name:** `name` indicates the name of the snapshot. - - **path:** `path` indicates the file path that was backed up in this snapshot. - - **totalSize:** `totalSize` indicates the size of data to backup from this path. - - **uploaded:** `uploaded` indicates the size of the data that was uploaded to the backend for this snapshot. This could be much smaller than `size` if some data was already uploaded in the backend in previous backup sessions. - - **processingTime:** `processingTime` indicates the time taken to process the data of the target path. - - **fileStats:** `fileStats` field show statics of files that were backed up in this snapshot. - - **totalFiles:** `totalFiles` shows the total number of files that were backed up in this snapshot. - - **newFiles:** `newFiles` shows the number of new files that were backed up in this snapshot. - - **modifiedFiles:** `modifiedFiles` shows the number of files that were modified since last backup of this directory. - - **unmodifiedFiles:** `unmodifiedFiles` shows the number of files that haven't changed since the last backup of this path. - - **error:** `error` shows the reason for failure if the backup process failed for this host. - -### Hosts of a backup process - -Stash uses two different models for backup depending on the target type. It uses **sidecar model** for Kubernetes workloads and **job model** for the rest of the targets. In the sidecar model, Stash injects a sidecar inside the targeted workload and the sidecar is responsible for taking backup. In the job model, Stash launches a job to take a backup of the target. - -Stash uses an identifier called **host** to separate the backed up data of different subjects in the backed. This host identification process depends on the backup model and the target types. The backup strategy and host identification strategy for different types of the target is explained below. - -**Kubernetes Workloads:** - -Stash uses the sidecar model to backup Kubernetes workloads. However, not every sidecar takes backup. How many sidecars will take backup depends on the type of the workload. We can divide them into the following categories: - -- **Deployment, ReplicaSet, and ReplicationController:** For these types of stateless workloads, all the replicas mount the same volumes. So, taking backup from only one replica is enough. In this case, Stash uses a leader election to elect the leader pod. Only the sidecar of the leader pod takes backup. The `alias` provided in the BackupConfiguration/BackupBatch is used as a host identifier. If the `alias` was not provided, then it defaults to `host-0`. The total number of hosts for these types of workload is 1. -- **StatefulSet:** Every replica of a StatefulSet mount different volumes. So, taking a backup from each replica is necessary. In this case, sidecar inside each replica takes backup. Stash identifies **pod-0** as **\-0**, **pod-1** as **\-1**, **pod-2** as **\-2** and so on. If the `alias` was not provided in the BackupConfiguration/BackupBatch, then the host identifiers are generated as `host-0`, `host-1`, and `host-2` etc. The total number of hosts for a StatefulSet is the number of replicas. -- **DaemonSet:** Daemon replicas on every node may contain different data. So, taking a backup of each daemon pod is necessary. In this case, sidecar inside each daemon pod takes backup. Stash considers the individual daemon pod as a separate host and the host identifiers are generated as **\-\**. The total number of hosts for a DaemonSet is the number of daemon pod running in the cluster. +#### status.retried -**Stand-alone PVC:** +`status.retried` is a boolean field which specifies whether this session was retried or not in case of failed backup. This field will exist only if the `retryConfig` has been set in the respective backup invoker. -Stash uses the job model to backup a stand-alone PVC. Stash launches a job to backup the targeted PVC. The `alias` provided in the BackupConfiguration/BackupBatch is used as the host identifier. If the `alias` was not provided, it defaults to `host-0`. The total number of hosts for a stand-alone PVC backup is 1. +#### status.nextRetry -**Databases:** +`status.nextRetry` specifies the timestamp when this backup will be retried if it has failed. This field will exist only if the `retryConfig` has been set in the respective backup invoker. -Stash uses the job model to backup a database. Stash launches a job to backup the targeted database. In this case, the number of hosts depends on the database type. +#### status.conditions -- **Stand-alone database:** For stand-alone database, the backup target is identified by the `alias` and the total number of hosts is 1. -- **Replicated cluster:** For replicated clustered databases such as MongoDB ReplicaSet, all the replicas contain the same data. In this case, taking a backup of only one replica is enough. This replica is identified by the `alias` and the total number of hosts is 1. -- **Sharded cluster:** For the sharded database cluster, Stash takes a backup of all shards. Hence, the number of hosts for a sharded database is the number of shards and they are identified as **\-0**, **\-1**, **\-2**, etc. However, the number of hosts may increase based on the database type. +`status.conditions` shows the conditions of different operations/steps of the backup process. The following conditions are set by the KubeStash operator on a `BackupSession`. -**VolumeSnapshot:** +| Condition Type | Usage | +|-------------------------------------|--------------------------------------------------------------------------------------------| +| `BackupSkipped` | indicates that the current session was skipped. | +| `SessionHistoryCleaned` | indicates whether the backup history was cleaned or not according to `backupHistoryLimit`. | +| `PreBackupHooksExecutionSucceeded` | indicates whether the pre-backup hooks were executed successfully or not. | +| `PostBackupHooksExecutionSucceeded` | indicates whether the pre-backup hooks were executed successfully or not. | +| `BackupExecutorEnsured` | indicates whether the Backup Executor is ensured or not. | +| `SnapshotsEnsured` | indicates whether Snapshots are ensured for each Repository or not | +| `DeadlineExceeded` | indicates whether the session deadline was exceeded or not. | +| `MetricsPushed` | indicates whether Metrics are pushed or not | -Stash uses the job model for taking volume snapshots. Each volume is considered as different hosts and they are identified by their name. Hence, the number of total hosts for VolumeSnapshot is the number of targeted volumes. However, since VolumeSnapshot is handled by the respective CSI driver, the host identifier does not play any role to separate their data. ## Next Steps diff --git a/docs/concepts/crds/backupstorage/index.md b/docs/concepts/crds/backupstorage/index.md new file mode 100644 index 0000000..b239c89 --- /dev/null +++ b/docs/concepts/crds/backupstorage/index.md @@ -0,0 +1,183 @@ +--- +title: BackupStorage Overview +menu: + docs_{{ .version }}: + identifier: backupstorage-overview + name: BackupStorage + parent: crds + weight: 5 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: concepts +--- + +> New to KubeStash? Please start [here](/docs/concepts/README.md). + +# BackupStorage + +## What is BackupStorage +A `BackupStorage` is a Kubernetes `CustomResourceDefinition` (CRD) which specifies the backend information where the backed up data of different applications will be stored. +A `BackupStorage` can be considered as a representation of a bucket in Kubernetes native way. This is a namespaced object. However, the `BackupStorage` can be used from any namespace +as long as it is permitted by the usage policy. + +You have to create at least one `BackupStorage` object and refer the `BackupStorage` in the `BackupConfiguration`. + +## BackupStorage CRD Specification +Like any official Kubernetes resource, a `BackupStorage` has `TypeMeta`, `ObjectMeta`, `Spec` and `Status` sections. + +A sample `BackupStorage` object that uses Google Cloud Storage(GCS) bucket as storage is shown below: +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-demo + prefix: demo + secret: gcs-secret + default: true + deletionPolicy: WipeOut + runtimeSettings: {} + usagePolicy: + allowedNamespaces: + from: All +status: + conditions: + - lastTransitionTime: "2023-12-05T13:14:04Z" + message: Successfully initialized backend. + reason: BackendInitializationSucceeded + status: "True" + type: BackendInitialized + - lastTransitionTime: "2023-12-05T13:14:04Z" + message: Backend secret exists. + reason: BackendSecretAvailable + status: "True" + type: BackendSecretFound + phase: Ready + repositories: + - name: gcs-demo-repo + namespace: demo + path: /demo/data + size: 12.222 MiB + synced: true + totalSize: 12.222 MiB +``` +Here, we are going to describe the various sections of the `BackupStorage` crd. + +## BackupStorage `Spec` +`BackupStorage` CRD has the following fields in the `.spec` section: +- **spec.storage** `spec.storage` specifies the storage information where the backed up data will be stored. To learn how to configure `BackupStorage` crd for various storages, please visit [here](/docs/guides/storages/overview). +- **storage prefix/subPath** `prefix` of any storage denotes the directory inside the storage where the backed up data will be stored. In case of **Local** storage, `subPath` is used for this purpose. +- **spec.default** `spec.default` specifies whether to use this `BackupStorage` as default storage for the current namespace as well as the allowed namespaces. One namespace can have **at most one** default BackupStorage configured. +- **spec.deletionPolicy** `spec.deletionPolicy` specifies whether KubeStash operator should delete backed up files from the storage or not when a `BackupStorage` is deleted. The valid values for this field are: + - **Delete** This will delete the respective `Repository` and `Snapshot` CRs from the cluster but keep the backed up data in the storage. This is the default behavior. + - **WipeOut** This will delete the respective `Repository` and `Snapshot` CRs as well as the backed up data from the storage. +- **spec.runtimeSettings** `spec.runtimeSettings` allow to specify Resources, NodeSelector, Affinity, Toleration, ReadinessProbe etc. for the storage initializer/cleaner jobs. To learn more visit [here](https://pkg.go.dev/kmodules.xyz/offshoot-api/api/v1#RuntimeSettings). +- **spec.usagePolicy** `spec.usagePolicy` specifies a policy of how this `BackupStorage` will be used. For example, you can use `allowedNamespaces` policy to restrict the usage of this `BackupStorage` to particular namespaces. This field is optional. If you don't provide the `usagePolicy`, then it can be used only from the current namespace. + +Here is an example of `spec.usagePolicy` that limits referencing the `BackupStorage` only from the same namespace, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: Same +``` +Here is an example of `spec.usagePolicy` that allows referencing the `BackupStorage` from only `prod` and `staging` namespaces, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: Selector + selector: + matchExpressions: + - key: "kubernetes.io/metadata.name" + operator: In + values: ["prod","staging"] +``` +Here is an example of `spec.usagePolicy` that allows referencing the `BackupStorage` from all namespaces, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: All +``` + +## BackupStorage `Status` +KubeStash operator updates `.status` of a BackupStorage. BackupStorage shows the following statistics in status section: +- **status.phase** `status.phase` indicates the overall phase of the `BackupStorage`. Phase will be **Ready** only if the Backend is initialized, Backend secret exists and Repositories are synced. +- **status.totalSize** `status.totalSize` represents the total backed up data size in this storage. This is simply the summation of sizes of all Repositories using this `BackupStorage`. +- **status.repositories** `status.repositories` holds the information of all Repositories using this `BackupStorage`. Each entry has the following fields: + - **name** `name` of the `Repository`. + - **namespace** `namespace` of the `Repository`. + - **path** `path` is relative to the path of `BackupStorage` where the data of this `Repository` has been stored. + - **size** the amount of data that has been stored in this `Repository`. + - **synced** indicates whether the repo is synced with the storage state or not. + - **error** specifies the reason in case of `Repository` sync failure. +- **status.conditions** `status.conditions` represents list of conditions regarding this `BackupStorage`. The following conditions are set by the KubeStash operator on a `BackupStorage`. + +| Condition Type | Usage | +|----------------------|---------------------------------------------| +| `BackendInitialized` | indicates that the backend is initialized. | +| `BackendSecretFound` | indicates that the backend secret is found. | + +## Deleting BackupStorage +KubeStash allows users to delete only `BackupStorage` or `BackupStorage` along with respective backed up data depending on `.spec.deletionPolicy`. Here, we are going to show how to perform these operations. + +**Delete only BackupStorage keeping backed up data:** + +First check the value in `.spec.deletionPolicy`. If the value of this field is `Delete`, then just run the following command: +```bash +$ kubectl delete backupstorage + +# Example +$ kubectl delete backupstorage gcs-storage +backupstorage "gcs-storage" deleted +``` + +If the value of the `.spec.deletionPolicy` field is `WipeOut`, then run the following commands: +```bash +$ kubectl patch backupstorage --type="merge" --patch='{"spec": {"deletionPolicy": "Delete"}}' + +# Example +$ kubectl patch backupstorage gcs-storage --type="merge" --patch='{"spec": {"deletionPolicy": "Delete"}}' +backupstorage "gcs-storage" patched + +# Then delete the BackupStorage +$ kubectl delete backupstorage gcs-storage +backupstorage "gcs-storage" deleted +``` +This will delete only `BackupStorage` and its corresponding Repositories. It won’t delete any backed up data from the storage. You can recreate the `BackupStorage` object later to reuse existing data. It will sync the corresponding Repositories. + +**Delete BackupStorage along with backed up data:** + +First check the value in `.spec.deletionPolicy`. If the value of this field is `WipeOut`, then just run the following command: +```bash +$ kubectl delete backupstorage + +# Example +$ kubectl delete backupstorage gcs-storage +backupstorage "gcs-storage" deleted +``` + +If the value of the `.spec.deletionPolicy` field is `Delete`, then run the following commands: +```bash +$ kubectl patch backupstorage --type="merge" --patch='{"spec": {"deletionPolicy": "WipeOut"}}' + +# Example +$ kubectl patch backupstorage gcs-storage --type="merge" --patch='{"spec": {"deletionPolicy": "WipeOut"}}' +backupstorage "gcs-storage" patched + +# Then delete the BackupStorage +$ kubectl delete backupstorage gcs-storage +backupstorage "gcs-storage" deleted +``` +This will delete `BackupStorage` and its corresponding Repositories along with the backed up data from the storage. You can browse your storage bucket to verify that the backed up data has been wiped out. + +## Next Steps +- Learn how to create `BackupStorage` for different storages from [here](/docs/guides/backends/overview/index.md). +- Learn how KubeStash backup workloads data from [here](/docs/guides/workloads/overview/index.md). +- Learn how KubeStash backup databases from [here](/docs/guides/addons/overview/index.md). diff --git a/docs/concepts/crds/function/index.md b/docs/concepts/crds/function/index.md index 9ab7d83..0b2e979 100644 --- a/docs/concepts/crds/function/index.md +++ b/docs/concepts/crds/function/index.md @@ -5,23 +5,23 @@ menu: identifier: function-overview name: Function parent: crds - weight: 30 + weight: 35 product_name: kubestash menu_name: docs_{{ .version }} section_menu_id: concepts --- -> New to Stash? Please start [here](/docs/concepts/README.md). +> New to KubeStash? Please start [here](/docs/concepts/README.md). # Function ## What is Function -A complete backup or restore process may consist of several steps. For example, in order to backup a PostgreSQL database we first need to dump the database and upload the dumped file to a backend. Then we need to update the respective`Repository` and `BackupSession` status and send Prometheus metrics. In Stash, we call such individual steps a `Function`. +A complete process of a task such as, backup or restore, is called a `Function` in KubeStash. -A `Function` is a Kubernetes `CustomResourceDefinition`(CRD) which basically specifies a template for a container that performs only a specific action. For example, `postgres-backup-*` function only dumps and uploads the dumped file into the backend where `update-status` function updates the status of respective `BackupSession` and `Repository` and sends Prometheus metrics to pushgateway based on the output of `postgres-backup-*` function. +A `Function` is a Kubernetes `CustomResourceDefinition`(CRD) which basically specifies a template for a container that performs only a specific action or task. For example, `postgres-backup` function only take backup of PostgreSQL Database. -When you install Stash, some `Function`s will be pre-installed for supported targets like databases, etc. However, you can create your own function to customize or extend the backup/restore process. +When you install KubeStash, some `Function`s will be pre-installed for supported targets like workloads, pvc etc. However, you can create your own function to customize or extend the backup/restore process. ## Function CRD Specification @@ -30,84 +30,35 @@ Like any official Kubernetes resource, a `Function` has `TypeMeta`, `ObjectMeta` A sample `Function` object to backup a PostgreSQL is shown below, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: addons.kubestash.com/v1alpha1 kind: Function metadata: - name: postgres-backup-11.2 + annotations: + meta.helm.sh/release-name: kubedb + meta.helm.sh/release-namespace: kubedb + creationTimestamp: "2023-12-12T12:55:59Z" + generation: 1 + labels: + app.kubernetes.io/instance: kubedb + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: kubedb-kubestash-catalog + app.kubernetes.io/version: v2023.12.11 + helm.sh/chart: kubedb-kubestash-catalog-v2023.12.11 + name: postgres-backup + resourceVersion: "277644" + uid: f64449f8-1111-4a4d-8c6e-96c5b877aef6 spec: - image: stashed/postgres-stash:11.2 args: - - backup-pg - # setup information - - --provider=${REPOSITORY_PROVIDER:=} - - --bucket=${REPOSITORY_BUCKET:=} - - --endpoint=${REPOSITORY_ENDPOINT:=} - - --region=${REPOSITORY_REGION:=} - - --path=${REPOSITORY_PREFIX:=} - - --secret-dir=/etc/repository/secret - - --scratch-dir=/tmp - - --enable-cache=${ENABLE_CACHE:=true} - - --max-connections=${MAX_CONNECTIONS:=0} # 0 indicates use default connection limit - - --hostname=${HOSTNAME:=} - - --backup-cmd=${backupCMD:=} # can specify dump command with either pg_dump or pg_dumpall - - --pg-args=${args:=} # optional arguments pass to pgdump command - - --wait-timeout=${waitTimeout:=} - # target information - - --namespace=${NAMESPACE:=default} - - --appbinding=${TARGET_NAME:=} - - --backupsession=${BACKUP_SESSION:=} - # cleanup information - - --retention-keep-last=${RETENTION_KEEP_LAST:=0} - - --retention-prune=${RETENTION_PRUNE:=false} - # output & metric information - - --output-dir=${outputDir:=} - volumeMounts: - - name: ${secretVolume} - mountPath: /etc/repository/secret - runtimeSettings: - resources: - requests: - memory: 256M - limits: - memory: 256M - securityContext: - runAsUser: 5000 - runAsGroup: 5000 -``` - -A sample `Function` that updates `BackupSession` and `Repository` status and sends metrics to Prometheus pushgateway is shown below, - -```yaml -apiVersion: stash.appscode.com/v1beta1 -kind: Function -metadata: - name: update-status -spec: - image: appscode/stash:0.10.0 - args: - - update-status - - --provider=${REPOSITORY_PROVIDER:=} - - --bucket=${REPOSITORY_BUCKET:=} - - --endpoint=${REPOSITORY_ENDPOINT:=} - - --path=${REPOSITORY_PREFIX:=} - - --secret-dir=/etc/repository/secret - - --scratch-dir=/tmp - - --enable-cache=${ENABLE_CACHE:=true} - - --max-connections=${MAX_CONNECTIONS:=0} - - --namespace=${NAMESPACE:=default} - - --backupsession=${BACKUP_SESSION:=} - - --repository=${REPOSITORY_NAME:=} - - --invoker-kind=${INVOKER_KIND:=} - - --invoker-name=${INVOKER_NAME:=} - - --target-kind=${TARGET_KIND:=} - - --target-name=${TARGET_NAME:=} - - --output-dir=${outputDir:=} - - --metrics-enabled=true - - --metrics-pushgateway-url=http://stash.kube-system.svc:56789 - - --prom-job-name=${PROMETHEUS_JOB_NAME:=} - volumeMounts: - - mountPath: /etc/repository/secret - name: ${secretVolume} + - backup + - --namespace=${namespace:=default} + - --backupsession=${backupSession:=} + - --enable-cache=${enableCache:=} + - --scratch-dir=${scratchDir:=} + - --wait-timeout=${waitTimeout:=300} + - --pg-args=${args:=} + - --backup-cmd=${backupCmd:=} + - --user=${user:=} + image: ghcr.io/kubedb/postgres-restic-plugin:v0.1.0 ``` Here, we are going to describe the various sections of a `Function` crd. @@ -126,67 +77,28 @@ A `Function` object has the following fields in the `spec` section: #### spec.args -`spec.args` specifies a list of arguments that will be passed to the entrypoint. You can templatize this section using `envsubst` style variables. Stash will resolve all the variables before creating the respective container. A variable should follow the following patterns: - -- ${VARIABLE_NAME:=default-value} -- ${VARIABLE_NAME:=} - -In the first case, if Stash can't resolve the variable, the default value will be used in place of this variable. In the second case, if Stash can't resolve the variable, an empty string will be used to replace the variable. - -##### Stash Provided Variables - -Stash operator provides the following built-in variables based on `BackupConfiguration`, `BackupSession`, `RestoreSession`, `Repository`, `Task`, `Function`, `BackupBlueprint` etc. - -| Environment Variable | Usage | -| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `NAMESPACE` | Namespace of backup or restore job/workload | -| `BACKUP_SESSION` | Name of the respective BackupSession object | -| `RESTORE_SESSION` | Name of the respective RestoreSession object | -| `REPOSITORY_NAME` | Name of the Repository object that holds respective backend information | -| `REPOSITORY_PROVIDER` | Type of storage provider. i.e. gcs, s3, aws, local etc. | -| `REPOSITORY_SECRET_NAME` | Name of the secret that holds the credentials to access the backend | -| `REPOSITORY_BUCKET` | Name of the bucket where backed up data will be stored | -| `REPOSITORY_PREFIX` | A prefix of the directory inside bucket where backed up data will be stored | -| `REPOSITORY_ENDPOINT` | URL of S3 compatible Minio/Rook server | -| `REPOSITORY_URL` | URL of the REST server for REST backend | -| `HOSTNAME` | An identifier for the backed up data. If multiple pods backup in same Repository (i.e. StatefulSet or DaemonSet) this host name is to used identify data of the individual host. | -| `SOURCE_HOSTNAME` | An identifier of the host whose backed up data will be restored | -| `TARGET_NAME` | Name of the target of backup or restore | -| `TARGET_API_VERSION` | API version of the target of backup or restore | -| `TARGET_KIND` | Kind of the target of backup or restore | -| `TARGET_NAMESPACE` | Namespace of the target object for backup or restore | -| `TARGET_MOUNT_PATH` | Directory where target PVC will be mounted in stand-alone PVC backup or restore | -| `TARGET_PATHS` | Array of file paths that are subject to backup | -| `RESTORE_PATHS` | Array of file paths that are subject to restore | -| `RESTORE_SNAPSHOTS` | Name of the snapshot that will be restored | -| `TARGET_APP_VERSION` | Version of the application pointed by an AppBinding | -| `TARGET_APP_GROUP` | The application group where the app pointed by an AppBinding belongs | -| `TARGET_APP_RESOURCE` | The resource kind under an application group that the app pointed by an AppBinding works with | -| `TARGET_APP_TYPE` | The total types of the application. It's simply `TARGET_APP_GROUP/TARGET_APP_RESOURCE` | -| `TARGET_APP_REPLICAS` | Number of replicas of an application targeted for backup or restore | -| `RETENTION_KEEP_LAST` | Number of latest snapshots to keep | -| `RETENTION_KEEP_HOURLY` | Number of hourly snapshots to keep | -| `RETENTION_KEEP_DAILY` | Number of daily snapshots to keep | -| `RETENTION_KEEP_WEEKLY` | Number of weekly snapshots to keep | -| `RETENTION_KEEP_MONTHLY` | Number of monthly snapshots to keep | -| `RETENTION_KEEP_YEARLY` | Number of yearly snapshots to keep | -| `RETENTION_KEEP_TAGS` | Keep only those snapshots that have these tags | -| `RETENTION_PRUNE` | Specify whether to remove data of old snapshot completely from the backend | -| `RETENTION_DRY_RUN` | Specify whether to run cleanup in test mode | -| `ENABLE_CACHE` | Specify whether to use cache while backup or restore | -| `MAX_CONNECTIONS` | Specifies number of parallel connections to upload/download data to/from backend | -| `NICE_ADJUSTMENT` | Adjustment value to configure `nice` to throttle the load on cpu. | -| `IONICE_CLASS` | Name of the `ionice` class | -| `IONICE_CLASS_DATA` | Value of the `ionice` class data | -| `ENABLE_STATUS_SUBRESOURCE` | Specifies whether crd has subresource enabled | -| `PROMETHEUS_PUSHGATEWAY_URL` | URL of the Prometheus pushgateway that collects the backup/restore metrics | -| `INTERIM_DATA_DIR` | Directory to store backed up or restored data temporarily before uploading to the backend or injecting into the target | - -If you want to use a variable that is not present this table, you have to provide its value in `spec.task.params` section of `BackupConfiguration` crd. - -#### spec.workDir - -`spec.workDir` specifies the container's working directory. If this field is not specified, the container's runtime default will be used. +`spec.args` specifies a list of arguments that will be passed to the entrypoint. You can templatize this section using `envsubst` style variables. KubeStash will resolve all the variables before creating the respective container. A variable should follow the following patterns: + +- ${variableName:=default-value} +- ${variableName:=} + +In the first case, if KubeStash can't resolve the variable, the default value will be used in place of this variable. In the second case, if KubeStash can't resolve the variable, an empty string will be used to replace the variable. + +##### KubeStash Provided Variables + +KubeStash operator provides the following built-in variables based on `BackupSession`, `RestoreSession`, `Function` etc. + +| Environment Variable | Usage | +|----------------------|------------------------------------------------------| +| `namespace` | Namespace of backup or restore job/workload | +| `backupSession` | Name of the respective BackupSession object | +| `restoreSession` | Name of the respective RestoreSession object | +| `enableCache` | Specify whether to use cache while backup or restore | +| `snapshot` | Name of the respective Snapshot object | + +#### spec.workingDir + +`spec.workingDir` specifies the container's working directory. If this field is not specified, the container's runtime default will be used. #### spec.ports @@ -202,10 +114,10 @@ If you want to use a variable that is not present this table, you have to provid #### spec.runtimeSettings -`spec.runtimeSettings.container` allows to configure runtime environment of a backup job at container level. You can configure the following container level parameters: +`spec.runtimeSettings` allows to configure runtime environment of a backup job at container level. You can configure the following container level parameters: | Field | Usage | -| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +|-------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `resources` | Compute resources required by sidecar container or backup job. To know how to manage resources for containers, please visit [here](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/). | | `livenessProbe` | Periodic probe of backup sidecar/job container's liveness. Container will be restarted if the probe fails. | | `readinessProbe` | Periodic probe of backup sidecar/job container's readiness. Container will be removed from service endpoints if the probe fails. | @@ -216,12 +128,6 @@ If you want to use a variable that is not present this table, you have to provid | `env` | A list of the environment variables to set in the container that will be created for this function. | | `envFrom` | This allows to set environment variables to the container that will be created for this function from a Secret or ConfigMap. | -#### spec.podSecurityPolicyName - -If you are using a [PSP enabled cluster](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) and the function needs any specific permission then you can specify the PSP name using `spec.podSecurityPolicyName` field. Stash will add this PSP in the respective RBAC roles that will be created for this function. - ->Note that Stash operator can't give permission to use a PSP to a backup job if the operator itself does not have permission to use it. So, if you want to specify PSP name in this section, make sure to add that in `stash-operator` ClusterRole too. ## Next Steps -- Learn how to use `Function` to create a `Task` from [here](/docs/concepts/crds/task/index.md). diff --git a/docs/concepts/crds/hooktemplate/index.md b/docs/concepts/crds/hooktemplate/index.md new file mode 100644 index 0000000..3781336 --- /dev/null +++ b/docs/concepts/crds/hooktemplate/index.md @@ -0,0 +1,123 @@ +--- +title: HookTemplate Overview +menu: + docs_{{ .version }}: + identifier: hooktemplate-overview + name: HookTemplate + parent: crds + weight: 50 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: concepts +--- + +> New to KubeStash? Please start [here](/docs/concepts/README.md). + +# HookTemplate + +## What is HookTemplate + +A `HookTemplate` is a Kubernetes `CustomResourceDefinition`(CRD) which basically specifies a template for some action that will be executed before or/and after backup/restore process. For example, there could be a `HookTemplate` that pause an application before backup and another `HookTemplate` that resume the application after backup. + +## HookTemplate CRD Specification + +Like any official Kubernetes resource, a `HookTemplate` has `TypeMeta`, `ObjectMeta` and `Spec` sections. However, unlike other Kubernetes resources, it does not have a `Status` section. + +A sample `HookTemplate` object is shown below, +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: HookTemplate +metadata: + name: sample-hook + namespace: demo +spec: + usagePolicy: + allowedNamespaces: + from: All + params: + - name: TEST + usage: This is a test param + required: false + action: + exec: + command: + - /bin/sh + - -c + - echo data_test > /source/data/data.txt + executor: + type: Pod + pod: + selector: name=test-app, test=hook +``` + +Here, we are going to describe the various sections of a `HookTemplate` crd. + +### HookTemplate `Spec` + +A `HookTemplate` object has the following fields in the `spec` section: + +#### spec.usagePolicy +`spec.usagePolicy` specifies a policy of how this `HookTemplate` will be used. For example, you can use `allowedNamespaces` policy to restrict the usage of this `HookTemplate` to particular namespaces. This field is optional. If you don't provide the `usagePolicy`, then it can be used only from the current namespace. + +Here is an example of `spec.usagePolicy` that limits referencing the `HookTemplate` only from the same namespace, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: Same +``` +Here is an example of `spec.usagePolicy` that allows referencing the `HookTemplate` from only `prod` and `staging` namespaces, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: Selector + selector: + matchExpressions: + - key: "kubernetes.io/metadata.name" + operator: In + values: ["prod","staging"] +``` +Here is an example of `spec.usagePolicy` that allows referencing the `HookTemplate` from all namespaces, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: All +``` + +#### spec.params +`spec.params` defines a list of parameters that is used by the `HookTemplate` to execute its logic. Each param consists of the following fields: +- **name** specifies the name of the parameter. +- **usage** specifies the usage of this parameter. +- **required** specify whether this parameter is required or not. +- **default** specifies a default value for the parameter. + +#### spec.action +`spec.action` specifies the operation that is performed by this `HookTemplate`. Valid values are: +- **exec** Execute command in a shell +- **httpGet** Do an HTTP GET request +- **httpPost** Do an HTTP POST request +- **tcpSocket** Check if a TCP socket open or not + +For more details on how hook's action work in KubeStash and how to configure different types of hook, please visit [here](/docs/guides/hooks/overview/index.md). + +#### spec.executor +`spec.executor` specifies the entity specification which is responsible for executing the hook. It consists of the following fields: +- **type** indicate the types of entity that will execute the hook. Valid values are: + - **Function** KubeStash will create a job with the provided information in `function` section. The job will execute the hook. + - **Pod** KubeStash will select the pod that matches the selector provided in `pod` section. This pod(s) will execute the hook. + - **Operator** KubeStash operator itself will execute the hook. +- **function** specifies the function information which will be used to create the hook executor job. It consists of the following fields: + - **name** indicate the name of the `Function` that contains the container definition for executing the hook logic. + - **env** specifies a list of environment variables that will be passed to the executor container. + - **volumeMounts** specifies the volumes mounts for the executor container. + - **volumes** specifies the volumes that will be mounted in the executor container. +- **pod** specifies the criteria to use to select the hook executor pods. + - **selector** specifies list of key value pair that will be used as label selector to select the desired pods. You can use comma to separate multiple labels (i.e. "app=my-app,env=prod"). + - **owner** specifies a template for owner reference that will be used to filter the selected pods. + - **strategy** specifies what should be the behavior when multiple pods are selected. Valid values are: + - **ExecuteOnOne** Execute hook on only one of the selected pods. This is default behavior. + - **ExecuteOnAll** Execute hook on all the selected pods. + +## Next Steps \ No newline at end of file diff --git a/docs/concepts/crds/repository/index.md b/docs/concepts/crds/repository/index.md index e4784df..2d18914 100644 --- a/docs/concepts/crds/repository/index.md +++ b/docs/concepts/crds/repository/index.md @@ -5,170 +5,105 @@ menu: identifier: repository-overview name: Repository parent: crds - weight: 5 + weight: 10 product_name: kubestash menu_name: docs_{{ .version }} section_menu_id: concepts --- -> New to Stash? Please start [here](/docs/concepts/README.md). +> New to KubeStash? Please start [here](/docs/concepts/README.md). # Repository ## What is Repository +A `Repository` is a Kubernetes `CustomResourceDefinition`(CRD) which represents the information about the targeted application that has been backed up and the `BackupStorage` +where the backed up data is being stored in a Kubernetes native way. -A `Repository` is a Kubernetes `CustomResourceDefinition`(CRD) which represents [backend](/docs/guides/backends/overview/index.md) information in a Kubernetes native way. +KubeStash operator creates `Repository` objects according to the repositories information provided in a `BackupConfiguration`. It specifies the application reference, the +`BackupStorage` reference and the encryption secret reference. -You have to create a `Repository` object for each backup target. Since v1beta1 api, a `Repository` object has 1-1 mapping with a target. Thus, only one target can be backed up into one `Repository`. +KubeStash operator syncs `Repository` objects in the cluster if the storage contains the information of the respective `Repository`. When a `BackupStorage` is created, KubeStash +will automatically sync the `Repositories` and `Snapshots` associated with it. -## Repository CRD Specification - -Like any official Kubernetes resource, a `Repostiory` object has `TypeMeta`, `ObjectMeta`, `Spec` and `Status` sections. +A `Repository` object has 1-1 mapping with a backup target. A backup target can be a workload, database or a PV/PVC. -A sample `Repository` object that uses Google Cloud Storage(GCS) bucket as backend is shown below: +## Repository CRD Specification +Like any official Kubernetes resource, a `Repository` object has `TypeMeta`, `ObjectMeta`, `Spec` and `Status` sections. +A sample `Repository` object that uses Google Cloud Storage(GCS) bucket as storage is shown below: ```yaml -apiVersion: stash.appscode.com/v1alpha1 +apiVersion: storage.kubestash.com/v1alpha1 kind: Repository metadata: name: gcs-demo-repo namespace: demo spec: - backend: - gcs: - bucket: stash-demo-backup - prefix: demo - storageSecretName: gcs-secret - usagePolicy: - allowedNamespaces: - from: Same - wipeOut: false + appRef: + apiGroup: apps + kind: StatefulSet + name: sample-sts + namespace: demo + deletionPolicy: Delete + encryptionSecret: + name: encry-secret + namespace: demo + path: /demo/data + storageRef: + name: gcs-storage + namespace: demo status: - firstBackupTime: "2019-04-15T06:08:16Z" + componentPaths: + - repository/v1/demo-session/pod-0 + - repository/v1/demo-session/pod-1 + - repository/v1/demo-session/pod-2 + conditions: + - lastTransitionTime: "2023-12-07T06:37:40Z" + message: Successfully initialized repository + reason: RepositoryInitializationSucceeded + status: "True" + type: RepositoryInitialized integrity: true - lastBackupTime: "2019-04-15T06:14:15Z" - totalSize: 2.567 KiB - snapshotCount: 5 - snapshotsRemovedOnLastCleanup: 1 + lastBackupTime: "2023-12-07T06:38:00Z" + phase: Ready + recentSnapshots: + - name: gcs-demo-repo-sample-backup-sts-demo-session-1701940920 + phase: Succeeded + session: demo-session + size: 12.222 MiB + snapshotTime: "2023-12-07T06:38:10Z" + size: 12.222 MiB + snapshotCount: 1 ``` - Here, we are going to describe the various sections of the `Repository` crd. -### Repository `Spec` - -`Repository` CRD has the following fields in the `.spec` section. - -- **spec.backend** -`spec.backend` specifies the storage location where the backed up snapshots will be stored. To learn how to configure `Repository` crd for various backends, please visit [here](/docs/guides/backends/overview/index.md). - -- **backend prefix/subPath** -`prefix` of any backend denotes the directory inside the backend where the backed up snapshots will be stored. In case of **Local** backend, `subPath` is used for this purpose. - -- **spec.wipeOut** -As the name implies, `spec.wipeOut` field indicates whether Stash operator should delete backed up files from the backend when `Repository` crd is deleted. The default value of this field is `false` which tells Stash to not delete backed up data when a user deletes a `Repository` crd. - -- **spec.usagePolicy** -This lets you control which namespaces are allowed to use the Repository and which are not. If you refer to a Repository from a restricted namespace, Stash will reject creating the respective BackupConfiguration/RestoreSession from validating webhook. You can use the `usagePolicy` to allow only the same namespace, a subset of namespaces, or all the namespaces to refer to the Repository. If you don't specify any `usagePolicy`, Stash will allow referencing the Repository only from the namespace where the Repository has been created. - - -Here is an example of `spec.usagePolicy` that limits referencing the Repository only from the same namespace, -```yaml -spec: - usagePolicy: - allowedNamespaces: - from: Same -``` - -Here is an example of `spec.usagePolicy` that allows referencing it from all namespaces, -```yaml -spec: - usagePolicy: - allowedNamespaces: - from: All -``` - -Here is an example of `spec.usagePolicy` that allows referencing it from only `prod` and `staging` namespace, -```yaml -spec: - usagePolicy: - allowedNamespaces: - from: Selector - selector: - matchExpressions: - - key: "kubernetes.io/metadata.name" - operator: In - values: ["prod","staging"] -``` - -### Repository `Status` - -Stash operator updates `.status` of a Repository crd every time a backup operation is completed. `Repository` crd shows the following statistics in status section: - -- **status.firstBackupTime** -`status.firstBackupTime` indicates the timestamp when the first backup was taken. - -- **status.lastBackupTime** -`status.lastBackupTime` indicates the timestamp when the latest backup was taken. - -- **status.integrity** -Stash checks the integrity of backed up files after each backup. `status.integrity` shows the result of the integrity check. - -- **status.totalSize** -`status.totalSize` shows the total size of a repository after last backup. - -- **status.snapshotCount** -`status.SnapshotCount` shows the number of snapshots stored in the Repository. - -- **status.snapshotsRemovedOnLastCleanup** -`status.snapshotsRemovedOnLastCleanup` shows the number of old snapshots that has been cleaned up according to retention policy on last backup session. - - -## Deleting Repository - -Stash allows users to delete **only `Repository` crd** or **`Repository` crd along with respective backed up data**. Here, we are going to show how to perform these operations. - -**Delete only `Repository` keeping backed up data :** - - You can delete only `Repository` crd by, - -```bash -$ kubectl delete repository - -# Example -$ kubectl delete repository gcs-demo-repo -repository "gcs-demo-repo" deleted -``` - -This will delete only `Repository` crd. It won't delete any backed up data from the backend. You can recreate the `Repository` object later to reuse existing data as long as your restic password in unchanged. - ->If you delete `Repository` crd while respective stash sidecar still exists on the workload, it will fail to take further backup. - -**Delete `Repository` along with backed up data :** - -In order to prevent the users from accidentally deleting backed up data, Stash uses a special `wipeOut` flag in `spec` section of `Repository` crd. By default, this flag is set to `wipeOut: false`. If you want to delete respective backed up data from backend while deleting `Repository` crd, you must set this flag to `wipeOut: true`. - -> Currently, Stash does not support wiping out backed up data for local backend. If you want to cleanup backed up data from local backend, you must do it manually. - -Here, is an example of deleting backed up data from GCS backend, - -- First, set `wipeOut: true` by patching `Repository` crd. - - ```bash - $ kubectl patch repository gcs-demo-repo --type="merge" --patch='{"spec": {"wipeOut": true}}' - repository "gcs-demo-repo" patched - ``` - -- Finally, delete `Repository` object. It will delete backed up data from the backend. - - ```bash - $ kubectl delete repository gcs-demo-repo - repository "gcs-demo-repo" deleted - ``` - -You can browse your backend storage bucket to verify that the backed up data has been wiped out. +## Repository `Spec` +`Repository` CRD has the following fields in the `.spec` section: +- **spec.appRef** `spec.AppRef` refers to the application that is being backed up in this `Repository`. +- **spec.storageRef** `spec.storageRef` refers to the `BackupStorage` CR which contain the storage information where the backed up data will be stored. The `BackupStorage` could be in a different namespace. However, the `Repository` namespace must be allowed to use the `BackupStorage`. +- **spec.path** `spec.path` represents the directory inside the `BackupStorage` where this Repository is storing its data. This path is relative to the path of `BackupStorage`. This path must be **unique** for each `Repository` referring same `BackupStorage`. +- **spec.deletionPolicy** `spec.deletionPolicy` specifies what to do when a `Repository` CR is deleted. The valid values for this field are: + - **Delete** This will delete the respective `Snapshot` CRs from the cluster but keep the backed up data in the storage. This is the default behavior. + - **WipeOut** This will delete the respective `Snapshot` CRs as well as the backed up data from the storage. +- **spec.encryptionSecret** refers to the `Secret` containing the encryption key which will be used to encode/decode the backed up data. You can refer to a `Secret` of a different namespace. +- **spec.paused** `spec.paused` specifies whether the Repository is paused or not. If the Repository is paused, KubeStash will not process any further event for the Repository. + +## Repository `Status` +`Repository` crd shows the following statistics in `.status` section: +- **status.phase** `status.phase` represents the current state of the `Repository`. +- **status.lastBackupTime** `status.lastBackupTime` specifies the timestamp when the last successful backup has been taken. +- **status.integrity** `status.integrity` specifies whether the backed up data of this `Repository` has been corrupted or not. +- **status.snapshotCount** `status.snapshotCount` specifies the number of current `Snapshots` stored in this `Repository`. +- **status.size** `status.size` specifies the amount of backed up data stored in the `Repository`. +- **status.recentSnapshots** `status.recentSnapshots` holds a list of recent `Snapshot` (maximum 5) information that has been taken in this `Repository`. +- **status.conditions** `status.conditions` represents list of conditions regarding this `Repository`. The following condition is set by the KubeStash operator on a `Repository`. + +| Condition Type | Usage | +|-------------------------|-------------------------------------------| +| `RepositoryInitialized` | indicates the `Repository` is initialized | + +- **status.componentPaths** `status.componentPaths` represents list of component paths in this `Repository`. ## Next Steps - -- Learn how to create `Repository` crd for different backends from [here](/docs/guides/backends/overview/index.md). -- Learn how Stash backup workloads data from [here](/docs/guides/workloads/overview/index.md). -- Learn how Stash backup databases from [here](/docs/guides/addons/overview/index.md). +- Learn how to create `BackupStorage` for different storages from [here](/docs/guides/backends/overview/index.md). +- Learn how KubeStash backup workloads data from [here](/docs/guides/workloads/overview/index.md). +- Learn how KubeStash backup databases from [here](/docs/guides/addons/overview/index.md). \ No newline at end of file diff --git a/docs/concepts/crds/restorebatch/index.md b/docs/concepts/crds/restorebatch/index.md deleted file mode 100644 index 4104450..0000000 --- a/docs/concepts/crds/restorebatch/index.md +++ /dev/null @@ -1,204 +0,0 @@ ---- -title: RestoreBatch Overview -menu: - docs_{{ .version }}: - identifier: restorebatch-overview - name: RestoreBatch - parent: crds - weight: 27 -product_name: kubestash -menu_name: docs_{{ .version }} -section_menu_id: concepts ---- - -# RestoreBatch - -## What is RestoreBatch - -A `RestoreBatch` is a Kubernetes `CustomResourceDefinition`(CRD) which allows you to restore multiple co-related components( eg, workloads, databases, etc.) together that were backed up via a `BackupBatch`. - -## RestoreBatch CRD Specification - -Like any official Kubernetes resource, a `RestoreBatch` has `TypeMeta`, `ObjectMeta`, `Spec` and `Status` sections. A sample `RestoreBatch` object to restore multiple co-related components is shown below: - -```yaml -apiVersion: stash.appscode.com/v1beta1 -kind: RestoreBatch -metadata: - name: batch-restore - namespace: demo -spec: - driver: Restic - repository: - name: minio-repo - namespace: demo - executionOrder: Parallel - members: - - target: - alias: app - ref: - apiVersion: apps/v1 - kind: Deployment - name: wordpress - rules: - - paths: - - /var/www/html - volumeMounts: - - mountPath: /var/www/html - name: wordpress-persistent-storage - - target: - alias: db - ref: - apiVersion: appcatalog.appscode.com/v1alpha1 - kind: AppBinding - name: wordpress-mysql - rules: - - snapshots: - - latest - task: - name: mysql-restore-8.0.14 - timeOut: 30m -status: - phase: Succeeded - sessionDuration: 15.145032437s - conditions: - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Repository demo/minio-repo exist. - reason: RepositoryAvailable - status: "True" - type: RepositoryFound - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Backend Secret demo/minio-secret exist. - reason: BackendSecretAvailable - status: "True" - type: BackendSecretFound - members: - - conditions: - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Restore target apps/v1 deployment/wordpress - found. - reason: TargetAvailable - status: "True" - type: RestoreTargetFound - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Successfully injected stash init-container. - reason: InitContainerInjectionSucceeded - status: "True" - type: StashInitContainerInjected - phase: Succeeded - ref: - apiVersion: apps/v1 - kind: Deployment - name: wordpress - stats: - - duration: 822.861322ms - hostname: app - phase: Succeeded - totalHosts: 1 - - conditions: - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Restore target appcatalog.appscode.com/v1alpha1 appbinding/wordpress-mysql - found. - reason: TargetAvailable - status: "True" - type: RestoreTargetFound - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Successfully created restore job. - reason: RestoreJobCreationSucceeded - status: "True" - type: RestoreJobCreated - phase: Succeeded - ref: - apiVersion: appcatalog.appscode.com/v1alpha1 - kind: AppBinding - name: wordpress-mysql - stats: - - duration: 4.597812876s - hostname: db - phase: Succeeded - totalHosts: 1 -``` - -Here, we are going to describe the various sections of `RestoreBatch` crd. - -### RestoreBatch `Spec` - -A `RestoreBatch` object has the following fields in the `spec` section. - -#### spec.driver - -`spec.driver` indicates the mechanism used to restore. Currently, Stash supports `Restic` and `VolumeSnapshotter` as drivers. The default value of this field is `Restic`. For more details, please see [here](/docs/concepts/crds/restoresession/index.md#specdriver). - -#### spec.repository - -`spec.repository.name` indicates the `Repository` crd name that holds necessary backend information from where data will be restored. - -#### spec.executionOrder - -`spec.executionOrder` specifies whether Stash should restore the targets sequentially or parallelly. If `spec.executionOrder` is set to `Parallel`, Stash will start to restore of all the targets simultaneously. If it is set to `Sequential`, Stash will not start restoring a target until all the previous members have completed their restore process. The default value of this field is `Parallel`. - -#### spec.members - -`spec.members` field specifies a list of targets to restore. Each member consists of the following fields: - -- **target :** Each member has a target specification. The target specification of a member is the same as the target specification of a `RestoreSession` explained [here](/docs/concepts/crds/restoresession/index.md#spectarget). - -- **task :** `task` specifies the name and parameters of the [Task](/docs/concepts/crds/task/index.md) crd to use to restore the member. For more details, please see [here](/docs/concepts/crds/restoresession/index.md#spectask). - -- **runtimeSettings :** `runtimeSettings` allows to configure runtime environment for the restore init-container or job. You can specify runtime settings at both pod level and container level. For more details, please see [here](/docs/concepts/crds/restoresession/index.md#specruntimesettings). - -- **tempDir :** Stash mounts an `emptyDir` for holding temporary files. It is also used for `caching` for faster restore performance. You can configure the `emptyDir` using `tempDir` section. You can also disable `caching` using this field. For more details, please see [here](/docs/concepts/crds/restoresession/index.md#spectempdir). - -- **interimVolumeTemplate :** For some targets (i.e. some databases), Stash can't directly pipe the restored data into the target. In this case, it has to store the restored data temporarily before injecting into the target. `spec.interimVolumeTemplate` specifies a PVC template for holding those data temporarily. Stash will create a PVC according to the template and use it to store the data temporarily. This PVC will be deleted automatically if you delete the `RestoreBatch`. - -- **hooks :** Each member has it's own `hooks` field which allows you to execute member-specific pre-restore or post-restore hooks. For more details about hooks, please visit [here](/docs/concepts/crds/restoresession/index.md#spechooks). - -#### spec.hooks - -`spec.hooks` allows performing some global actions before and after the restoration process. You can send HTTP requests to a remote server via `httpGet` or `httpPost`. You can check whether a TCP port is open using `tcpSocket` hooks. You can also execute some commands using `exec` hook. - -- **spec.hooks.preRestore:** `spec.hooks.preRestore` hooks are executed before restoring any of the members. -- **spec.hooks.postRestore:** `spec.hooks.postRestore` hooks are executed after restoring all the members. - -For more details on how hooks work in Stash and how to configure different types of hook, please visit [here](/docs/guides/hooks/overview/index.md). - -#### spec.timeOut - -`spec.timeOut` specifies the amount of time to wait for the restore to complete. For more details, please visit [here](/docs/concepts/crds/restoresession/index.md#spectimeout). -### RestoreBatch `Status` - -A `RestoreBatch` object has the following fields in the `status` section. - -- **phase :** `phase` shows the overall phase of the restore process. It will be `Succeeded` only if all the targets are restored successfully. - -- **sessionDuration :** `sessionDuration` shows the total time taken to complete the entire restoration process. - -- **sessionDeadline :** `sessionDeadline` indicates the the deadline of the restore process. `RestoreBatch` will be considered `Failed` if the restore does not complete within this deadline. - -- **conditions :** The `conditions` field shows conditions of different steps of the restore process. The following conditions are set by the Stash operator for a RestoreBatch: - -| Condition Type | Usage | -| -------------------------------- | ----------------------------------------------------------------------------- | -| `RepositoryFound` | Indicates whether the respective Repository object was found or not. | -| `BackendSecretFound` | Indicates whether the respective backend secret was found or not. | -| `GlobalPreRestoreHookSucceeded` | Indicates whether the global PreRestoreHook was executed successfully or not. | -| `GlobalPostRestoreHookSucceeded` | Indicates whether the global PostRestoreHook was executed successfully or no. | -| `DeadlineExceeded` | Indicates whether the session deadline was exceeded or not.| - -- **members :** `members` section shows the restore status of the individual members. Each entry has the following fields: - - **ref :** `ref` points to the respective target whose status is shown here. - - **phase :** `phase` shows the restore phase of the member. - - **conditions:** `conditions` shows the conditions of different steps of restoring this member. Stash set the following conditions for each restore members. - -| Condition Type | Usage | -| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -| `RestoreTargetFound` | Indicates whether the restore target was found or not. | -| `StashInitContainerInjected` | Indicates whether stash init-container was injected into the targeted workload. This condition is applicable only for the sidecar model. | -| `RestoreJobCreated` | Indicates whether the restore job was created or not. This condition is only applicable in the job model. | - - - **totalHosts :** `totalHosts` field specifies the total number of hosts that will be restored for this member. - - **stats :** `stats` section is an array of restore statistics of individual hosts. Individual host stats entry consists of the following fields: - - **hostname :** `hostname` indicates the name of the host. Usually it is the `alias` or `alias-`. For more details, please visit [here](/docs/concepts/crds/backupsession/index.md#hosts-of-a-backup-process). - - **phase :** `phase` indicates the restore phase of this host. - - **duration :** `duration` indicates the total time taken to complete the restore process for this host. - - **error :** `error` shows the reason for failure if the restore process fails for this host. diff --git a/docs/concepts/crds/restorebatch/restorebatch.yaml b/docs/concepts/crds/restorebatch/restorebatch.yaml deleted file mode 100644 index d715285..0000000 --- a/docs/concepts/crds/restorebatch/restorebatch.yaml +++ /dev/null @@ -1,95 +0,0 @@ -apiVersion: stash.appscode.com/v1beta1 -kind: RestoreBatch -metadata: - name: batch-restore - namespace: demo -spec: - driver: Restic - repository: - name: minio-repo - namespace: demo - executionOrder: Parallel - members: - - target: - alias: app - ref: - apiVersion: apps/v1 - kind: Deployment - name: wordpress - rules: - - paths: - - /var/www/html - volumeMounts: - - mountPath: /var/www/html - name: wordpress-persistent-storage - - target: - alias: db - ref: - apiVersion: appcatalog.appscode.com/v1alpha1 - kind: AppBinding - name: wordpress-mysql - rules: - - snapshots: - - latest - task: - name: mysql-restore-8.0.14 - timeOut: 30m -status: - phase: Succeeded - sessionDuration: 15.145032437s - conditions: - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Repository demo/minio-repo exist. - reason: RepositoryAvailable - status: "True" - type: RepositoryFound - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Backend Secret demo/minio-secret exist. - reason: BackendSecretAvailable - status: "True" - type: BackendSecretFound - members: - - conditions: - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Restore target apps/v1 deployment/wordpress - found. - reason: TargetAvailable - status: "True" - type: RestoreTargetFound - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Successfully injected stash init-container. - reason: InitContainerInjectionSucceeded - status: "True" - type: StashInitContainerInjected - phase: Succeeded - ref: - apiVersion: apps/v1 - kind: Deployment - name: wordpress - stats: - - duration: 822.861322ms - hostname: app - phase: Succeeded - totalHosts: 1 - - conditions: - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Restore target appcatalog.appscode.com/v1alpha1 appbinding/wordpress-mysql - found. - reason: TargetAvailable - status: "True" - type: RestoreTargetFound - - lastTransitionTime: "2020-07-25T17:41:52Z" - message: Successfully created restore job. - reason: RestoreJobCreationSucceeded - status: "True" - type: RestoreJobCreated - phase: Succeeded - ref: - apiVersion: appcatalog.appscode.com/v1alpha1 - kind: AppBinding - name: wordpress-mysql - stats: - - duration: 4.597812876s - hostname: db - phase: Succeeded - totalHosts: 1 diff --git a/docs/concepts/crds/restoresession/index.md b/docs/concepts/crds/restoresession/index.md index a9db2b0..741ffda 100644 --- a/docs/concepts/crds/restoresession/index.md +++ b/docs/concepts/crds/restoresession/index.md @@ -11,7 +11,7 @@ menu_name: docs_{{ .version }} section_menu_id: concepts --- -> New to Stash? Please start [here](/docs/concepts/README.md). +> New to KubeStash? Please start [here](/docs/concepts/README.md). # RestoreSession @@ -19,7 +19,7 @@ section_menu_id: concepts A `RestoreSession` is a Kubernetes `CustomResourceDefinition`(CRD) which specifies a target to restore and the source of data that will be restored in a Kubernetes native way. -You have to create a `RestoreSession` object whenever you want to restore. When a `RestoreSession` object is created, Stash injects an `init-container` into the target workload and restarts it. The `init-container` restores the desired data. If the target is a database or a stand-alone PVC, Stash launches a job to perform the restore process. +You have to create a `RestoreSession` object whenever you want to restore. ## RestoreSession CRD Specification @@ -28,122 +28,66 @@ Like any official Kubernetes resource, a `RestoreSession` has `TypeMeta`, `Objec A sample `RestoreSession` object to restore backed up data of a StatefulSet is shown below: ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: - name: statefulset-restore + name: sample-restore namespace: demo spec: - driver: Restic - repository: - name: minio-repo - namespace: demo - # task: - # name: workload-restore # task field is not required for workload data backup but it is necessary for database backup. + addon: + name: workload-addon + tasks: + - name: LogicalBackupRestore + dataSource: + components: + - pod-0 + - pod-2 + encryptionSecret: + name: encry-secret + namespace: demo + repository: gcs-demo-repo + snapshot: latest target: - alias: my-sts - ref: - apiVersion: apps/v1 - kind: StatefulSet - name: recovered-statefulset - volumeMounts: - - mountPath: /source/data - name: source-data - rules: - - targetHosts: ["my-sts-3","my-sts-4"] # "pod-3" and "pod-4" will have restored data of backed up host "pod-1" - sourceHost: "my-sts-1" # source host - paths: - - /source/data - include: - - /source/data/*.json - - targetHosts: [] # empty host match all hosts - sourceHost: "" # no source host indicates that the host is pod itself - paths: - - /source/data - exclude: - - /source/data/tmp.json - - /source/data/*.txt - hooks: - preRestore: - exec: - command: - - /bin/sh - - -c - - echo "Sample PreRestore hook demo" - containerName: stash-init - postRestore: - executionPolicy: Always - exec: - command: - - /bin/sh - - -c - - echo "Sample PostRestore hook demo" - containerName: stash-init - runtimeSettings: - container: - resources: - limits: - memory: 256M - requests: - memory: 256M - securityContext: - runAsGroup: 2000 - runAsUser: 2000 - ionice: - class: 2 - classData: 4 - nice: - adjustment: 5 - pod: - imagePullSecrets: - - name: my-private-registry-secret - serviceAccountName: my-backup-sa - tempDir: - disableCaching: false - medium: Memory - sizeLimit: 2Gi - timeOut: 30m + apiGroup: apps + kind: StatefulSet + name: restore-sts + namespace: demo status: - totalHosts: 5 - phase: Succeeded - sessionDuration: 2m40.595857548s + components: + pod-0: + duration: 10.089085545s + phase: Succeeded + pod-2: + duration: 6.509171185s + phase: Succeeded conditions: - - lastTransitionTime: "2020-07-25T17:55:56Z" - message: Repository demo/minio-repo exist. - reason: RepositoryAvailable + - lastTransitionTime: "2023-12-18T09:13:35Z" + message: Validation has been passed successfully. + reason: ResourceValidationPassed status: "True" - type: RepositoryFound - - lastTransitionTime: "2020-07-25T17:55:56Z" - message: Backend Secret demo/minio-secret exist. - reason: BackendSecretAvailable + type: ValidationPassed + - lastTransitionTime: "2023-12-18T09:13:36Z" + message: Restore Executor has been ensured successfully. + reason: SuccessfullyEnsuredRestoreExecutor status: "True" - type: BackendSecretFound - - lastTransitionTime: "2020-07-25T17:55:56Z" - message: Restore target apps/v1 statefulset/recovered-statefulset found. - reason: TargetAvailable + type: RestoreExecutorEnsured + - lastTransitionTime: "2023-12-18T09:13:59Z" + message: Metrics have been pushed successfully. + reason: SuccessfullyPushedMetrics status: "True" - type: RestoreTargetFound - - lastTransitionTime: "2020-07-25T17:55:56Z" - message: Successfully injected stash init-container. - reason: InitContainerInjectionSucceeded - status: "True" - type: StashInitContainerInjected - stats: - - duration: 884.431745ms - hostname: host-1 - phase: Succeeded - - duration: 769.924342ms - hostname: host-2 - phase: Succeeded - - duration: 868.694738ms - hostname: host-3 - phase: Succeeded - - duration: 792.097784ms - hostname: host-4 - phase: Succeeded - - duration: 833.139795ms - hostname: host-0 - phase: Succeeded + type: MetricsPushed + dependencies: + - found: true + kind: Snapshot + name: gcs-demo-repo-sample-backup-sts-demo-session-1702543201 + namespace: demo + - found: true + kind: Addon + name: workload-addon + duration: 25s + phase: Succeeded + targetFound: true + totalComponents: 2 ``` Here, we are going to describe the various sections of a `RestoreSession` object. @@ -152,188 +96,230 @@ Here, we are going to describe the various sections of a `RestoreSession` object A `RestoreSession` object has the following fields in the `spec` section. -#### spec.driver - -`spec.driver` indicates the mechanism used to restore a target. Currently, Stash supports `Restic` and `VolumeSnapshotter` as drivers. The default value of this field is `Restic`. - -| Driver | Usage | -| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `Restic` | Used to restore workload data, persistent volumes data and databases. It uses [restic](https://restic.net) to restore the target. | -| `VolumeSnapshotter` | Used to initialize PersistentVolumeClaim from VolumeSnapshot. It leverages Kubernetes [VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/) crd and CSI driver to initialize the target PVCs from respective snapshots. Currently, it can restore only in the new PVC. | - #### spec.target -`spec.target` field indicates the target where data will be restored. This section consists of the following fields: - -- **spec.target.alias :** `spec.target.alias` field specifies the alias that was used as the identifier of the backed up data. It must match with the alias used during backup. - -- **spec.target.ref :** `spec.target.ref` refers to the restore target. You have to specify `apiVersion`, `kind`, and `name` of the target. Stash will use this information to inject an `init-container` or to create a restore job. - -- **spec.target.volumeMounts :** `spec.target.volumeMounts` specifies a list of volumes and their `mountPath` where the data will be restored. Stash will mount these volumes inside the `init-container` or restore job. - -> Note: Stash stores the absolute path of the backed up files. Hence, your restored volume must be mounted on the same `mountPath` as the original volume. Otherwise, the backed up files will not be restored into your desired volume. - -- **spec.target.volumeClaimTemplates :** You can specify a list of PVC template using `spec.target.volumeClaimTemplates` field. Stash will create those PVCs then it will restore the desired data into them. Then, you can use those PVCs to deploy your desired workload. - -- **spec.target.replicas :** If you want to restore the volumes of a StatefulSet through `spec.target.volumeClaimTemplate` field, you can specify the number of replicas of the StatefulSet using `spec.target.replicas`. In this case, you have to use `${POD_ORDINAL}` variable suffix in the claim name. Stash will replace that variable with respective ordinal and it will create the volumes for each replica. For more details, please visit [here](/docs/guides/use-cases/clone-pvc/index.md#clone-the-volumes-of-a-satefulset). - -- **spec.target.rules :** `spec.target.rules` is an array of restore rules that specify how Stash should restore data for each host. For example, Stash runs the restore process in all pods of a StatefulSet. You can configure this `spec.target.rules` section to control what data will be restored into which pod. Each restore rule has the following fields: - - - **targetHosts :** `targetHosts` field contains a list of host names that are subject to this rule. If `targetHosts` field is empty, this rule applies to all hosts for which there is no specific rule. In the sample `RestoreSession` given above, the first rule applies to only `pod-3` and `pod-4` and the second rule is applicable to all hosts. - - **sourceHost :** `sourceHost` specifies the name of host whose backed up data will be restored by this rule. In the sample `RestoreSession`, the first rule specifies that backed up data of `pod-0` will be restored into `pod-3` and `pod-4`. If you keep `sourceHost` field empty as the second rule of the above example, data from a similar backup host will be restored on the respective restore host. That means, backed up data of `pod-0` will be restored into `pod-0`, backed up data of `pod-1` will be restored into `pod-1` and so on. - - **paths :** `paths` specifies a list of file paths that will be restored into the hosts who are subject to this rule. - - **snapshots :** `snapshots` specifies the list of snapshots that will be restored into the hosts who are subject to this rule. If you don't specify the snapshot field, the latest snapshot of the file paths specified in the `paths` section will be restored. - - **include :** `include` field specifies a list of patterns for the files that should be restored. Stash only restore the files that match these patterns. - - **exclude :** `exclude` field specifies a list of patterns for the files that should be ignored during. Stash will not restore the files that match these patterns. - - Restore rules comply with the following conditions: - - - There could be at most one rule with an empty `targetHosts` field. - - No two rules with non-empty `targetHosts` can't be matched for a single host. - - Stash restore only one file path in a single snapshot. So, if you specify `snapshots` field in a rule, you can't specify `paths` field as it may cause restore failure if a file path wasn't backed up in the snapshot specified in the `snapshots` field. - - If no rule matches for a host, no data will be restored on that host. - - The order of the rules does not have any effect on the restoration process. - -#### spec.repository +`spec.target` indicates the target application where the data will be restored. You have to specify `apiGroup`, `kind`, `name` and `namespace` of the target. -`spec.repository` specifies the name and namespace of the Repository CR that holds the necessary backend information where the backed up data has been stored. +#### spec.dataSource -- `spec.repository.name` specifies the name of the Repository CR. -- `spec.repository.namespace` specifies the namespace of the Repository. If you don't provide this field, Stash will look for the Repository CR in the same namespace as the RestoreSession. +`spec.dataSource` specifies the information about the data that will be restored. It consists of the following fields: -#### spec.task +- **repository** points to the `Repository` name from which the data will be restored. The `Repository` must be in the same namespace as the `RestoreSession` CR. This field can be empty if the snapshot is explicitly given. +- **snapshot** specifies the `Snapshot` name that will be restored. If you want to use **Point-In-Time** recovery option, don't specify this field. Specify `pitr` field instead. If you want to use the latest `Snapshot`, use `latest` value in this field. KubeStash will automatically find the latest `Snapshot`. +- **pitr** stands for Point-In-Time Recovery. You can provide a target time instead of specifying a particular `Snapshot`. KubeStash will automatically find the latest `Snapshot` that satisfies the targeted time and restore it. It consists of the following fields: + - **targetTime** specifies the desired date and time at which you want to roll back your application data. + - **exclusive** specifies whether to exclude the `Snapshot` that falls in the exact time specified in the `targetTime` field. By default, KubeStash will select the `Snapshot` that fall in the exact time. +- **components** specifies the list of components that will be restored. If you keep this field empty, then all the components that were backed up in the desired `Snapshot` will be restored. +- **encryptionSecret** refers to the Secret containing the encryption key which will be used to encode/decode the backed up data. You can refer to a Secret of a different namespace. You have to specify `name` and `namespace` of the secret. -`spec.task` specifies the name and parameters of the [Task](/docs/concepts/crds/task/index.md) crd to use to restore the target data. +#### spec.addon -- **spec.task.name:** `spec.task.name` indicates the name of the `Task` template to use for this restore process. -- **spec.task.params:** `spec.task.params` is an array of custom parameters to use to configure the task. - -> `spec.task` section is not necessary for restoring workload data (i.e. Deployment, DaemonSet, StatefulSet, etc.). However, it is necessary for restoring the database and stand-alone PVC. +`spec.addon` specifies addon configuration that will be used to restore the target. Addon has the following fields: + - **name** specifies the name of the addon that will be used for the restore purpose. + - **tasks** specifies a list of restore tasks and their configuration parameters. To learn about the fields under `task`, see [Task Reference](#task-reference). + - **containerRuntimeSettings** specifies runtime settings for the restore executor container. More information can be found [here](#container-level-runtime-settings). + - **jobTemplate** specifies runtime configurations for the restore Job. More information can be found [here](#podtemplate-spec). #### spec.hooks +`spec.hooks` specifies the restore hooks that should be executed before and/or after the restore. Hooks has two fields: + - **preRestore** specifies a list of hooks that will be executed before restore. To learn about the fields under `preRestore`, see [HookInfo](#hookinfo). + - **postRestore** specifies a list of hooks that will be executed after restore. To learn about the fields under `postRestore`, see [HookInfo](#hookinfo). -`spec.hooks` allows performing some actions before and after the restoration process. You can send HTTP requests to a remote server via `httpGet` or `httpPost` hooks. You can check whether a TCP socket is open using `tcpSocket` hook. You can also execute some commands into your application pod using `exec` hook. - -- **spec.hooks.preRestore:** `spec.hooks.preRestore` hooks are executed before the restore process. -- **spec.hooks.postRestore:** `spec.hooks.postRestore` hooks are executed after the restore process. Unlike the `preRestore` hook, `postRestore` hook has an extra field named `executionPolicy` which let you execute hook based on the restore status. Currently, it support the following values: - - `Always`: The hook will be executed after the restore process no matter the restore has failed or succeeded. This is the default behavior. - - `OnSuccess`: The hook will be executed after the restore process only if the restore has succeeded. - - `OnFailure`: The hook will be executed after the restore process only if the restore has failed. - -For more details on how hooks work in Stash and how to configure different types of hook, please visit [here](/docs/guides/hooks/overview/index.md). +#### spec.timeout +`spec.timeout` specifies a duration that KubeStash should wait for the session execution to be completed. If the session execution does not finish within this time period, KubeStash will consider this session as a failure. #### spec.runtimeSettings +`spec.runtimeSettings` allow to specify Resources, NodeSelector, Affinity, Toleration, ReadinessProbe etc. To know more about the fields in `runtimeSettings`, see [Runtime Settings](#runtime-settings). + +#### spec.manifestOptions +`spec.manifestOptions` provide options to select particular manifest object to restore. It consists of the following fields: +- **restoreNamespace** specifies the Namespace where the restored files will be applied. +- **mongoDB** specifies the options for selecting particular `MongoDB` components to restore in manifest restore. To know more about the fields in MongoDB manifest option, see [here](#kubedb-manifestoption). +- **postgres** specifies the options for selecting particular `Postgres` components to restore in manifest restore. To know more about the fields in Postgres manifest option, see [here](#kubedb-manifestoption). +- **mySQL** specifies the options for selecting particular `MySQL` components to restore in manifest restore. To know more about the fields in MySQL manifest option, see [here](#kubedb-manifestoption). +- **mariaDB** specifies the options for selecting particular `MariaDB` components to restore in manifest restore. To know more about the fields in MariaDB manifest option, see [here](#kubedb-manifestoption). + +#### Task Reference +Task Reference specifies a task and its configuration parameters. A `task` contains the following fields: +- **name** indicates to the name of the task. +- **variables** specifies a list of variables and their sources that will be used to resolve the task. For more details, please visit [here](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/) +- **params** specifies parameters for the task. You must provide the parameter in the Addon desired structure. +- **targetVolumes** specifies which volumes from the target should be mounted in the restore job/container. It contains the following fields: + - **volumes** indicates the list of volumes of targeted application that should be mounted on the restore job. + - **volumeMounts** specifies the mount for the volumes specified in `Volumes` section. + - **volumeClaimTemplates** specifies a template for the PersistentVolumeClaims that will be created for each Pod in a StatefulSet. +- **addonVolumes** lets you overwrite the volume sources used in the VolumeTemplate section of Addon. Make sure that name of your volume matches with the name of the volume you want to overwrite. Each `addonVolume` contains the following fields: + - **name** specifies the name of the volume. + - **source** specifies the source of this volume or a template for volume to use. + +#### HookInfo +HookInfo specifies the information about the restore hooks. It consists of the following fields: + +| Field | Usage | +|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `name` | specifies a name for the hook | +| `hookTemplate` | points to a HookTemplate CR that will be used to execute the hook. You can refer to a HookTemplate from other namespaces as long as your current namespace is allowed by the `usagePolicy` in the respective HookTemplate. | +| `params` | specifies parameters for the hook. You must provide the parameter in the HookTemplates desired structure. | +| `maxRetry` | MaxRetry specifies how many times Stash should retry the hook execution in case of failure. The default value of this field is 0 which means no retry. | +| `timeout` | Timeout specifies a duration in seconds that KubeStash should wait for the hook execution to be completed. If the hook execution does not finish within this time period, KubeStash will consider this hook execution as failure. Then, it will be re-tried according to MaxRetry policy. | +| `executionPolicy` | ExecutionPolicy specifies when to execute the hook. Valid values are:
  • **Always** KubeStash will execute this hook no matter the backup/restore failed. This is the default execution policy.
  • **OnSuccess** KubeStash will execute this hook only if the backup/restore has succeeded.
  • **OnFailure** KubeStash will execute this hook only if the backup/restore has failed.
| +| `variables` | specifies a list of variables and their sources that will be used to resolve the HookTemplate. | +| `volumes` | indicates the list of volumes of targeted application that should be mounted on the hook executor. Use this field only for `Function` type hook executor. | +| `volumeMounts` | specifies the mount for the volumes specified in `Volumes` section. Use this field only for `Function` type hook executor. | +| `runtimeSettings` | specifies runtime configurations for the hook executor Job. Use this field only for `Function` type hook executor. To know more about the fields in `runtimeSettings`, see [Runtime Settings](#runtime-settings) | + +#### Runtime Settings +Runtime Settings allows to configure runtime environment for the `Function` type hook executor job. You can specify runtime settings at both pod level and container level. + +##### Container Level Runtime Settings +`runtimeSettings.container` is used to configure the corresponding job at container level. You can configure the following container level parameters: + +| Field | Usage | +|-------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `resources` | Compute resources required by the corresponding job containers. To learn how to manage resources for containers, please visit [here](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/). | +| `livenessProbe` | Periodic probe of corresponding job container's liveness. Container will be restarted if the probe fails. | +| `readinessProbe` | Periodic probe of corresponding job container's readiness. Container will be removed from service endpoints if the probe fails. | +| `lifecycle` | Actions that the management system should take in response to container lifecycle events. | +| `securityContext` | Security options that corresponding job's container should run with. For more details, please visit [here](https://kubernetes.io/docs/concepts/policy/security-context/). | +| `nice` | Set CPU scheduling priority for current process. For more details about `nice`, please visit [here](https://www.askapache.com/optimize/optimize-nice-ionice/#nice). | +| `ionice` | Set I/O scheduling class and priority for current process. For more details about `ionice`, please visit [here](https://www.askapache.com/optimize/optimize-nice-ionice/#ionice). | +| `env` | A list of the environment variables to set in the corresponding job's container. | +| `envFrom` | This allows to set environment variables to the container that will be created for this function from a Secret or ConfigMap. | + +##### Pod Level Runtime Settings +`runtimeSettings.pod` is used to configure the corresponding job in pod level. You can configure the following pod level parameters: + +| Field | Usage | +|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `podLabels` | The labels that will be attached with the respective Pod | +| `serviceAccountName` | Name of the `ServiceAccount` to use for the corresponding job. | +| `nodeSelector` | Selector which must be true for corresponding job pod to fit on a node. | +| `automountServiceAccountToken` | Indicates whether a service account token should be automatically mounted into the restore pod. | +| `nodeName` | `nodeName` is used to request to schedule restore job's pod onto a specific node. | +| `securityContext` | Security options that restore job's pod should run with. For more details, please visit [here](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/). | +| `imagePullSecrets` | A list of secret names in the same namespace that will be used to pull image from private Docker registry. For more details, please visit [here](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). | +| `affinity` | Affinity and anti-affinity to schedule restore job's pod on a desired node. For more details, please visit [here](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity). | +| `schedulerName` | Name of the scheduler that should dispatch the restore job. | +| `tolerations` | Taints and Tolerations to ensure that restore job's pod is not scheduled in inappropriate nodes. For more details about `toleration`, please visit [here](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/). | +| `priorityClassName` | Indicates the restore job pod's priority class. For more details, please visit [here](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/). | +| `priority` | Indicates the restore job pod's priority value. | +| `readinessGates` | Specifies additional conditions to be evaluated for Pod readiness. For more details, please visit [here](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate). | +| `runtimeClassName` | RuntimeClass is used for selecting the container runtime configuration. For more details, please visit [here](https://kubernetes.io/docs/concepts/containers/runtime-class/) | +| `enableServiceLinks` | EnableServiceLinks indicates whether information about services should be injected into pod's environment variables. | + +#### PodTemplate Spec +PodTemplate Spec describes the data a pod should have when created from a template. `template` has the following fields: +- **metadata** specifies standard object's metadata. It contains `labels` and `annotations` fields. +- **controller** specifies workload controller's metadata. It contains `labels` and `annotations` fields. +- **spec** specifies the desired behavior of the pod. It contains the following fields: + +| Field | Usage | +|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `volumes` | List of volumes that can be mounted by containers belonging to the pod. More info can be found [here](https://kubernetes.io/docs/concepts/storage/volumes) | +| `initContainers` | List of initialization containers belonging to the pod. More info can be found [here](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) | +| `terminationGracePeriodSeconds` | Optional duration in seconds the pod needs to terminate gracefully. May be decreased in delete request. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds. | +| `dnsPolicy` | Set DNS policy for the pod. Defaults to "ClusterFirst". Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`. DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy. To have DNS options set along with hostNetwork, you have to specify DNS policy explicitly to `ClusterFirstWithHostNet`. | +| `nodeSelector` | NodeSelector is a selector which must be true for the pod to fit on a node. Selector which must match a node's labels for the pod to be scheduled on that node. More info [here](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) | +| `serviceAccountName` | ServiceAccountName is the name of the ServiceAccount to use to run this pod. More info [here](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) | +| `hostNetwork` | Host networking requested for this pod. Use the host's network namespace. If this option is set, the ports that will be used must be specified. Default to false. | +| `hostPID` | Use the host's pid namespace. It is optional. Default to false. | +| `hostIPC` | Use the host's ipc namespace. It is optional. Default to false. | | +| `shareProcessNamespace` | Share a single process namespace between all of the containers in a pod. When this is set containers will be able to view and signal processes from other containers in the same pod, and the first process in each container will not be assigned PID 1. HostPID and ShareProcessNamespace cannot both be set. It is optional. Default to false. | +| `securityContext` | SecurityContext holds pod-level security attributes and common container settings. It is optional. Defaults to empty. See type description for default values of each field. | +| `imagePullSecrets` | ImagePullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images used by this PodSpec. If specified, these secrets will be passed to individual puller implementations for them to use. More info [here](https://kubernetes.io/docs/concepts/containers/images#specifying-imagepullsecrets-on-a-pod) | +| `affinity` | Affinity is a group of affinity scheduling rules. If specified, the pod's scheduling constraints. | +| `schedulerName` | If specified, the pod will be dispatched by specified scheduler. If not specified, the pod will be dispatched by default scheduler. | +| `tolerations` | If specified, the pod's tolerations. | +| `priorityClassName` | If specified, indicates the pod's priority. "system-node-critical" and "system-cluster-critical" are two special keywords which indicate the highest priorities with the former being the highest priority. Any other name must be defined by creating a PriorityClass object with that name. If not specified, the pod priority will be default or zero if there is no default. | +| `priority` | The priority value. Various system components use this field to find the priority of the pod. When Priority Admission Controller is enabled, it prevents users from setting this field. The admission controller populates this field from PriorityClassName. The higher the value, the higher the priority. | +| `dnsConfig` | Specifies the DNS parameters of a pod. Parameters specified here will be merged to the generated DNS configuration based on DNSPolicy. | +| `runtimeClassName` | RuntimeClassName refers to a RuntimeClass object in the node.k8s.io group, which should be used to run this pod. If no RuntimeClass resource matches the named class, the pod will not be run. If unset or empty, the "legacy" RuntimeClass will be used, which is an implicit class with an empty definition that uses the default runtime handler. More info [here](https://git.k8s.io/enhancements/keps/sig-node/585-runtime-class) | +| `enableServiceLinks` | EnableServiceLinks indicates whether information about services should be injected into pod's environment variables, matching the syntax of Docker links. It is optional. Defaults to true. | +| `topologySpreadConstraints` | TopologySpreadConstraints describes how a group of pods ought to spread across topology domains. Scheduler will schedule pods in a way which abides by the constraints. All topologySpreadConstraints are ANDed. | +| `args` | Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info [here](https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell) | +| `env` | List of environment variables to set in the container. Cannot be updated. | +| `resources` | Compute Resources required by the sidecar container. | +| `livenessProbe` | Periodic probe of container liveness. Container will be restarted if the probe fails. Controllers may set default LivenessProbe if no liveness probe is provided. To ignore defaulting, set the value to empty LivenessProbe "{}". Cannot be updated. More info [here](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes) | +| `readinessProbe` | Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. Controllers may set default ReadinessProbe if no readyness probe is provided. To ignore defaulting, set the value to empty ReadynessProbe "{}". More info [here](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes) | +| `lifecycle` | Actions that the management system should take in response to container lifecycle events. Cannot be updated. | +| `containerSecurityContext` | Security options the pod should run with. More info [here](https://kubernetes.io/docs/concepts/policy/security-context/) and [here](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) | | +| `volumeMounts` | Pod volumes to mount into the container's filesystem. Cannot be updated. | + +#### KubeDB ManifestOption + +KubeDB ManifestOption consists of the following fields: + +| Field | Usage | +|--------------------|----------------------------------------------------------------| +| `db` | specifies whether to restore the DB manifest or not. | +| `dbName` | specifies the new name of the DB yaml after restore. | +| `authSecret` | specifies whether to restore the AuthSecret manifest or not. | +| `authSecretName` | specifies new name of the AuthSecret yaml after restore. | +| `configSecret` | specifies whether to restore the ConfigSecret manifest or not. | +| `configSecretName` | specifies new name of the ConfigSecret yaml after restore. | +| `issuerRefName` | specifies new name of the IssuerRef after restore. | -`spec.runtimeSettings` allows to configure runtime environment for restore `init-container` or job. You can specify runtime settings in both the pod level and container level. - -- **spec.runtimeSettings.container** - - `spec.runtimeSettings.container` is used to configure restore init-container/job in container level. You can configure the following container level parameters, - -| Field | Usage | -| :---------------: | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `resources` | Compute resources required by restore init-container or restore job. To know how to manage resources for containers, please visit [here](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/). | -| `livenessProbe` | Periodic probe of restore init-container/job's container liveness. The container will be restarted if the probe fails. | -| `readinessProbe` | Periodic probe of restore init-container/job's container readiness. The container will be removed from service endpoints if the probe fails. | -| `lifecycle` | Actions that the management system should take in response to container lifecycle events. | -| `securityContext` | Security options that restore init-container/job's container should run with. For more details, please visit [here](https://kubernetes.io/docs/concepts/policy/security-context/). | -| `nice` | Set CPU scheduling priority for the restore process. For more details about `nice`, please visit [here](https://www.askapache.com/optimize/optimize-nice-ionice/#nice). | -| `ionice` | Set I/O scheduling class and priority for the restore process. For more details about `ionice`, please visit [here](https://www.askapache.com/optimize/optimize-nice-ionice/#ionice). | -| `env` | A list of the environment variables to set in the restore init-container/job's container. | -| `envFrom` | This allows to set environment variables to the restore init-container/job's container from a Secret or ConfigMap. | - -- **spec.runtimeSettings.pod** - - `spec.runtimeSettings.pod` is used to configure restore job in pod level. You can configure following pod level parameters, - -| Field | Usage | -| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `serviceAccountName` | Name of the `ServiceAccount` to use for restore job. Stash init-container will use the same `ServiceAccount` as the target. | -| `nodeSelector` | Selector which must be true for restore job pod to fit on a node. | -| `automountServiceAccountToken` | Indicates whether a service account token should be automatically mounted into the restore job's pod. | -| `nodeName` | NodeName is used to request to schedule restore job's pod onto a specific node. | -| `securityContext` | Security options that restore job's pod should run with. For more details, please visit [here](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/). | -| `imagePullSecrets` | A list of secret names in the same namespace that will be used to pull images from the private docker registry. For more details, please visit [here](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). | -| `affinity` | Affinity and anti-affinity to schedule restore job's pod in the desired node. For more details, please visit [here](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity). | -| `schedulerName` | Name of the scheduler that should dispatch the restore job. | -| `tolerations` | Taints and Tolerations to ensure that restore job's pod is not scheduled in inappropriate nodes. For more details about `toleration`, please visit [here](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/). | -| `priorityClassName` | Indicates the restore job pod's priority class. For more details, please visit [here](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/). | -| `priority` | Indicates the restore job pod's priority value. | -| `readinessGates` | Specifies additional conditions to be evaluated for Pod readiness. For more details, please visit [here](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate). | -| `runtimeClassName` | RuntimeClass is used for selecting the container runtime configuration. For more details, please visit [here](https://kubernetes.io/docs/concepts/containers/runtime-class/) | -| `enableServiceLinks` | EnableServiceLinks indicates whether information about services should be injected into pod's environment variables. | - -#### spec.tempDir +### RestoreSession `Status` -Stash mounts an `emptyDir` for holding temporary files. It is also used for `caching` for faster restore performance. You can configure the `emptyDir` using `spec.tempDir` section. You can also disable `caching` using this field. The following fields are configurable in `spec.tempDir` section: +`.status` section of `RestoreSession` shows progress, stats, and overall phase of the restore process. `.status` section consists of the following fields: -- **spec.tempDir.medium :** Specifies the type of storage medium should back this file path. -- **spec.tempDir.sizeLimit :** Maximum limit of storage for this volume. -- **spec.tempDir.disableCaching :** Disable caching while restoring. This may negatively impact restore performance. This field is set to `false` by default. +#### status.phase -#### spec.timeOut +`status.phase` represents the current state of the restore process for this `RestoreSession`. `status.phase` will be `Succeeded` only if the phase of all components are `Succeeded`. If any of the components fail to complete restore, `status.phase` will be `Failed`. -`spec.timeOut` specifies the maximum amount of time to wait for the restore to complete. If the restore doesn't complete within this time limit, Stash will mark the respective RestoreSession as `Failed`. You can specify the timeout in the following format: +#### status.targetFound -- Seconds `30s` -- Minutes `10m` -- Hours `1h` -- Combination of seconds, minutes, and hours `10m30s`, `1h30m` etc. +`status.targetFound` specifies whether the restore target exist or not. -Stash does not support providing days (`d`) in the `timeOut` field. Use the equivalent hours instead. +#### status.duration -#### spec.interimVolumeTemplate +`status.duration` specifies the total time taken to complete the restore process. -For some targets (i.e. some databases), Stash can't directly pipe the restored data into the target. In this case, it has to store the restored data temporarily before injecting into the target. `spec.interimVolumeTemplate` specifies a PVC template for holding those data temporarily. Stash will create a PVC according to the template and use it to store the data temporarily. This PVC will be deleted automatically if you delete the `RestoreSession`. +#### status.deadline ->Note that the usage of this field is different from `spec.tempDir` which is used for caching purpose. Stash has introduced this field because the `emptyDir` volume that is used for `spec.tempDir` does not play nice with large databases( i.e. 100Gi database). Also, it provides debugging capability as Stash keeps it until you delete the `RestoreSession`. +`status.deadline` specifies a timestamp till this session is valid. If the session does not complete within this deadline, it will be considered as failed. -### RestoreSession `Status` +#### status.totalComponents -`.status` section of `RestoreSession` shows progress, stats, and overall phase of the restore process. The restore init-container or job adds its respective stats in `.status` section after it completes its task. `.status` section consists of the following fields: +`status.totalComponents` represents the number of total components for this `RestoreSession`. -#### status.totalHosts +#### status.components -Not every pod or replica of the target will run the restore process. Thus, we refer those entities that run the restore process as a host. `status.totalHosts` specifies the total number of hosts that will run the restore process for this RestoreSession. For more details on how many hosts will run restore process for which types of workload, please visit [here](#hosts-of-a-restore-process). - -#### status.phase +`status.components` represents the individual component restore status. Each component consists of the following fields: +- **phase** represents the restore phase of the component. +- **duration** specifies the total time taken to complete the restore process for this component. +- **error** specifies the reason in case of restore failure for the component. -`status.phase` indicates the overall phase of the restore process for this RestoreSession. `status.phase` will be `Succeeded` only if the phase of all hosts are `Succeeded`. If any of the hosts fail to complete restore, `status.phase` will be `Failed`. +#### status.hooks -#### status.sessionDuration +`status.hooks` represents the hook execution status. It consists of the following fields: +- **preHooks** represents the pre-restore hook execution status. +- **postHooks** represents the post-restore hook execution status. -`status.sessionDuration` indicates the total time taken to complete the restoration of all hosts. +Each `preHook` or `postHook` has the following fields: +- **name** indicates the name of the hook whose status is being shown here. +- **phase** represents the hook execution phase. -#### status.sessionDeadline +#### status.dependencies +`status.dependencies` specifies whether the objects required by this `RestoreSession` exist or not. This field contains the resource reference and indicates whether the resource was found or not. -`status.sessionDeadline` indicates the the deadline of the restore process. `RestoreSession` will be considered `Failed` if the restore does not complete within this deadline. +#### status.pausedBackups +`status.pausedBackups` represents the list of backups that have been paused before restore. #### status.conditions -`status.conditions` shows the conditions of various steps of the restore process. Stash sets the following conditions for a RestoreSession: - -| Condition Type | Usage | -| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | -| `RepositoryFound` | Indicates whether the respective Repository object was found or not. | -| `BackendSecretFound` | Indicates whether the respective backend secret was found or not. | -| `RestoreTargetFound` | Indicates whether the restore target was found or not. | -| `StashInitContainerInjected` | Indicates whether stash init-container was injected into the targeted workload or not. This condition is applicable only in the sidecar model. | -| `RestorerEnsured` | Indicates whether the Restorer job/init-container was created or not. | -| `ValidationPassed` | Indicates whether the resource has passed validation checks or not. | -| ` MetricsPushed` | Indicates whether the metrics have been pushed to the Pushgateway or not. | -| `DeadlineExceeded` | Indicates whether the session deadline was exceeded or not.| - -#### status.stats - -`status.stats` section is an array of restore statistics of individual hosts. Each host adds their statistics in this array after completing their restore process. This field is only available for the `Restic` driver but not available for the `VolumeSnapshotter` driver. The default value of the driver is `Restic`. - -Individual host stats entry consists of the following fields: - -- **hostname :** `hostname` indicates the name of the host. Usually, it is the `alias` or `alias-`. For more details, please visit [here](/docs/concepts/crds/backupsession/index.md#hosts-of-a-backup-process). -- **phase :** `phase` indicates the restore phase of this host. -- **duration :** `duration` indicates the total time taken to complete the restore process for this host. -- **error :** `error` shows the reason for failure if the restore process fails for this host. +`status.conditions` shows the conditions of various steps of the restore process. KubeStash sets the following conditions for a `RestoreSession`: + +| Condition Type | Usage | +|--------------------------------------|---------------------------------------------------------------------------| +| `RestoreExecutorEnsured` | Indicates whether the restore executor was ensured or not. | +| `RestoreTargetFound` | Indicates whether the restore target was found or not. | +| `PreRestoreHooksExecutionSucceeded` | Indicates whether the preRestore hooks are successfully executed or not. | +| `PostRestoreHooksExecutionSucceeded` | Indicates whether the postRestore hooks are successfully executed or not. | +| `ValidationPassed` | Indicates whether the resource has passed validation checks or not. | +| ` MetricsPushed` | Indicates whether the metrics have been pushed or not. | +| `DeadlineExceeded` | Indicates whether the session deadline was exceeded or not. | ## Next Steps diff --git a/docs/concepts/crds/retentionpolicy/index.md b/docs/concepts/crds/retentionpolicy/index.md new file mode 100644 index 0000000..873093b --- /dev/null +++ b/docs/concepts/crds/retentionpolicy/index.md @@ -0,0 +1,105 @@ +--- +title: RetentionPolicy Overview +menu: + docs_{{ .version }}: + identifier: retentionpolicy-overview + name: RetentionPolicy + parent: crds + weight: 45 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: concepts +--- + +> New to KubeStash? Please start [here](/docs/concepts/README.md). + +# RetentionPolicy + +## What is RetentionPolicy + +A `RetentionPolicy` is a Kubernetes `CustomResourceDefinition`(CRD) which specifies how the old Snapshots should be cleaned up. + +You have to create at least one `RetentionPolicy` object and refer the `RetentionPolicy` in the `BackupConfiguration`. + +## RetentionPolicy CRD Specification +Like any official Kubernetes resource, a `RetentionPolicy` has `TypeMeta`, `ObjectMeta` and `Spec` sections. However, unlike other Kubernetes resources, it does not have a `Status` section. + +A sample `RetentionPolicy` object is shown below, +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` +Here, we are going to describe the various sections of the `RetentionPolicy` crd. + +## RetentionPolicy `Spec` +A `RetentionPolicy` object has the following fields in the `spec` section: + +#### spec.maxRetentionPeriod +MaxRetentionPeriod specifies a duration up to which the old Snapshots should be kept. KubeStash will remove all the Snapshots that are older than the MaxRetentionPeriod. For example, MaxRetentionPeriod of `30d` will keep only the Snapshots of last 30 days. + +Sample duration format: +- years: `2y` +- months: `6mo` +- days: `30d` +- hours: `12h` +- minutes: `30m` + +You can also combine the above durations. For example: `30d12h30m` + +#### spec.usagePolicy +UsagePolicy specifies a policy of how this RetentionPolicy will be used. For example, you can use `allowedNamespaces` policy to restrict the usage of this RetentionPolicy to particular namespaces. This field is optional. If you don't provide the usagePolicy, then it can be used only from the current namespace. + +Here is an example of `spec.usagePolicy` that limits referencing the `RetentionPolicy` only from the same namespace, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: Same +``` +Here is an example of `spec.usagePolicy` that allows referencing the `RetentionPolicy` from only `prod` and `staging` namespaces, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: Selector + selector: + matchExpressions: + - key: "kubernetes.io/metadata.name" + operator: In + values: ["prod","staging"] +``` +Here is an example of `spec.usagePolicy` that allows referencing the `RetentionPolicy` from all namespaces, +```yaml +spec: + usagePolicy: + allowedNamespaces: + from: All +``` + +#### spec.successfulSnapshots +SuccessfulSnapshots specifies how many successful Snapshots should be kept. It consists of the following fields: +- **last** specifies how many last Snapshots should be kept. +- **hourly** specifies how many hourly Snapshots should be kept. +- **daily** specifies how many daily Snapshots should be kept. +- **weekly** specifies how many weekly Snapshots should be kept. +- **monthly** specifies how many monthly Snapshots should be kept. +- **yearly** specifies how many yearly Snapshots should be kept. + +#### spec.failedSnapshots +FailedSnapshots specifies how many failed Snapshots should be kept. It consists of only one field **last** which specifies how many last failed Snapshots should be kept. By default, KubeStash will keep only the last 1 failed Snapshot. + +#### spec.default +Default specifies whether to use this `RetentionPolicy` as a default `RetentionPolicy` for the current namespace as well as the permitted namespaces. One namespace can have at most one default `RetentionPolicy` configured. \ No newline at end of file diff --git a/docs/concepts/crds/snapshot/index.md b/docs/concepts/crds/snapshot/index.md index 4126d2e..6b7269a 100644 --- a/docs/concepts/crds/snapshot/index.md +++ b/docs/concepts/crds/snapshot/index.md @@ -5,142 +5,218 @@ menu: identifier: snapshot-overview name: Snapshot parent: crds - weight: 50 + weight: 55 product_name: kubestash menu_name: docs_{{ .version }} section_menu_id: concepts --- -> New to Stash? Please start [here](/docs/concepts/README.md). +> New to KubeStash? Please start [here](/docs/concepts/README.md). # Snapshot ## What is Snapshot -A `Snapshot` is a representation of backup snapshot in a Kubernetes native way. Stash uses an [Aggregated API Server](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/aggregated-api-servers.md) to provide `get` and `list` capabilities for snapshots from the backend. +A `Snapshot` is a Kubernetes `CustomResourceDefinition`(CRD) which represents the state of a backup run to a particular `Repository`. Multiple components of the same target may be backed up in the same `Snapshot`. This is a namespaced CRD. It should be in the same namespace as the respective `Repository`. -This enables you to view some useful information such as `creationTimestamp`, `snapshot id`, `backed up path` etc of a snapshot. This also provides the capability to restore a specific snapshot. +KubeStash operator is responsible for creating `Snapshot` CR. `Snapshot` is not supposed to be created/edited by the end user. -## Snapshot structure +## Snapshot CRD Specification -Like other official Kuberentes resources, a `Snapshot` has `TypeMeta`, `ObjectMeta` and `Status` sections. However, unlike other Kubernetes resources, it does not have a `Spec` section. +Like any official Kubernetes resource, a `Snapshot` has `TypeMeta`, `ObjectMeta`, `Spec` and `Status` sections. A sample `Snapshot` object is shown below, ```yaml -apiVersion: repositories.stash.appscode.com/v1alpha1 +apiVersion: storage.kubestash.com/v1alpha1 kind: Snapshot metadata: - creationTimestamp: "2020-07-25T17:41:31Z" labels: - hostname: app - repository: minio-repo - name: minio-repo-b54ee4a0 + kubestash.com/app-ref-kind: StatefulSet + kubestash.com/app-ref-name: sample-sts + kubestash.com/app-ref-namespace: demo + kubestash.com/repo-name: gcs-demo-repo + name: gcs-demo-repo-sample-backup-sts-demo-session-1702543201 namespace: demo - uid: b54ee4a0e9c9084696dc976f125c4fd0e6b1a31abfd82cfc857b3bc9e559fa2f +spec: + appRef: + apiGroup: apps + kind: StatefulSet + name: sample-sts + namespace: demo + backupSession: sample-backup-sts-demo-session-1702543201 + deletionPolicy: Delete + repository: gcs-demo-repo + session: demo-session + snapshotID: 01HHKQQ59XN4979HMDFNCANA9V + type: FullBackup + version: v1 status: - gid: 0 - hostname: app - paths: - - /var/lib/html - repository: minio-repo - tree: 11527d99281bf3725d58cd637d1f3c19ab9d397d6cff1887a1cd1f9c8c5ebb80 - uid: 0 - username: "" + components: + pod-0: + driver: Restic + duration: 10.595356615s + integrity: true + path: repository/v1/demo-session/pod-0 + phase: Succeeded + resticStats: + - hostPath: /source/data + id: cb64ffb35297f15cd93cfedc280b7e10267aaef283626fa564a32f69468a6bc2 + size: 10.240 MiB + uploaded: 10.242 MiB + size: 10.242 MiB + pod-1: + driver: Restic + duration: 8.789930136s + integrity: true + path: repository/v1/demo-session/pod-1 + phase: Succeeded + resticStats: + - hostPath: /source/data + id: 489d5a7614daa663df0c69a53a36d353936b6c8be61b8472694eeabcedbde16f + size: 1.978 MiB + uploaded: 1.979 MiB + size: 1.979 MiB + pod-2: + driver: Restic + duration: 6.261958475s + integrity: true + path: repository/v1/demo-session/pod-2 + phase: Succeeded + resticStats: + - hostPath: /source/data + id: 17a7bc582a1f39fa54beb96f37f6f806f86d39ca575777391fcf69ddfc1bbab7 + size: 13 B + uploaded: 1.056 KiB + size: 809 B + conditions: + - lastTransitionTime: "2023-12-14T08:40:01Z" + message: Recent snapshot list updated successfully + reason: SuccessfullyUpdatedRecentSnapshotList + status: "True" + type: RecentSnapshotListUpdated + - lastTransitionTime: "2023-12-18T08:50:57Z" + message: Metadata uploaded to backend successfully + reason: SuccessfullyUploadedSnapshotMetadata + status: "True" + type: SnapshotMetadataUploaded + integrity: true + phase: Succeeded + size: 12.222 MiB + snapshotTime: "2023-12-14T08:40:10Z" + totalComponents: 3 ``` Here, we are going to describe the various sections of a `Snapshot` object. ### Snapshot `Metadata` -- **metadata.name** +**metadata.name** - `metadata.name` specifies the name of the `Snapshot` object. It follows the following pattern, `-`. +`metadata.name` specifies the name of the `Snapshot` object. It follows the following pattern, `-`. -- **metadata.uid** +**metadata.labels** - `metadata.uid` specifies the complete id of the respective restic snapshot in the backend. +A `Snapshot` object holds `repository` and target application `kind`, `name` and `namespace` as a label in `metadata.labels` section. This helps a user to query the Snapshots of a particular repository and/or a particular target application. -- **metadata.creationTimestamp** - `metadata.creationTimestamp` represents the time when the snapshot was created. +### Snapshot `Spec` -- **metadata.labels** +**spec.snapshotID** - A `Snapshot` object holds `repository` and `hostname` as a label in `metadata.labels` section. This helps a user to query the snapshots of a particular repository and/or a particular host. +`spec.snapshotID` represents a **Universally Unique Lexicographically Sortable Identifier**(ULID) for the Snapshot. For more details about ULID, please see [here](https://github.com/oklog/ulid) -### Snapshot `Status` +**spec.type** -`Snapshot` object has the following fields in `.status` section: +`spec.type` specifies whether this snapshot represents a `FullBackup` or `IncrementalBackup`. + +**spec.repository** + +`spec.repository` specifies the name of the `Repository` where this `Snapshot` is being stored. -- **status.gid** -`status.gid` indicates the group identifier of the user who took this backup. +**spec.session** -- **status.hostname** -`status.hostname` indicates the host identifier whose data has been backed up in this snapshot. In order to know how this host identifier are generated, please visit [here](/docs/concepts/crds/backupsession/index.md#hosts-of-a-backup-process). +`spec.session` specifies the name of the session which is responsible for this `Snapshot`. -- **status.paths** -`status.paths` indicates the paths that have been backed up in this snapshot. +**spec.backupSession** -- **status.repository** -`status.repository` indicates the name of the Repository crd where this Snapshot came from. +`spec.backupSession` represents the name of the respective `BackupSession` which is responsible for this `Snapshot`. -- **status.tree** -`status.tree` indicates `tree` of the restic snapshot. For more details, please visit [here](https://restic.readthedocs.io/en/stable/100_references.html#trees-and-data). +**spec.version** -- **status.uid** -`status.uid` indicates `uid` of the user who took this backup. For `root` user it is 0. +`spec.version` denotes the respective data organization structure inside the `Repository`. -- **status.username** -`status.username` indicates the name of the user who runs the backup process that took the backup. +**spec.appRef** -- **status.tags** -`status.tags` indicates the tags of the snapshot. +`spec.appRef` specifies the reference of the application that has been backed up in this `Snapshot`. -## Working with Snapshot +**spec.deletionPolicy** -In this section, we are going to show different types of operations you can perform on the Snapshots. +`spec.deletionPolicy` specifies what to do when you delete a `Snapshot` CR. The valid values are: +- **Delete** This will delete just the Snapshot CR from the cluster but keep the backed up data in the remote backend. This is the default behavior. +- **WipeOut** This will delete the Snapshot CR as well as the backed up data from the backend. -**Listing Snapshots:** +**spec.paused** -Stash lists Snapshots directly from the backend. This operation can take more time than the default request timeout of `kubectl`. So, we are going to increase the request timeout through the `--request-timeout` flag for get/list commands. +`spec.paused` specifies whether the `Snapshot` is paused or not. If the `Snapshot` is paused, KubeStash will not process any further event for the `Snapshot`. -```bash -# List Snapshots of all Repositories in the current namespace -$ kubectl get snapshot --request-timeout=300s +### Snapshot `Status` -# List Snapshots of all Repositories of all namespaces -$ kubectl get snapshot --all-namespaces --request-timeout=300s +`Snapshot` object has the following fields in `.status` section: -# List Snapshots of all Repositories of a particular namespace -$ kubectl get snapshot -n demo --request-timeout=300s +**status.phase** -# List Snapshots of a particular Repository -$ kubectl get snapshot -l repository=local-repo --request-timeout=300s +`status.phase` represents the backup state of this `Snapshot`. -# List Snapshots from multiple Repositories -$ kubectl get snapshot -l 'repository in (local-repo,gcs-repo)' --request-timeout=300s +**status.snapshotTime** -# List Snapshots of a particular host -$ kubectl get snapshot -l hostname=db --request-timeout=300s +`status.snapshotTime` represents the timestamp when this `Snapshot` was taken. -# List Snapshots of a particular Repository and particular host -$ kubectl get snapshot -l repository=local-repo,hostname=db --request-timeout=300s -``` +**status.lastUpdateTime** -**Viewing information of a particular Snapshot:** +`status.lastUpdateTime` specifies the timestamp when this Snapshot was last updated. -```bash -$ kubectl get snapshot [-n ] -o yaml +**status.size** -# Example: -$ kubectl get snapshot -n demo local-repo-02b0ed42 -o yaml -``` +`status.size` represents the size of the `Snapshot`. + +**status.integrity** + +`status.integrity` represents whether the `Snapshot` data has been corrupted or not. + +**status.conditions** + +`status.conditions` represents list of conditions regarding this `Snapshot`. KubeStash sets the following conditions for a `Snapshot`: + +| Field | Usage | +|-----------------------------|----------------------------------------------------------------| +| `SnapshotMetadataUploaded` | Indicates whether the metadata of Snapshot is uploaded or not. | +| `RecentSnapshotListUpdated` | Indicates whether the recent Snapshot list is updated or not. | + +**status.totalComponents** + +`status.totalComponents` represents the number of total components for this `Snapshot`. -## Preconditions for Snapshot +**status.components** -1. Stash provides `Snapshots` listing facility with the help of an Aggregated API Server. Your cluster must support Aggregated API Server. Otherwise, you won't be able to perform `get` or `list` operation on `Snapshot`. +`status.components` represents the backup information of the individual components of this `Snapshot`. Each component consists of the following fields: -2. If you are using [local](/docs/guides/backends/local/index.md) backend, the respective pod that took the backup must be in `Running` state. It is not necessary if you use cloud backends. +- **path** specifies the path inside the `Repository` where the backed up data for this component has been stored. This path is relative to `Repository` path. +- **phase** represents the backup phase of the component. +- **size** represents the size of the restic repository for this component. +- **duration** specifies the total time taken to complete the backup process for this component. +- **integrity** represents the result of the restic repository integrity check for this component. +- **error** specifies the reason in case of backup failure for the component. +- **driver** specifies the name of the tool that has been used to upload the underlying backed up data. +- **resticStats** specifies the **Restic** driver specific information. Each resticStat consists of the following fields: + - **id** represents the restic snapshot id. + - **uploaded** specifies the amount of data that has been uploaded in the restic snapshot. + - **hostPath** represents the backup path for which restic snapshot is taken. + - **size** represents the restic snapshot size. +- **volumeSnapshotterStats** specifies the **VolumeSnapshotter** driver specific information. Each volumeSnapshotterStat consists of the following fields: + - **pvcName** represents the backup PVC name for which volumeSnapshot is created. + - **hostPath** represents the corresponding path of PVC for which volumeSnapshot is created. + - **volumeSnapshotName** represents the name of created volumeSnapshot. +- **walSegments** specifies a list of wall segment for individual component. Each walSegment consists of `start` time and `end` time fields. ## Next Steps diff --git a/docs/concepts/crds/task/index.md b/docs/concepts/crds/task/index.md deleted file mode 100644 index 591a9e0..0000000 --- a/docs/concepts/crds/task/index.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -title: Task Overview -menu: - docs_{{ .version }}: - identifier: task-overview - name: Task - parent: crds - weight: 35 -product_name: kubestash -menu_name: docs_{{ .version }} -section_menu_id: concepts ---- - -> New to Stash? Please start [here](/docs/concepts/README.md). - -# Task - -## What is Task - -An entire backup or restore process needs an ordered execution of one or more steps. A [Function](/docs/concepts/crds/function/index.md) represents a step of a backup or restore process. A `Task` is a Kubernetes `CustomResourceDefinition`(CRD) which specifies a sequence of functions along with their parameters in a Kubernetes native way. - -When you install Stash, some `Task`s will be pre-installed for supported targets like databases, etc. However, you can create your own `Task` to customize or extend the backup/restore process. Stash will execute these steps in the order you have specified. - -## Task CRD Specification - -Like any official Kubernetes resource, a `Task` has `TypeMeta`, `ObjectMeta` and `Spec` sections. However, unlike other Kubernetes resources, it does not have a `Status` section. - -A sample `Task` object to backup a PostgreSQL database is shown below: - -```yaml -apiVersion: stash.appscode.com/v1beta1 -kind: Task -metadata: - name: postgres-backup-11.2 -spec: - steps: - - name: postgres-backup-11.2 - params: - - name: outputDir # specifies where to write output file - value: /tmp/output - - name: secretVolume # specifies where backend secret has been mounted - value: secret-volume - - name: update-status - params: - - name: outputDir - value: /tmp/output - - name: secretVolume - value: secret-volume - volumes: - - name: secret-volume - secret: - secretName: ${REPOSITORY_SECRET_NAME} -``` - -This `Task` uses two functions to backup a PostgreSQL database. The first step uses `postgres-backup-11.2` function that dumps PostgreSQL database and uploads the dumped file. The second step uses `update-status` function which updates the status of the `BackupSession` and `Repository` crd for respective backup. - -Here, we are going to describe the various sections of a `Task` crd. - -### Task `Spec` - -A `Task` object has the following fields in the `spec` section: - -#### spec.steps - -`spec.steps` section specifies a list of functions and their parameters in the order they should be executed. You can also templatize this section using the [variables](/docs/concepts/crds/function/index.md#stash-provided-variables) that Stash can resolve itself. Stash will resolve all the variables and create a pod definition with a container specification for each `Function` specified in `steps` section. - -Each `step` consists of the following fields: - -- **name :** `name` specifies the name of the `Function` that will be executed at this step. -- **params :** `params` specifies an optional list of variables names and their values that Stash should use to resolve the respective `Function`. If you use a variable in a `Function` specification whose value Stash cannot provide, you can pass the value of that variable using this `params` section. You have to specify the following fields for a variable: - - **name :** `name` of the variable. - - **value :** value of the variable. - -In the above example `Task`, we have used `outputDir` variable in `postgres-backup-11.2` function that Stash can't resolve automatically. So, we have passed the value using the `params` section in the `Task` object. - ->Stash executes the `Functions` in the order they appear in `spec.steps` section. All the functions except the last one will be used to create `init-container` specification and the last function will be used to create `container` specification for respective backup job. This guarantees an ordered execution of the steps. - -#### spec.volumes - -`spec.volumes` specifies a list of volumes that should be mounted in the respective job created for this `Task`. In the sample we have shown above, we need to mount storage secret for the backup job. So, we have added the secret volume in `spec.volumes` section. Note that, we have used `REPOSITORY_SECRET_NAME` variable as secret name. This variable will be resolved by Stash from `Repository` specification. - -## Why Function and Task? - -You might be wondering why we have introduced `Function` and `Task` crd. We have designed `Function-Task` model for the following reasons: - -- **Customizability:** `Function` and `Task` enables you to customize backup/recovery process. For example, currently we use [mysqldump](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html) in `mysql-backup` Function to backup MySQL database. You can build a custom `Function` using Percona's [xtrabackup](https://www.percona.com/software/mysql-database/percona-xtrabackup) tool instead of `mysqldump`. Then you can write a `Task` with this custom `Function` and use it to backup your target MySQL database. - - You can also customize backup/restore process by executing hooks before or after the backup/restore process. For example, if you want to execute some logic to prepare your apps for backup or you want to send an email notification after each backup, you just need to add `Function` with your custom logic and respective `Task` to execute them. - -- **Extensibility:** Currently, Stash supports backup of MySQL, MongoDB and PostgreSQL databases. You can easily backup the databases that are not officially supported by Stash. You just need to create a `Function` and a `Task` for your desired database. - -- **Re-usability:** `Function`'s are self-sufficient and independent of Stash. So, you can reuse them in any application that uses `Function-Task` model. - -## Next Steps - -- Learn how Stash backup databases using `Function-Task` model from [here](/docs/guides/addons/overview/index.md). -- Learn how Stash backup stand-alone PVC using `Function-Task` model from [here](/docs/guides/volumes/overview/index.md). diff --git a/docs/concepts/crds/task/task.yaml b/docs/concepts/crds/task/task.yaml deleted file mode 100644 index 375a9f6..0000000 --- a/docs/concepts/crds/task/task.yaml +++ /dev/null @@ -1,22 +0,0 @@ -apiVersion: stash.appscode.com/v1beta1 -kind: Task -metadata: - name: postgres-backup-11.2 -spec: - steps: - - name: postgres-backup-11.2 - params: - - name: outputDir # specifies where to write output file - value: /tmp/output - - name: secretVolume # specifies where backend secret has been mounted - value: secret-volume - - name: update-status - params: - - name: outputDir - value: /tmp/output - - name: secretVolume - value: secret-volume - volumes: - - name: secret-volume - secret: - secretName: ${REPOSITORY_SECRET_NAME} diff --git a/docs/concepts/what-is-stash/_index.md b/docs/concepts/what-is-kubestash/_index.md similarity index 100% rename from docs/concepts/what-is-stash/_index.md rename to docs/concepts/what-is-kubestash/_index.md diff --git a/docs/concepts/what-is-stash/architecture/images/stash_architecture.svg b/docs/concepts/what-is-kubestash/architecture/images/stash_architecture.svg similarity index 100% rename from docs/concepts/what-is-stash/architecture/images/stash_architecture.svg rename to docs/concepts/what-is-kubestash/architecture/images/stash_architecture.svg diff --git a/docs/concepts/what-is-stash/architecture/index.md b/docs/concepts/what-is-kubestash/architecture/index.md similarity index 100% rename from docs/concepts/what-is-stash/architecture/index.md rename to docs/concepts/what-is-kubestash/architecture/index.md diff --git a/docs/concepts/what-is-stash/overview/index.md b/docs/concepts/what-is-kubestash/overview/index.md similarity index 89% rename from docs/concepts/what-is-stash/overview/index.md rename to docs/concepts/what-is-kubestash/overview/index.md index 6f8dd71..7bad970 100644 --- a/docs/concepts/what-is-stash/overview/index.md +++ b/docs/concepts/what-is-kubestash/overview/index.md @@ -12,9 +12,9 @@ menu_name: docs_{{ .version }} section_menu_id: concepts --- -# Stash +# KubeStash -[Stash](https://stash.run) by AppsCode is a cloud native data backup and recovery solution for Kubernetes workloads. If you are running production workloads in Kubernetes, you might want to take backup of your disks, databases etc. Traditional tools are too complex to setup and maintain in a dynamic compute environment like Kubernetes. Stash is a Kubernetes operator that uses [restic](https://github.com/restic/restic) or Kubernetes CSI Driver VolumeSnapshotter functionality to address these issues. Using Stash, you can backup Kubernetes volumes mounted in workloads, stand-alone volumes and databases. User may even extend Stash via [addons](https://stash.run/docs/{{< param "info.version" >}}/guides/addons/overview/) for any custom workload. +[KubeStash](https://kubestash.Com) by AppsCode is a cloud native data backup and recovery solution for Kubernetes workloads. If you are running production workloads in Kubernetes, you might want to take backup of your disks, databases etc. Traditional tools are too complex to setup and maintain in a dynamic compute environment like Kubernetes. KubeStash is a Kubernetes operator that uses [restic](https://github.com/restic/restic) or Kubernetes CSI Driver VolumeSnapshotter functionality to address these issues. Using KubeStash, you can backup Kubernetes volumes mounted in workloads, stand-alone volumes and databases. User may even extend KubeStash via [addons](https://stash.run/docs/{{< param "info.version" >}}/guides/addons/overview/) for any custom workload. ## Features diff --git a/docs/setup/install/stash/images/enterprise_license_form.png b/docs/setup/install/kubestash/images/enterprise_license_form.png similarity index 100% rename from docs/setup/install/stash/images/enterprise_license_form.png rename to docs/setup/install/kubestash/images/enterprise_license_form.png diff --git a/docs/setup/install/stash/index.md b/docs/setup/install/kubestash/index.md similarity index 100% rename from docs/setup/install/stash/index.md rename to docs/setup/install/kubestash/index.md diff --git a/docs/setup/uninstall/stash/index.md b/docs/setup/uninstall/kubestash/index.md similarity index 100% rename from docs/setup/uninstall/stash/index.md rename to docs/setup/uninstall/kubestash/index.md diff --git a/docs/support.md b/docs/support.md index 3b72eb0..7ccbb25 100644 --- a/docs/support.md +++ b/docs/support.md @@ -1,9 +1,9 @@ --- -title: Support | Stash +title: Support | KubeStash description: Support menu: docs_{{ .version }}: - identifier: support-stash + identifier: support-kubestash name: Support parent: welcome weight: 1020 @@ -18,4 +18,4 @@ aliases: To speak with us, please leave a message on [our website](https://appscode.com/contact/). To receive product announcements, follow us on [Twitter](https://twitter.com/KubeStash). -If you have found a bug with Stash or want to request for new features, please [file an issue](https://github.com/kubestash/project/issues/new). +If you have found a bug with KubeStash or want to request for new features, please [file an issue](https://github.com/kubestash/project/issues/new).