From 7dce12822df33921002b0268da99c3d7b6910d68 Mon Sep 17 00:00:00 2001 From: Phantom-Intruder Date: Mon, 7 Aug 2023 17:13:59 +0530 Subject: [PATCH 1/5] Keda SQS started --- Keda101/keda-lab.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/Keda101/keda-lab.md b/Keda101/keda-lab.md index 62ef60a2..0b358838 100644 --- a/Keda101/keda-lab.md +++ b/Keda101/keda-lab.md @@ -120,6 +120,22 @@ serviceAccount: The part that needs to be modified is the `annotations` section. So if you want to scale an EKS cluster based on SQS messages, then you first need an IAM role that has access to SQS, and you need to add this role arn as an annotation. +If you added the arn, then setting up authentication is a simple matter. While Keda provides resources specifically geared towards authentication, you won't need to use any of that. In the Keda authentication types, there exists a type called `operator`. This type allows the keda service account to directly acquire the role of the IAM arn you provided. As long as the arn has the permissions necessary, keda can function. The triggers will look like the following: + +```yaml + triggers: + - type: aws-sqs-queue + authenticationRef: + name: keda-trigger-auth-aws-credentials-activity-distributor + metadata: + queueURL: + queueLength: "1" + awsRegion: "us-east-1" + identityOwner: operator # This is where the identityOwner needs to be set +``` + +If you set the `identityOwner` to something else, such as `pod`, you could set up Keda to authenticate by assuming a role that has the necessary permissions instead of acquiring the IAM role itself. You could also completely scrap this part and choose to provide access keys. + ## Conclusion This wraps up the lesson on KEDA. What we tried out was a simple demonstration of a MySQL scaler, but it is a good representation of what you can expect with other data sources. If you want to try out other scalers, make sure you check out the [official samples page](https://github.com/kedacore/samples). \ No newline at end of file From 9167ec9a921941aa1b43ad244841491792c7cacc Mon Sep 17 00:00:00 2001 From: Phantom-Intruder Date: Tue, 8 Aug 2023 15:23:13 +0530 Subject: [PATCH 2/5] Keda SQS authentication --- Keda101/keda-lab.md | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/Keda101/keda-lab.md b/Keda101/keda-lab.md index 0b358838..05dd527a 100644 --- a/Keda101/keda-lab.md +++ b/Keda101/keda-lab.md @@ -134,7 +134,43 @@ If you added the arn, then setting up authentication is a simple matter. While K identityOwner: operator # This is where the identityOwner needs to be set ``` -If you set the `identityOwner` to something else, such as `pod`, you could set up Keda to authenticate by assuming a role that has the necessary permissions instead of acquiring the IAM role itself. You could also completely scrap this part and choose to provide access keys. +If you set the `identityOwner` to something else, such as `pod`, you could set up Keda to authenticate by assuming a role that has the necessary permissions instead of acquiring the IAM role itself. You could also completely scrap this part and choose to provide access keys. In this case, you would use several additional resources. For starters, you need to include your access keys in a secret. So start by defining a resource of kind `secret`: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-secret + namespace: keda +data: + AWS_ACCESS_KEY_ID: + AWS_SECRET_ACCESS_KEY: +``` + +You should then assign this resource to a Keda-specific custom resource called "TriggerAuthentication": + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-authentication + namespace: keda +spec: + secretTargetRef: + - parameter: awsAccessKeyID + name: keda-secret + key: AWS_ACCESS_KEY_ID + - parameter: awsSecretAccessKey + name: keda-secret + key: AWS_SECRET_ACCESS_KEY +``` + +This `TriggerAuthentication` resource should then be referenced within the actual `ScaledJob` resource under the `triggered` section: + +```yaml +authenticationRef: + name: keda-trigger-authentication +``` ## Conclusion From ebfac626e22e9eeec856a46f9729c8b7a55f0f1d Mon Sep 17 00:00:00 2001 From: Phantom-Intruder Date: Wed, 9 Aug 2023 16:49:03 +0530 Subject: [PATCH 3/5] Keda SQS authentication --- Keda101/keda-lab.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Keda101/keda-lab.md b/Keda101/keda-lab.md index 05dd527a..4e542d8a 100644 --- a/Keda101/keda-lab.md +++ b/Keda101/keda-lab.md @@ -172,6 +172,10 @@ authenticationRef: name: keda-trigger-authentication ``` +This will allow your `ScaledJob` resource to read the authentication keys that you added to your secret via the `TriggerAuthentication` resource. Of course, if you don't want to have your access keys even as a secret, you can use the operator authentication type described above. Additionally, Keda support [several different authentication types](https://keda.sh/docs/2.11/concepts/authentication/) out of the box. + +With the above configuration, a new Keda job will start every time a message is sent to the SQS queue. The job should have the necessary configurations to read the content of the message sent to the queue, and the message in SQS should get consumed by the job that starts. Once the job succeeds, it will terminate. If there is a failure, the job will exit and a new job will get created. It will then attempt to consume the message. + ## Conclusion This wraps up the lesson on KEDA. What we tried out was a simple demonstration of a MySQL scaler, but it is a good representation of what you can expect with other data sources. If you want to try out other scalers, make sure you check out the [official samples page](https://github.com/kedacore/samples). \ No newline at end of file From feee24534c984b5606c1fb32d08a063c3e531da8 Mon Sep 17 00:00:00 2001 From: Phantom-Intruder Date: Thu, 10 Aug 2023 18:22:19 +0530 Subject: [PATCH 4/5] Keda SQS finished --- Keda101/keda-lab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Keda101/keda-lab.md b/Keda101/keda-lab.md index 4e542d8a..4de31835 100644 --- a/Keda101/keda-lab.md +++ b/Keda101/keda-lab.md @@ -178,4 +178,4 @@ With the above configuration, a new Keda job will start every time a message is ## Conclusion -This wraps up the lesson on KEDA. What we tried out was a simple demonstration of a MySQL scaler, but it is a good representation of what you can expect with other data sources. If you want to try out other scalers, make sure you check out the [official samples page](https://github.com/kedacore/samples). \ No newline at end of file +This wraps up the lesson on KEDA. What we tried out was a simple demonstration of a MySQL scaler followed by a demonstration of using various authentication methods to connect and consume messages from AWS SQS. This is a good representation of what you can expect from other data sources. If you were considering using this with a different Kubernetes engine running on a different cloud provider, the concept would still work. Make sure you read through the authentication page, which contains different methods of authentication for different cloud providers. If you want to try out other scalers, make sure you check out the [official samples page](https://github.com/kedacore/samples). \ No newline at end of file From 081f28f9de58e32a0cc1fe8e088e38defbabf62b Mon Sep 17 00:00:00 2001 From: Phantom-Intruder Date: Tue, 21 Nov 2023 12:22:13 +0530 Subject: [PATCH 5/5] Conflic resolution --- Keda101/keda-lab.md | 36 +++++++++++++++++------------------- 1 file changed, 17 insertions(+), 19 deletions(-) diff --git a/Keda101/keda-lab.md b/Keda101/keda-lab.md index 8e41cacc..56a92b13 100644 --- a/Keda101/keda-lab.md +++ b/Keda101/keda-lab.md @@ -120,7 +120,23 @@ serviceAccount: The part that needs to be modified is the `annotations` section. So if you want to scale an EKS cluster based on SQS messages, then you first need an IAM role that has access to SQS, and you need to add this role arn as an annotation. -<<<<<<< HEAD +``` +annotations: + eks.amazonaws.com/role-arn: arn:aws:iam:::role/ +``` + +Next, you need to change the ScaleObject resource. The mysql-hpa.yaml has the trigger specified as the mysql db. However, it does not have an option called `identityOwner`. This is becase we are not using authentication here, and therefore do not need such a thing. In order to add authentication, this key should be added and the value set to `operator`: + +``` +metadata: + ... + identityOwner: operator +``` + +And that's it! You only needed to modify two lines and you have full authorization among the cluster. + +While this is the easiest way to provide authentication, it is not the only way to do it. You could also change the `identityOwner` to `pod`, and create a `TriggerAuthentication` resource and feed in the AWS access keys (which isn't very secure), or have the keda service account assume a role that has access to the necessary resources (which is much more secure). There is a number of different ways to authorize, and these are covered in the [KEDA documentation](https://keda.sh/docs/1.4/concepts/authentication/). + If you added the arn, then setting up authentication is a simple matter. While Keda provides resources specifically geared towards authentication, you won't need to use any of that. In the Keda authentication types, there exists a type called `operator`. This type allows the keda service account to directly acquire the role of the IAM arn you provided. As long as the arn has the permissions necessary, keda can function. The triggers will look like the following: ```yaml @@ -176,24 +192,6 @@ authenticationRef: This will allow your `ScaledJob` resource to read the authentication keys that you added to your secret via the `TriggerAuthentication` resource. Of course, if you don't want to have your access keys even as a secret, you can use the operator authentication type described above. Additionally, Keda support [several different authentication types](https://keda.sh/docs/2.11/concepts/authentication/) out of the box. With the above configuration, a new Keda job will start every time a message is sent to the SQS queue. The job should have the necessary configurations to read the content of the message sent to the queue, and the message in SQS should get consumed by the job that starts. Once the job succeeds, it will terminate. If there is a failure, the job will exit and a new job will get created. It will then attempt to consume the message. -======= -``` -annotations: - eks.amazonaws.com/role-arn: arn:aws:iam:::role/ -``` - -Next, you need to change the ScaleObject resource. The mysql-hpa.yaml has the trigger specified as the mysql db. However, it does not have an option called `identityOwner`. This is becase we are not using authentication here, and therefore do not need such a thing. In order to add authentication, this key should be added and the value set to `operator`: - -``` -metadata: - ... - identityOwner: operator -``` - -And that's it! You only needed to modify two lines and you have full authorization among the cluster. - -While this is the easiest way to provide authentication, it is not the only way to do it. You could also change the `identityOwner` to `pod`, and create a `TriggerAuthentication` resource and feed in the AWS access keys (which isn't very secure), or have the keda service account assume a role that has access to the necessary resources (which is much more secure). There is a number of different ways to authorize, and these are covered in the [KEDA documentation](https://keda.sh/docs/1.4/concepts/authentication/). ->>>>>>> b68da965d466f37a94ab063b6a26328c2aa1c470 ## Conclusion