Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ansible config to make backend-hel as the primary test server #117

Merged
merged 32 commits into from
Dec 6, 2024

Conversation

DecFox
Copy link
Contributor

@DecFox DecFox commented Dec 4, 2024

This PR adds configuration files to setup backend-hel as the main test server for our backend services.

Closes #110

@DecFox DecFox requested a review from hellais December 4, 2024 22:54
Copy link

github-actions bot commented Dec 4, 2024

Ansible Run Output 🤖

Ansible Playbook Recap 🔍



Ansible playbook output 📖success

Show Execution

$ ansible-playbook playbook.yml --check --diff -i ../tf/modules/ansible_inventory/inventories/inventory-dev.ini
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'
ERROR! the role 'adm' was not found in ansible.legacy:/home/runner/work/devops/devops/ansible/roles:/home/runner/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/runner/work/devops/devops/ansible/roles:/home/runner/work/devops/devops/ansible

The error appears to be in '/home/runner/work/devops/devops/ansible/roles/base-backend/meta/main.yml': line 3, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

dependencies:
  - role: adm
    ^ here
Pusher @DecFox
Action pull_request
Working Directory
Workflow .github/workflows/check_ansible.yml
Last updated Fri, 06 Dec 2024 20:56:07 GMT

* Split playbooks into tiers
* Add support for clickhouse cluster monitoring
* oonidata: (23 commits)
  rollback node job name
  Align job_name
  Re-organization of monitoring
  Don't setup node_exporter as part of bootstrap role
  Add more schema updates
  Add scripts related to migrating backend-fsn tables over to cluster
  Add ssm permission to service task
  Use readonly user password
  Bump up size of ecs_cluster
  Fix typo
  Fix labeling of reverse proxy
  Add extra path rules for oonidata endpoint
  Add clickhouse proxy nftable rules
  Add clickhouseproxy setup
  Point oonifindings to oonidata branch
  Drop backendproxy
  Fix duplicate service name
  Change default task memory
  Add support for deploying oonibackend proxy nginx proxy
  Change ECS cluster instance sizes
  ...
Copy link

github-actions bot commented Dec 4, 2024

Terraform Run Output 🤖

Format and Style 🖌failure

Initialization ⚙️success

Validation 🤖success

Validation Output

$ terraform validate
Success! The configuration is valid.

Plan 📖success

  • Plan: 0 to add, 4 to change, 0 to destroy.
Show Plan

$ terraform plan
Acquiring state lock. This may take a few moments...
random_password.prometheus_metrics_password: Refreshing state... [id=none]
random_id.artifact_id: Refreshing state... [id=8Ujqew]
module.ansible_inventory.local_file.ansible_inventory: Refreshing state... [id=b6de844ed8d384f890fa6f467502390de843f758]
random_password.jwt_secret: Refreshing state... [id=none]
module.adm_iam_roles.tls_private_key.oonidevops: Refreshing state... [id=b49a9fdb9f720320340226016efe24808dd68203]
module.ansible_inventory.null_resource.ansible_update_known_hosts: Refreshing state... [id=236461505953331670]
data.aws_ssm_parameter.clickhouse_readonly_url: Reading...
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
data.aws_ssm_parameter.do_token: Reading...
module.ooniapi_oonirun.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonirun]
module.ooniapi_reverseproxy.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-reverseproxy]
module.ooni_backendproxy.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniprobe]
module.adm_iam_roles.aws_secretsmanager_secret.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Reading...
module.ooniapi_ooniauth_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniauth]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
aws_s3_bucket.ooniapi_codepipeline_bucket: Refreshing state... [id=codepipeline-ooniapi-eu-central-1-f148ea7b]
aws_s3_bucket.oonith_codepipeline_bucket: Refreshing state... [id=codepipeline-oonith-eu-central-1-f148ea7b]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooni_backendproxy.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 0s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.ooniapi_oonirun.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role]
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-oonifindings-td:27/ooniapi-service-oonifindings]
module.adm_iam_roles.aws_key_pair.oonidevops: Refreshing state... [id=oonidevops]
data.aws_ssm_parameter.do_token: Read complete after 1s [id=/oonidevops/secrets/digitalocean_access_token]
aws_secretsmanager_secret.prometheus_metrics_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/prometheus_metrics_password-M8BbRw]
module.ooniapi_oonifindings_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonifindings]
data.aws_ssm_parameter.clickhouse_readonly_url: Read complete after 1s [id=/oonidevops/secrets/clickhouse_readonly_url]
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_cluster.aws_iam_role.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role]
module.ooniapi_reverseproxy_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-reverseproxy]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 1s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-ooniprobe-td:44/ooniapi-service-ooniprobe]
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_reverseproxy.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-oonirun-td:61/ooniapi-service-oonirun]
module.ooniapi_user.aws_secretsmanager_secret.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx]
module.ooniapi_user.aws_ses_email_identity.ooniapi: Refreshing state... [[email protected]]
module.ooniapi_oonifindings.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonifindings]
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-ooniauth-td:66/ooniapi-service-ooniauth]
data.aws_availability_zones.available: Reading...
module.ooniapi_ooniauth.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniauth]
module.ooniapi_oonifindings.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role]
module.ooniapi_reverseproxy.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-reverseproxy-td:4/ooniapi-service-reverseproxy]
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Reading...
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Read complete after 0s [id=/aws/service/ecs/optimized-ami/amazon-linux-2/recommended]
module.adm_iam_roles.aws_iam_policy.oonidevops: Refreshing state... [id=arn:aws:iam::905418398257:policy/OONIDevopsPolicy]
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Reading...
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Reading...
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Reading...
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Read complete after 0s [id=4022892340]
module.oonidevops_github_user.aws_secretsmanager_secret.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd]
module.ooniapi_reverseproxy.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-reverseproxy-task-role]
module.ooniapi_user.aws_secretsmanager_secret.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr]
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_ooniauth.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role]
module.ooniapi_user.aws_iam_user.ooniapi: Refreshing state... [id=oonidevops-ooniapi]
aws_secretsmanager_secret.jwt_secret: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/jwt_secret-NUESvS]
module.ooniapi_ooniprobe.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role]
data.aws_availability_zones.available: Read complete after 0s [id=eu-central-1]
aws_acm_certificate.ooniapi_frontend: Refreshing state... [id=arn:aws:acm:eu-central-1:905418398257:certificate/190205f1-392d-425c-a059-7006ca8c8c46]
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_cluster.aws_cloudwatch_log_group.ooniapi_services: Refreshing state... [id=ooni-ecs-group/ooniapi-ecs-cluster]
module.oonidevops_github_user.aws_iam_user.oonidevops_github: Refreshing state... [id=oonidevops-github]
aws_secretsmanager_secret.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ]
module.ooniapi_ooniprobe.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniprobe]
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Reading...
module.oonidevops_github_user.aws_iam_policy.oonidevops_github: Refreshing state... [id=arn:aws:iam::905418398257:policy/oonidevops-github-policy]
module.ooniapi_reverseproxy_deployer.data.aws_caller_identity.current: Reading...
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_oonirun_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonirun]
module.ooniapi_ooniauth_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniauth]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniprobe]
module.ooniapi_reverseproxy_deployer.data.aws_caller_identity.current: Read complete after 1s [id=905418398257]
module.adm_iam_roles.aws_secretsmanager_secret_version.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|terraform-20240925140131946100000002]
module.ooniapi_oonifindings_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonifindings]
module.ooniapi_oonirun.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role:ooniapi-service-oonirun-task-role]
data.aws_secretsmanager_secret_version.prometheus_metrics_password: Reading...
aws_secretsmanager_secret_version.prometheus_metrics_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/prometheus_metrics_password-M8BbRw|terraform-20240314200140936700000008]
module.ooniapi_reverseproxy_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-reverseproxy]
module.ooniapi_cluster.aws_iam_role_policy.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role:ooniapi-ecs-cluster-instance-role-policy]
module.ooniapi_cluster.aws_iam_instance_profile.container_host: Refreshing state... [id=ooniapi-ecs-cluster]
module.ooniapi_oonifindings.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role:ooniapi-service-oonifindings-task-role]
module.adm_iam_roles.aws_iam_role.oonidevops: Refreshing state... [id=oonidevops]
data.aws_secretsmanager_secret_version.prometheus_metrics_password: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/prometheus_metrics_password-M8BbRw|AWSCURRENT]
module.ooniapi_user.aws_iam_access_key.ooniapi: Refreshing state... [id=AKIA5FTZELIYSK2XEVOT]
module.ooniapi_user.aws_iam_user_policy.ooniapi: Refreshing state... [id=oonidevops-ooniapi:oonidevops-ooniapi-policy]
module.ooniapi_reverseproxy.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-reverseproxy-task-role:ooniapi-service-reverseproxy-task-role]
module.oonidevops_github_user.aws_iam_access_key.oonidevops_github: Refreshing state... [id=AKIA5FTZELIYXDN55SMS]
module.ooniapi_ooniauth.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role:ooniapi-service-ooniauth-task-role]
module.ooniapi_ooniprobe.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role:ooniapi-service-ooniprobe-task-role]
module.ooniapi_cluster.aws_ecs_cluster.main: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:cluster/ooniapi-ecs-cluster]
module.oonidevops_github_user.aws_iam_user_policy_attachment.oonidevops_github: Refreshing state... [id=oonidevops-github-20240313195612421500000001]
aws_secretsmanager_secret_version.jwt_secret: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/jwt_secret-NUESvS|terraform-20240310182536838400000005]
module.ooniapi_oonirun_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonirun]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr|terraform-20240314200140914600000006]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx|terraform-20240314200140918400000007]
module.oonidevops_github_user.aws_secretsmanager_secret_version.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd|terraform-20240519071250187000000004]
module.ooniapi_reverseproxy.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-reverseproxy-td]
aws_codestarconnections_connection.oonidevops: Refreshing state... [id=arn:aws:codestar-connections:eu-central-1:905418398257:connection/6bd492f6-c11d-43ec-92b0-24c47700d528]
data.aws_secretsmanager_secret_version.deploy_key: Reading...
aws_route53_record.ooniapi_frontend_cert_validation["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__ef17825e5fd9713f596344bdd9626f5e.8.th.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__05c891caeb4509d4cd7f9c24d8b6dbd0.oonirun.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__a064be8aa084a037ff9fa5e3e541c87d.ooniprobe.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__48cd4e71cee9930614228176b7deefb9.ooniauth.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["api.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__cd4729fc0c282e771d056e719a7bdf4f.api.dev.ooni.io._CNAME]
module.terraform_state_backend.data.aws_region.current: Reading...
module.terraform_state_backend.data.aws_region.current: Read complete after 0s [id=eu-central-1]
module.oonipg.random_password.pg_password: Refreshing state... [id=none]
module.network.aws_vpc.main: Refreshing state... [id=vpc-0e382f3ad89286de9]
data.aws_secretsmanager_secret_version.deploy_key: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|AWSCURRENT]
module.oonipg.aws_secretsmanager_secret.pg_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/pg_password-OjzOJC]
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Reading...
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Read complete after 0s [id=1719060339]
module.ooni_th_droplet.digitalocean_droplet.ooni_th_docker[0]: Refreshing state... [id=459912318]
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Reading...
aws_acm_certificate_validation.ooniapi_frontend: Refreshing state... [id=0001-01-01 00:00:00 +0000 UTC]
module.terraform_state_backend.aws_dynamodb_table.with_server_side_encryption[0]: Refreshing state... [id=oonidevops-dev-terraform-state-lock]
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Read complete after 0s [id=2666303363]
module.terraform_state_backend.aws_s3_bucket.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Reading...
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Read complete after 0s [id=2666303363]
module.oonipg.aws_secretsmanager_secret_version.pg_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/pg_password-OjzOJC|terraform-20240310155428358300000002]
module.ooniapi_reverseproxy_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-reverseproxy-eu-central-1]
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniprobe-eu-central-1]
module.ooniapi_oonirun_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonirun-eu-central-1]
module.ooniapi_oonifindings_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonifindings-eu-central-1]
module.ooniapi_ooniauth_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniauth-eu-central-1]
module.ooniapi_oonirun_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonirun]
module.ooniapi_reverseproxy_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-reverseproxy]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniprobe]
module.ooniapi_ooniauth_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniauth]
module.ooniapi_oonifindings_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonifindings]
module.ooniapi_reverseproxy_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-reverseproxy]
module.ooni_th_droplet.aws_route53_record.ooni_th["0"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_0.do.th.dev.ooni.io_A]
module.ooni_backendproxy.data.cloudinit_config.ooni_backendproxy: Reading...
module.ooniapi_oonirun_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonirun]
module.ooni_backendproxy.data.cloudinit_config.ooni_backendproxy: Read complete after 0s [id=3085866265]
module.ooniapi_ooniauth_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniauth]
module.ooniapi_ooniprobe_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniprobe]
module.ooniapi_oonifindings_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonifindings]
module.ooniapi_ooniauth.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OautM-20240917211808752700000001/91bcad756924a3a7]
module.network.aws_internet_gateway.gw: Refreshing state... [id=igw-0c080e9b235ed29d1]
module.oonipg.aws_security_group.pg: Refreshing state... [id=sg-005ca579eb9c08cda]
module.ooniapi_reverseproxy.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OrevM-20241126221437689400000001/e05c012d99ff36ad]
module.ooni_backendproxy.aws_alb_target_group.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oobpx20241125233815786000000001/9faf953fc36002da]
module.ooniapi_oonifindings.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OfinM-20240917211808752800000002/48adcfb18ae34d30]
module.ooniapi_ooniprobe.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OproM-20240917211808753100000004/1f2ea1732205872c]
module.ooniapi_cluster.aws_security_group.web: Refreshing state... [id=sg-0187eedfe39538357]
module.ooniapi_oonirun.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OrunM-20240917211808753100000003/93d4b2b6dc76acac]
module.network.aws_route_table.private: Refreshing state... [id=rtb-011463437da96c77b]
module.network.aws_subnet.private[0]: Refreshing state... [id=subnet-09314a43ec89d6331]
module.network.aws_subnet.private[1]: Refreshing state... [id=subnet-0b899a7ad10406d06]
module.network.aws_subnet.public[0]: Refreshing state... [id=subnet-0e7a4478be988463f]
module.network.aws_subnet.public[1]: Refreshing state... [id=subnet-0b18966cccfc9d5ef]
module.network.aws_route_table.public: Refreshing state... [id=rtb-0ccb0852e6a365a95]
module.ooniapi_cluster.aws_security_group.container_host: Refreshing state... [id=sg-0aa6a97400b619de3]
module.terraform_state_backend.aws_s3_bucket_server_side_encryption_configuration.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_versioning.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_public_access_block.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.network.aws_route_table_association.private[0]: Refreshing state... [id=rtbassoc-0e7933e6b804ff2c1]
module.network.aws_route_table_association.private[1]: Refreshing state... [id=rtbassoc-0c9cc0f117ef15fe7]
module.network.aws_route_table_association.public[0]: Refreshing state... [id=rtbassoc-0dbd7fb16801ee049]
module.network.aws_route_table_association.public[1]: Refreshing state... [id=rtbassoc-08ab18165bf481054]
module.ooniapi_cluster.aws_launch_template.container_host: Refreshing state... [id=lt-0e328a8671f870c64]
module.terraform_state_backend.aws_s3_bucket_policy.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.oonipg.aws_db_subnet_group.pg: Refreshing state... [id=ooni-tier0-postgres-dbsng]
module.ooniapi_frontend.aws_alb.ooniapi: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:loadbalancer/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6]
module.ooni_backendproxy.aws_security_group.nginx_sg: Refreshing state... [id=sg-0aca60db34215f5c8]
module.ooniapi_reverseproxy.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-reverseproxy]
module.terraform_state_backend.time_sleep.wait_for_aws_s3_bucket_settings[0]: Refreshing state... [id=2024-03-10T15:06:17Z]
module.terraform_state_backend.aws_s3_bucket_ownership_controls.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooni_backendproxy.aws_launch_template.ooni_backendproxy: Refreshing state... [id=lt-07dcdc7496cf92d29]
module.ooniapi_reverseproxy_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-reverseproxy]
module.ooniapi_cluster.aws_autoscaling_group.container_host: Refreshing state... [id=ooniapi-ecs-cluster20240310192644083800000003]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_https: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_http: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/d9b2448464179cd1]
aws_route53_record.ooniapi_frontend_alt["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniprobe.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_main: Refreshing state... [id=Z055356431RGCLK3JXZDL_api.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_oonirun.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_8.th.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniauth.dev.ooni.io_A]
module.ooni_backendproxy.aws_instance.oonibackend_proxy: Refreshing state... [id=i-069cdbd0840493854]
module.oonipg.aws_db_instance.pg: Refreshing state... [id=db-27N7Q6XIBNASFCOXN4N7C762L4]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/cefeff0d8aa3118a]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/5a872da4cc6b9135]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/b436b91883ae7c86]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/65e6f5e3aca0a4e5]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/cdc4e8e8eabb56f2]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/128c53ea760208fc]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/551c4128bb282fa4]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/2b09ed268181ba4f]
module.ooniapi_frontend.aws_alb_listener_rule.ooniapi_th: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/01ee7503374be8ff]
aws_secretsmanager_secret_version.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ|terraform-20240310182536837800000004]
aws_route53_record.postgres_dns: Refreshing state... [id=Z091407123AEJO90Z3H6D_postgres.dev.ooni.nu_CNAME]
module.ooniapi_oonirun.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonirun-td]
module.ooniapi_ooniprobe.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniprobe-td]
module.ooniapi_oonifindings.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonifindings-td]
module.ooniapi_ooniauth.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniauth-td]
module.ooniapi_ooniprobe.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe]
module.ooniapi_oonirun.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonirun]
module.ooniapi_ooniauth.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniauth]
module.ooniapi_oonifindings.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonifindings]
module.ooniapi_oonirun_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonirun]
module.ooniapi_ooniprobe_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniprobe]
module.ooniapi_ooniauth_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniauth]
module.ooniapi_oonifindings_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonifindings]
module.ooni_backendproxy.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oobpx20241125233815786000000001/9faf953fc36002da-20241125234416493100000002]
module.ooni_backendproxy.aws_route53_record.clickhouse_proxy_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_clickhouseproxy.dev.ooni.io_CNAME]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.ooni_backendproxy.aws_instance.oonibackend_proxy will be updated in-place
  ~ resource "aws_instance" "oonibackend_proxy" {
        id                                   = "i-069cdbd0840493854"
        tags                                 = {
            "Environment" = "dev"
            "Name"        = "ooni-tier0-backendproxy"
            "Repository"  = "https://github.com/ooni/devops"
        }
      ~ user_data                            = "f0e2f76c2f5dd67332408188b7e32165a4088ba3" -> "f1b95ad50ec382fc7090c683bf3dc7135e34e2b1"
        # (39 unchanged attributes hidden)

        # (9 unchanged blocks hidden)
    }

  # module.ooni_backendproxy.aws_security_group.nginx_sg will be updated in-place
  ~ resource "aws_security_group" "nginx_sg" {
        id                     = "sg-0aca60db34215f5c8"
      ~ ingress                = [
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - from_port        = 22
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "tcp"
              - security_groups  = []
              - self             = false
              - to_port          = 22
                # (1 unchanged attribute hidden)
            },
          - {
              - cidr_blocks      = [
                  - "10.0.100.0/24",
                  - "10.0.101.0/24",
                ]
              - from_port        = 9000
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "tcp"
              - security_groups  = []
              - self             = false
              - to_port          = 9000
                # (1 unchanged attribute hidden)
            },
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + from_port        = 80
              + ipv6_cidr_blocks = []
              + prefix_list_ids  = []
              + protocol         = "tcp"
              + security_groups  = []
              + self             = false
              + to_port          = 80
                # (1 unchanged attribute hidden)
            },
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + from_port        = 22
              + ipv6_cidr_blocks = []
              + prefix_list_ids  = []
              + protocol         = "tcp"
              + security_groups  = []
              + self             = false
              + to_port          = 22
            },
          + {
              + cidr_blocks      = [
                  + "10.0.100.0/24",
                  + "10.0.101.0/24",
                ]
              + from_port        = 9000
              + ipv6_cidr_blocks = []
              + prefix_list_ids  = []
              + protocol         = "tcp"
              + security_groups  = []
              + self             = false
              + to_port          = 9000
            },
        ]
        name                   = "ooni-bckprx20241125233815788600000002"
        tags                   = {
            "Environment" = "dev"
            "Name"        = "ooni-tier0-backendproxy"
            "Repository"  = "https://github.com/ooni/devops"
        }
        # (8 unchanged attributes hidden)
    }

  # module.ooniapi_oonifindings_deployer.aws_codepipeline.ooniapi will be updated in-place
  ~ resource "aws_codepipeline" "ooniapi" {
        id             = "ooniapi-oonifindings"
        name           = "ooniapi-oonifindings"
        tags           = {}
        # (5 unchanged attributes hidden)

      - trigger {
          - provider_type = "CodeStarSourceConnection" -> null

          - git_configuration {
              - source_action_name = "Source" -> null

              - push {
                  - branches {
                      - excludes = [] -> null
                      - includes = [
                          - "oonidata",
                        ] -> null
                    }
                }
            }
        }

        # (4 unchanged blocks hidden)
    }

  # module.ooniapi_reverseproxy_deployer.aws_codepipeline.ooniapi will be updated in-place
  ~ resource "aws_codepipeline" "ooniapi" {
        id             = "ooniapi-reverseproxy"
        name           = "ooniapi-reverseproxy"
        tags           = {}
        # (5 unchanged attributes hidden)

      - trigger {
          - provider_type = "CodeStarSourceConnection" -> null

          - git_configuration {
              - source_action_name = "Source" -> null

              - push {
                  - branches {
                      - excludes = [] -> null
                      - includes = [
                          - "master",
                        ] -> null
                    }
                }
            }
        }

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 4 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.
Pusher @DecFox
Action pull_request
Environment dev
Workflow .github/workflows/check_terraform.yml
Last updated Fri, 06 Dec 2024 20:56:30 GMT

@hellais hellais mentioned this pull request Dec 6, 2024
@hellais hellais marked this pull request as ready for review December 6, 2024 20:49
Copy link
Member

@hellais hellais left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed this with @DecFox in person and have deployed all the terraform stuff. Few changes will be applied additionally and then we land it.

@hellais hellais merged commit 6817c50 into main Dec 6, 2024
3 checks passed
@hellais hellais deleted the refactor/ansible-backend branch December 6, 2024 23:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Setup backend-hel.ooni.org as the primary test server
2 participants