Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

todo #10

Open
wintersys-projects opened this issue Oct 19, 2024 · 0 comments
Open

todo #10

wintersys-projects opened this issue Oct 19, 2024 · 0 comments

Comments

@wintersys-projects
Copy link
Owner

wintersys-projects commented Oct 19, 2024

To do Checklist

Next Week

  1. Generate snapshots on Exoscale Digital Ocean providerscripts/server/MonitorForSnapshotGenerated.sh (need to review how to generate snapshots and build machines from snapshots on Exoscale and follow that process) Is the snapshot private? Can test the process without deploying on a regular machine with the name "webserver"
  2. Test and solid up the snapshot and application/database update
    3 New configuration styles on wordpress, drupal, and moodle
  3. Why is there set -x output at the beginning of the script - looks untidy
  4. only do native firewall at end of build if its not a snapshot simply add the machine to the firewalls that existed from the regular build
  5. fingerprint thing just before templates

Over Christmas

  1. Error messages and repeat attempts for datastore methods on build machine
  2. Clean up utilities directories with new directories for config scripts
  3. Document all runtime directories and what each filetype does and the directory structure of runtime data on build machine

January

  1. document backups and snapshots and baselines
  2. update and test snapshots for exo and digitalocean
  3. Put in option for multiple region backups.
  4. updates to snapshots built machines do that snapshot can be built for Drupal and then deployed for joomla etc
  5. explain what needs to stay the same in template and what can change when deploying a snapshot. Can change SMTP but not we server type. Maybe call reconfigure webserver to install new webserver config for new application type
  6. Strip out other snapshotting but only once the new snapshotting has been tested
  7. Allow number_ws to be 5 and build them all initially. Set the default number of webservers in the datastore to be number_ws
  8. Do daily reboot on each machine type at 3:29 AM
    ${HOME}/providerscripts/utilities/UpdateInfrastructure.sh
    ${HOME}/providerscripts/utilities/UpdateSoftware.sh + reboot every day

https://www.digitalocean.com/community/tutorials/automate-the-creation-of-vpc-mesh-networks

  1. https://scotthelme.co.uk/heres-another-free-ca-as-an-alternative-to-lets-encrypt/

  2. Document/review maintenance mode

  3. Ssl for each provider dbaas Getting DbaaS hostname on build machine for digital ocean needs jq Check that manual dbaas setting works

  4. Review s3fs caching. Implement Vultr filesystem.

  5. authorisation mechanism

  6. Multi region deployments

  7. Final testing

  8. Review all documentation

  9. Reformat all script files

  10. Review docs/tutorial

encapsulation by reading values from env file within locally scoped functions


  1. weeks time

  2. build machine VPC check for each provider with BUILD_MACHINE_VPC=1 and build machine not attached to a VPC and attached to a VPC and check that the build script picks up both conditions correctly scale up to 5 and back down again for each one.

  3. Linode managed database see if can make it use ipv6 address instead of the hostname string


  1. weeks time

  2. Check if can get SSL certificate for each managed DB and install it on the webservers. If can, then, implement TLS certificates for joomla etc.

  3. Check if can get DBaaS certificate from DBaaS and does the certificate need to be checked for expiration.


  1. Look into Cunofs (compact S3fs goofys into datastoremount in build styles and add cunofs with license key to datadtire file). Maybe look into rclone as well.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountpoint-installation.html

https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountpoint-usage.html

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-options.html

S3fs -o ensure_diskfree -o use_cache

What about making entire webroot mounted using S3fs? Perdost assets to datadtire = 3 then mount S3fs as /var/www/html before application is installed????


January - look into authentication server

https://www.digitalocean.com/blog/announcing-global-load-balancer-beta

Multiple providers - have a template VAR NO_ACTIVE_REGIONS to divide up total number of web servers
When calling trusted DbaaS IP addresses get ips from the DNS system? Can list up addresses from dbaas and check them against DNS IP addresses and if there are different add the IP address to the trusted IP addresses in the dbaas
When adding ip to DNS get ips in DNS and add the IP that is being newly added and adjust DbaaS firewall
Tightdbaasfirewall two versions, multi region and single region. Single region as is. Multi region use the IP addresses from the DNS system

Dec - authentication server + multi provider webservers to DBaaS setup for resilience

  1. Authentication server
  2. Deploy webserver without and install authorisation server baseline but call the authorisation server authserver not webserver
  3. Authorisation server PHP file displays welcome page and asks for the users domain verified email address.
  4. If the user provides a domain verified email address, send them a one time link to a php file and generate a php file that will write their IP address to the file system. Every 2 seconds, check for newly authorised IP addresses and for each newly authorised IP address get list of current web servers and SSH onto each of them updating firewall to allow connections from newly authorised IP address.
  5. Kero list of authorised IP addresses on authorisation server as well as list of current webservers.
  6. Copy list of authorised client IP addresses to each webserver and have Setup firewall check for new authorised IP addresses maybe run setup firewall in each webserver from Auth server
  7. Every minute check for new webservers and if a new webservers is found ssh onto it and if WEBSERVER_READY exists enable the list of authorised ips on that webserver.

Multi-host-deploy ??? - strip out bastion from from docs


Dynamic scaling - monitors on each webserver and write the load to the datastore and aggregate the load values on the autoscaler/s and scale up according to some metric of how many new machines to scale to

/usr/bin/sar 1 1 | /usr/bin/tail -n -1 | /usr/bin/awk '{print $NF}' | /usr/bin/xargs printf "%.*f\n" "$p"

See if can do snasphots by remembering SERVER_USERNAME and SERVER_PASSWORD and reusing them from the template. ssh public key in template as well and purge ip addresses on the machines - just try it out and see if it works

format scripts open vi and immediately do gg=G and the script will be formatted nicely

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant