Automate rebuilds #100
Merged
Automate rebuilds #100
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Uses `nextstrain remote upload` command to deploy builds to a specified `deploy_url` provided in the config. Purposely uses the build name from the config instead of the wildcard build name to allow the rule to be a target rule without providing explicit output files. The default `deploy_url` is added to an optional config file `config/nextstrain_automation.yaml` that is intended for internal use by the Nextstrain team.
The `--output` option allows us to save the curl output in a file. This will allow us to parse the response for the Slack thread timestamp required to post threaded messages. The `--thread-ts` option allows us to provide a timestamp for the Slack message to be sent as a threaded message. The `--broadcast` option allows us to send a threaded Slack message that are also broadcasted to the channel for high priority messages such as error messages. This will make the Slack notifications for automated builds much cleaner where each build will have their own start message with threaded replies for further updates.
Sends a Slack notification that indicates the start of the build pipeline. Includes the AWS Batch and GitHub Action links if their ids are provided via environment variables `AWS_BATCH_JOB_ID` and `GITHUB_RUN_ID`. Saves the onstart Slack message timestamp to a file so that it can be accessed by later notification jobs to thread their responses.
Sends a Slack notification that indicates the build has successfully finished as a threaded reply to the onstart message.
Sends a Slack notification taht indicates an error has occurred in the build. The message is sent as a threaded reply that is broadcasted to the channel to get more visibility on the error.
Sends a Slack message that indicates the build has build deployed as a threaded reply to the onstart message.
Added the `hmpxv1` and `mpxv` builds as separate GitHub Action workflows so that they can be run separately as needed. They share a repository dispatch type (`rebuild`) so that they can both be triggered at the same time by the ingest workflow for automated builds.
Will be using this script to trigger the monkeypox builds after the latest data has been uploaded to S3.
Triggers the rebuild GitHub Action workflows via a repository dispatch type `rebuild`, which will trigger both the hmpxv1 and the mpxv builds. The ingest pipeline will only trigger the builds if the config sets `trigger_rebuild` to true as included in the optional config file. This currently will trigger rebuild regardless of whether there's new data included. The following commit will add changes to a check whether new data has been uploaded to S3.
We do not want to waste time and resources to run rebuilds when there's no new data from ingest. This commit checks the output from the uploads of the metadata TSV and sequences FASTA files to verify they have been uploaded to S3. If the output includes the message "files are identical" then we know they have not been updated and skip the trigger for rebuilds.
|
@joverlee521 is it intended that the output of the auto-builds only contains around 20 sequences? I downloaded the output tree and put it in Auspice. Looks good so I'll merge so we can see how it goes - not much can go wrong here I think - in worst case we'll revert ;) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of proposed changes
Automates rebuilds for
mpxvandhpmxv1to run when there's new data from ingest. Includes threaded Slack replies for builds since we don't want to mix up Slack messages from the separate builds. (Note this is separate from #36, which is specifically for ingest).Testing
Unfortunately cannot fully test the ingest trigger since repository dispatch events only trigger the workflow on the main branch.