Each release, cache the ICS files and only push the changed files #462
Labels
enhancement
New feature or request
good first issue
Good for newcomers
waiting-on-time
Not a high priority or require more time to implement than I have at the moment
At the moment when there's a change in the schedule, each of the 600+ calendar files will be updated, even if the old file and the new file are identical. It's quite common for there to be no change to the old and new files because the Cape Town loadshedding schedules and the Eskom loadshedding schedules are largely disjoint, so if Eskom announces an update then none of the CPT schedules will be changed.
This is a problem because uploading the files is the most expensive step of the release process. Uploading takes up to 10minutes each time, and it also consumes the hourly rate limit. In the past, there have been times when Eskom and Cape Town announced multiple schedule updates within an hour, and eskom-calendar exceeded the GitHub rate limit quota, and so had to wait about 30 minutes until the quota got reset and the files could be uploaded. Also, the number of calendar files isn't going down any time soon, so this problem of the rate limit quota will only get worse.
This could be fixed by modifying the
beyarkay/update-existing-release
repo to also upload a cachefile.cache
which would be a YAML-formatted file containing a list of all the files, their hashes, and the last time they were updated. Logic would need to be added to only upload a file if the new hash and the existing hash have changed.Ideally this change would be upstreamed to the original
IsaacShelton/update-existing-release
repo, but the in-house forkbeyarkay/update-existing-release
is quite behind and I'm not sure if upstreaming these changes might cause other issues.The text was updated successfully, but these errors were encountered: