Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce repository update periods #972

Closed
4 tasks done
AMDmi3 opened this issue Jan 10, 2020 · 1 comment
Closed
4 tasks done

Introduce repository update periods #972

AMDmi3 opened this issue Jan 10, 2020 · 1 comment

Comments

@AMDmi3
Copy link
Member

AMDmi3 commented Jan 10, 2020

Some repositories (crates.io, aur) are quite heavy to fetch, and fetching them also generates load on upstream. Since update duration is getting smaller and smaller with #956, we may need to introduce minimal update period for such repositories - this would

  • generate less load on upstream
  • make our updates even faster
  • be useful in future when repositories are update independently

Plan:

  • add repositories table field, last_update
  • update it after successful fetch
  • introduce repos.d property min_update_period
  • check these and skip extra fetches
@AMDmi3
Copy link
Member Author

AMDmi3 commented Jan 16, 2020

The first idea was to also set default update period to 1 hour. However for now it leads to pessimization, causing database updates without any package changes. Setting default update period to 10mins, which practically disabled the limit as any database update cycle is longer than that. The correct solution would be to untie database updates from fetches (#551).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant