backend: keep the proposals list and details up to date

[CammilleCC]

The current implementation creates the cache file (/tmp/damnit_proposals.json) once: if it doesn't exist. It then populates the cache file with the list of proposals and their details according to MyMDC. The list of proposals are obtained from GPFS filesystem glob, which takes around 15s. This doesn't get refreshed unfortunately, and one has to delete the cache file manually to force recreating it to keep it up to date.

To get the relevant details per proposal from MyMDC, we do two queries:

1. fetch the proposal information given a proposal number
2. fetch the user information given a user id; this is used to determine the full name of the PI.

There is also fetch for the (refreshed) MyMDC token before all these operations.

The main questions for now are:

• Do have to keep a cache for DAMNIT proposals?
  • If yes, how to keep these updated? We have to consider newly created folders and deleted folders.
  • If no, how do we get an updated list? Could this be an addition to MyMDC?
• Do we include the proposal details on the cache?
  • If yes, how to keep these updated? We have to consider changes in MyMDC.
  • If no, how do we query them and avoiding too much fetches?

Ideas are very much welcome 😊