Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save resources from validation execution #620

Closed
meganwolf0 opened this issue Aug 28, 2024 · 2 comments · Fixed by #612
Closed

Save resources from validation execution #620

meganwolf0 opened this issue Aug 28, 2024 · 2 comments · Fixed by #612
Assignees
Labels
enhancement New feature or request

Comments

@meganwolf0
Copy link
Collaborator

Is your feature request related to a problem? Please describe.

Sub-task of #301:

In order to reproduce observations, the resources that provided the input to the validation need to be saved, as they are transient across time.

Describe the solution you'd like

  • Given resources would like to be saved from a lula validate run
  • When --save-resources is added
  • Then resources are saved

Describe alternatives you've considered

none

Additional context

none

Expected Deliverable

Adding the functionality to "save resources" to both remote files or the back-matter

Updating existing tests + adding new tests as necessary

Update CLI command docs

@meganwolf0 meganwolf0 added the enhancement New feature or request label Aug 28, 2024
@meganwolf0 meganwolf0 self-assigned this Aug 28, 2024
@github-actions github-actions bot added the triage Awaiting triage from the team label Aug 28, 2024
@meganwolf0 meganwolf0 moved this from 🆕 New to 🏗 In Progress in Lula and Friends Aug 28, 2024
@meganwolf0 meganwolf0 removed the triage Awaiting triage from the team label Aug 28, 2024
@meganwolf0 meganwolf0 linked a pull request Aug 28, 2024 that will close this issue
6 tasks
@brandtkeller
Copy link
Member

In what format would the resources be saved to the back matter? 1 observation == 1 resource item in the backmatter? How much complexity is involved in storing the item and retrieving it from the backmatter?

Similar for external file -> are we expecting to specify the file path? Are we appending data to an existing default file?

Just to challenge intent here some - Do we need to put this data into the back-matter?

@meganwolf0
Copy link
Collaborator Author

In what format would the resources be saved to the back matter? 1 observation == 1 resource item in the backmatter? How much complexity is involved in storing the item and retrieving it from the backmatter?

Similar for external file -> are we expecting to specify the file path? Are we appending data to an existing default file?

Just to challenge intent here some - Do we need to put this data into the back-matter?

All great questions - I definitely was opinionated on the implementation (see PR) - so I'll answer with what implementation I picked, but obviously subject to a larger discussion

  • The resources are in the backmatter where the description field is the json data -> they are linked via the link property on the related observation - so yes it's 1:1 observation:resource. Complexity I don't think is bad, I have an initial dev command to pull those resources from the backmatter (need to update for the remote case, see Dev commands to extract validation and resources by observation uuid #621)
  • File path was a little more complicated, but I currently have an individual file output for each resource. It defaults to a directory at the same location as assessment-results called resources where all the .json resides. This was a little trickier, I thought for an initial POC this would suffice for dumping resources externally
  • Do we need to put the data into the back-matter - probably not.. but since that's the functionality I initially wrote I didn't want to discard it unless we are solidly in the camp of no resource in back-matter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants