Add ProtocolUnit
log preservation to compute services and object store, with reference in state store; expose via AlchemiscaleClient
#349
Labels
Milestone
It's currently the case that
ProtocolUnit
s for manyProtocol
s emit logs using the standardgufe
logging mechanism, using their ownProtocolUnit.logger
handler. However, withinalchemiscale
we neither capture nor preserve these logs in execution.We should empower
alchemiscale
users to examine the logs emitted by aTask
'sProtocolDAGResult
s by allowing them to retrieve the logs via:AlchemiscaleClient.get_result_logs(pdrr: ScopedKey, order='unit')
: get logs for allProtocolUnitResult
s in theProtocolDAGResult
, ordered by'time'
or by'unit'
AlchemiscaleClient.get_unit_logs(purr: ScopedKey)
: get the logs for a singleProtocolUnitResult
AlchemiscaleClient.get_result_units(pdrr: ScopedKey)
: getScopedKey
s for allProtocolUnitResultRef
s given theScopedKey
of theirProtocolDAGResultRef
This will require that our compute services capture logs from each
ProtocolUnit
they execute, then upload those logs to the object store alongside theProtocolDAGResult
itself, creating aProtocolUnitResultRef
and aProtocolUnitResultLogRef
in the state store to enable retrieval. TheProtocolUnitResultRef
for eachProtocolUnitResult
in theProtocolDAGResult
will need to preserve dependency relationships to the otherProtocolUnitResultRef
s in order to enable dependency-order reporting. The sameProtocolUnitResultRef
s would likely be used for STDERR, STDOUT, and result file retrieval implemented under a similar pattern later.Dependent somewhat on #348, since we assume
ProtocolDAGResultRef
ScopedKey
s are now exposed to users.The text was updated successfully, but these errors were encountered: