You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Dec 22, 2022. It is now read-only.
Lately, we had hanging tests on fv3net due to I/O problems with fv3gfs.util.open_restart. See ai2cm/fv3net#951. This only happened in the context of an FV3 run. open_restart worked fine with a simple testing script.
This PR solved these problems by using fsspec.get to pre-download the restart files: ai2cm/fv3net#978.
My guess is that it's related to multiple MPI jobs launching downloads and getting stuck. I've also found that fsppec.open does many more HTTP requests than fsspec.get. Maybe we should switch to use that here.
Since remote I/O paradigms are in constant flux, it might be simpler to deprecate this remote I/O capability, and focus fv3gfs-util on local usage.
The text was updated successfully, but these errors were encountered:
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Lately, we had hanging tests on fv3net due to I/O problems with fv3gfs.util.open_restart. See ai2cm/fv3net#951. This only happened in the context of an FV3 run. open_restart worked fine with a simple testing script.
This PR solved these problems by using
fsspec.get
to pre-download the restart files: ai2cm/fv3net#978.My guess is that it's related to multiple MPI jobs launching downloads and getting stuck. I've also found that
fsppec.open
does many more HTTP requests thanfsspec.get
. Maybe we should switch to use that here.Since remote I/O paradigms are in constant flux, it might be simpler to deprecate this remote I/O capability, and focus fv3gfs-util on local usage.
The text was updated successfully, but these errors were encountered: