Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Irregular obs dacycle #47

Merged
merged 65 commits into from
Aug 26, 2024
Merged

Irregular obs dacycle #47

merged 65 commits into from
Aug 26, 2024

Conversation

kysolvik
Copy link
Collaborator

  • Add support for irregular observations in time and space (i.e. each cycle does not need same number of observations) using padded matrices.
  • Replace cycle() argument for number of cycles to run: replaced "timesteps" with "n_cycles"
  • Correct etkf and 3dvar to run n_cycles instead of a certain number of model timesteps.
  • Add tests for dacyclers.
  • Update docstrings.

kysolvik added 30 commits July 12, 2024 15:47
@kysolvik
Support for irregular observation timing in ETKF
fe8148f
@kysolvik
Support missing time observations in var4d_backprop
1a85e5f
@kysolvik
Moving time filtering functionality to separte module, untested
d7bd91a
@kysolvik
Base test for dacycler
a88ea5b
@kysolvik
3DVar L96 tests
6b8cc3f
@kysolvik
ETKF tests
f353695
@kysolvik
Updated rcond (deprecated) to rcol
40e268b
@kysolvik
4D-Var and 4D-Var Backprop tests
f63e560
@kysolvik
Corrected default R dimensions for var4d and var4dBackprop
d5fb965
@kysolvik
Resizing needs to be done with numpy, not jax.numpy
ccb2521
@kysolvik
Use utils for obs filtering in var4d_backprop
48bcaac
@kysolvik
Add AND NOT for start/end_inclusive false in time filtering
7c1a913
@kysolvik
Use utils for etkf time filtering
5a557d8
@kysolvik
Add variable observation time support to var4d
93145a5
@kysolvik
Remove padding method from pbackprop (using _utils module instead
eb33368
@kysolvik
Use jax.lax.cond for obs masking
7cb3c2a
@kysolvik
Use jax.lax.cond for obs masking in backprop
ccab1fd
@kysolvik
Use .at[].get() instead of dynamic slice to fix bug when padded indic…
544eeff 
…es have len greater than available obs values
@kysolvik
Add ability to compute obs_window_indices on the fly
adc2bf4
@kysolvik
Small corrections to backprop obs_window_indices
7de6c58
@kysolvik
Automatically calculate obs)window_indices for 4dvar, WIP getting ver…
c97f4f2 
…y, very small differences
@kysolvik
var4d, bp tests with no provided obs_window_indices
1bc2888
@kysolvik
Don't print loss values but save as attribute, and fix obs_mask out o…
8d50749 
…f bounds issue
@kysolvik
Fix out of bounds indexing for os_mask
5b7ce6d
@kysolvik
Remove debugging printing for obs_window_indices
9aa95c6
@kysolvik
Remove extra indexing leftover from debugging
8c7deaa
@kysolvik
Restore testing to using obs_window_indices
148698f
@kysolvik
Working obs_loc_mask for etkf, but crude
38195a3
@kysolvik
Add stationary observer attribute to obs vector
e349291
@kysolvik
Update dacycler for irregular observation locations
8a45cd3
kysolvik added 24 commits August 12, 2024 16:11
@kysolvik
Set up H as int array
5ed85e5
@kysolvik
Removed commented stationary observers check
fca6a7b
@kysolvik
Added warning for nonstationary observers and store_as_jax=True
31703d8
@kysolvik
Updated var3d to run over analysis windows (not model timesteps) and …
e1c05ae 
…can adjust when in window analysis is valid
@kysolvik
Updated ETKF to properly round over analysis windows, not model times…
7645076 
…teps
@kysolvik
Updated 4dvar and backprop to use n_cycles and properly specific obs_…
3712469 
…time_in_window
@kysolvik
ETKF and Var3d updated to match inputs, model prediction lenght of va…
80b9ec3 
…r4d and backprop
@kysolvik
Updated dacycler tests
391623b
@kysolvik
Updated docstrings
eca2c47
@kysolvik
Fix calculation of model timesteps for 4dvar and backprop
7d499e9
@kysolvik
4Dvar and 4dvarbp require H to be jax array
f998fa6
@kysolvik
Don't alter custom Hs in 4DVar and 4DVar Backprop
0d79b4f
@kysolvik
Fix calling Ht before assignment
51c6615
@kysolvik
Improvements to var4d backprop speed
2a7234a
@kysolvik
Streamline iter looping
e174889
@kysolvik
Sped up backprop with irregular obs
f0de50f
@kysolvik
Workaround for obs_loc_mask, use conditional
4fff86c
@kysolvik
Rework obs window indices calc to use conditional
167cbb1
@kysolvik
Faster regular obs case for var4d
b04a1b3
@kysolvik
Fix linting errors for var4d_backprop
be37444
@kysolvik
Irregular obs option for exact hessian
9a40680
@kysolvik
Updated var4d_backprop LR and decay defaults to 0.5, 0.5
f21200d
@kysolvik
Removing pyqg tests
3bbcc1d
@kysolvik
Import warnings for observer
fdbe443
@kysolvik kysolvik mentioned this pull request Aug 23, 2024
Open
@kysolvik kysolvik mentioned this pull request Aug 23, 2024
Merged
@StevePny StevePny merged commit ccd797a into main Aug 26, 2024
1 check passed
@kysolvik kysolvik deleted the irregular-obs-dacycle branch August 27, 2024 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kysolvik @StevePny