Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow recompute via _make_file func #1093

Draft
wants to merge 20 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,25 @@
# Change Log

## [0.5.5] (Unreleased)

### Release Notes

<!-- Running draft to be removed immediately prior to release. -->

<!-- When altering tables, import all foreign key references. -->

```python
import datajoint as dj
from spyglass.spikesorting.v1.recording import * # noqa
from spyglass.linearization.v1.main import * # noqa

dj.FreeTable(dj.conn(), "common_nwbfile.analysis_nwbfile_log").drop()
dj.FreeTable(dj.conn(), "common_session.session_group").drop()
TrackGraph.alter() # Add edge map parameter
SpikeSortingRecording().alter()
SpikeSortingRecording().update_ids()
```

## [0.5.4] (December 20, 2024)

### Infrastructure
Expand All @@ -8,6 +28,7 @@
#1108, #1172, #1187
- Add docstrings to all public methods #1076
- Update DataJoint to 0.14.2 #1081
- Remove `AnalysisNwbfileLog` #1093
- Allow restriction based on parent keys in `Merge.fetch_nwb()` #1086, #1126
- Import `datajoint.dependencies.unite_master_parts` -> `topo_sort` #1116,
#1137, #1162
Expand All @@ -19,6 +40,7 @@
- Update DataJoint install and password instructions #1131
- Fix dandi upload process for nwb's with video or linked objects #1095, #1151
- Minor docs fixes #1145
- Add Nwb hashing tool #1093
- Test fixes
- Remove stored hashes from pytests #1152
- Remove mambaforge from tests #1153
Expand Down Expand Up @@ -78,6 +100,7 @@
- Fix bug in `get_group_by_shank` #1096
- Fix bug in `_compute_metric` #1099
- Fix bug in `insert_curation` returned key #1114
- Add fields to `SpikeSortingRecording` to allow recompute #1093
- Fix handling of waveform extraction sparse parameter #1132
- Limit Artifact detection intervals to valid times #1196

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ addopts = [
# "--pdb", # drop into debugger on failure
"-p no:warnings",
# "--no-teardown", # don't teardown the database after tests
"--quiet-spy", # don't show logging from spyglass
# "--quiet-spy", # don't show logging from spyglass
# "--no-dlc", # don't run DLC tests
"--show-capture=no",
"--pdbcls=IPython.terminal.debugger:TerminalPdb", # use ipython debugger
Expand Down
20 changes: 19 additions & 1 deletion src/spyglass/common/common_dandi.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,26 @@ class DandiPath(SpyglassMixin, dj.Manual):
dandi_instance = "dandi": varchar(32)
"""

def fetch_file_from_dandi(self, key: dict):
def key_from_path(self, file_path) -> dict:
return {"filename": os.path.basename(file_path)}

def has_file_path(self, file_path: str) -> bool:
return bool(self & self.key_from_path(file_path))

def raw_from_path(self, file_path) -> dict:
return {"filename": Path(file_path).name.replace("_.nwb", ".nwb")}

def has_raw_path(self, file_path: str) -> bool:
return bool(self & self.raw_from_path(file_path))

def fetch_file_from_dandi(
self, key: dict = None, nwb_file_path: str = None
):
"""Fetch the file from Dandi and return the NWB file object."""
if key is None and nwb_file_path is None:
raise ValueError("Must provide either key or nwb_file_path")
key = key or self.key_from_path(nwb_file_path)

dandiset_id, dandi_path, dandi_instance = (self & key).fetch1(
"dandiset_id", "dandi_path", "dandi_instance"
)
Expand Down
8 changes: 2 additions & 6 deletions src/spyglass/common/common_ephys.py
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,7 @@ def make(self, key):
"""
# get the NWB object with the data; FIX: change to fetch with
# additional infrastructure
lfp_file_name = AnalysisNwbfile().create(key["nwb_file_name"]) # logged
lfp_file_name = AnalysisNwbfile().create(key["nwb_file_name"])

rawdata = Raw().nwb_object(key)
sampling_rate, interval_list_name = (Raw() & key).fetch1(
Expand Down Expand Up @@ -563,7 +563,6 @@ def make(self, key):
},
replace=True,
)
AnalysisNwbfile().log(key, table=self.full_table_name)
self.insert1(key)

def nwb_object(self, key):
Expand Down Expand Up @@ -758,9 +757,7 @@ def make(self, key):
6. Adds resulting interval list to IntervalList table.
"""
# create the analysis nwb file to store the results.
lfp_band_file_name = AnalysisNwbfile().create( # logged
key["nwb_file_name"]
)
lfp_band_file_name = AnalysisNwbfile().create(key["nwb_file_name"])

# get the NWB object with the lfp data;
# FIX: change to fetch with additional infrastructure
Expand Down Expand Up @@ -956,7 +953,6 @@ def make(self, key):
"previously saved lfp band times do not match current times"
)

AnalysisNwbfile().log(lfp_band_file_name, table=self.full_table_name)
self.insert1(key)

def fetch1_dataframe(self, *attrs, **kwargs) -> pd.DataFrame:
Expand Down
Loading
Loading