make_video fails with opencv

[emreybroyles]

Describe the bug
Some of the populate calls in the DLC pipeline aren't working and output {'success_count': 0, 'error_list': []} after updating spyglass and the spyglass-dlc environment, in attempts to fix the bug in #1146

To Reproduce
Steps to reproduce the behavior:

dlc_key = {'nwb_file_name': 'SC100220231007_.nwb', 'epoch': 11, 'video_file_num': 16, 'project_name': 'sideSleep_LED', 'dlc_model_name': 'sideSleep_LED_tutorial_00', 'dlc_model_params_name': 'default', 'dlc_centroid_params_name': 'default', 'dlc_si_cohort_centroid': 'green_red_led', 'dlc_si_cohort_orientation': 'green_red_led', 'dlc_orientation_params_name': 'default'}

sgp.DLCPosV1().populate(dlc_key)
sgp.DLCPosVideo().populate(dlc_key)

1. This error is on file eb_notebooks/DLCNotebooks/21_DLC_sideSleep_LED.ipynb
2. see the cells with the populate calls listed above

Expected behavior
The populate calls will work

Screenshots

[CBroz1]

Please post bugs with a snippet that can be run directly, without referring to outside material

This is the expected outcome for populating a key that has already been run. Please check that your keys are present in the list of unprocessed keys, Table.key_source

[emreybroyles]

Oh okay I see. So in the case of the video population in particular sgp.DLCPosVideo().populate(dlc_key) that I use to create a video with centroid and orientation overlaid to ensure my model is accurately labeling specific body parts, it looks like I already ran it before (but something wasn't working properly the first time). I wanted to run it again to actually be able to look at the video it created. From the source code it seems like this video isn't actually saved anywhere/I can't find the directory; it usually just outputs the video in the notebook itself after the cell runs. @CBroz1 Do you know how I can view this video then?

[CBroz1]

This is what determines where your video is saved:

spyglass/src/spyglass/position/v1/position_dlc_selection.py

Lines 399 to 407 in 03e3996

	output_video_filename = (
	key["nwb_file_name"].replace(".nwb", "")
	+ f"_{epoch:02d}_"
	+ f'{key["dlc_si_cohort_centroid"]}_'
	+ f'{key["dlc_centroid_params_name"]}'
	+ f'{key["dlc_orientation_params_name"]}.mp4'
	)
	if Path(output_dir).exists():
	output_video_filename = Path(output_dir) / output_video_filename

With the output dir stored in DLCPoseEstimationSelection
https://github.com/LorenFrankLab/spyglass/blob/master/src/spyglass/position/v1/position_dlc_selection.py#L364-L371

If it did not display the video as normal, I'd recommend reverting to version 0.5.1 and rerunning. If that fixes the issue, then this is a bug introduced in #870

[emreybroyles]

Thank you for pointing that code out to me! So I did find thatsgp.DLCPoseEstimationSelection() & dlc_key gives pose_estimation_output_dir = /nimbus/deeplabcut/output/SC100220231007/SC100220231007_11_model_sideSleep_LED_tutorial_00 and there is no mp4 file in that directory.
Next I will revert spyglass to 0.5.1 to see if the video is created.

[emreybroyles]

Reverted to 0.5.1 and am still getting the same outputs with no video created

[CBroz1]

It appears that there's an upper limit on the number of digits that the video file name can contain, relevant post. I found this by googling the error I saw in the output: (-215:Assertion failed) number < max_number in function 'cv::icvExtractPattern'

I generated the video with a different name. Please check the folder you mentioned above for TEST_VIDEO.mp4 and let me know if that looks right. If so, I can make some edits to have the video maker save to a temporary directory, and then copy it to the right location afterward

[emreybroyles]

Thanks Chris. TEST_VIDEO.mp4 does not look correct- it doesn't have the centroid or orientation overlaid; it just looks like the video colors were changed- almost inverted

[CBroz1]

It looks like the correct video can be generated with 0.5.1 and using the matplotlib processor

[CBroz1]

Future work will ...

• Revise the opencv processor to function like 0.5.1
• make matplotlib the default processor
• save the video to a temp location, them move - to avoid too many digits in file name

[CBroz1]

Closed with #1174, #1168