Notes on install at OSU #13
Comments
|
We're on wifi, so we don't really have a fixed hostname. I'm setting up the |
|
On a fresh install, the vars-kb-server and the vars-user-server were throwing errors. I think this is due to a copy of MBARI's KB being loaded, which takes some time. I let it sit for a few minutes and restarted the services ( |
|
Running vars_build.sh results in: |
|
I've been fighting a battle with GitHub personal access tokens. The default (classic) are no longer able to read from MBARI's GitHub packages. I had to switch to the fine-grained tokens (in beta). Those tokens can read for GitHub packages, but I can't get it to write. This broke the KB build as it depends on an unreleased version of mbarix4j. The workaround was to copy most of the mbarix4j code into the project and remove that dependency. So it now builds. A note that the JVM has changed so much since the vars-kb was written that the app is barely usable anymore. Need to roll out https://github.com/mbari-org/oni and a new web interface ASAP. |
|
Building the vars-query is failing for the same reasons above, I can't publish updated artifacts to GitHub packages. Also, the build was failing with maven 3.9.7 (the latest) as noted at https://stackoverflow.com/questions/78542808/maven-project-fails-to-resolve-javafx-dependencies. Working on fixes. |
|
Workaround is to install sdkfx and eventbus locally. |
|
Note that the VARS KB is not deleting concepts. Need to investigate that. |
|
Installed vars-gridview, but it's built for SQL Server only and not postgres. Issue I ran into for the install are mbari-org/vars-gridview#77, mbari-org/vars-gridview#78, and mbari-org/vars-gridview#79 |
To register videos:tl;drcd ~/workspace/m3-quickstart/bin
conda activate m3-quickstart
# ./vars_register_medias_on_web.sh <camera name> <deployment name> <url to directory listing> -e
./vars_register_medias_on_web.sh "PICA" "PICA 008" "http://annotationstation.ceoas.oregonstate.edu/media/PICA008/" -eWarning The
So, it's very important to remember to use the The Note Make sure the url ends with a Details
|
Reading a CNV file in pythonThe PICA cam has a Seabird CTD attached. I'm including the sample file here. The best way to read it in python is using the seabird package ( >>> import seabird
>>> from seabird.cnv import fCNV
File "<stdin>", line 1
from seabird.cnv import fCNV
IndentationError: unexpected indent
>>> from seabird.cnv import fCNV
>>> profile = fCNV('/Users/annotationstation1/workspace/m3-quickstart/temp/media/PICA005/DunkTest1_20240522_23_24_59.cnv')
>>> profile.attributes
{'sbe_model': '19plus V2', 'seasave': 'V 7.26.7.121', 'instrument_type': 'CTD', 'nquan': '27', 'nvalues': '2607', 'start_time': 'May 22 2024 23:24:59 [System UTC, header]', 'bad_flag': '-9.990e-29', 'file_type': 'ascii', 'md5': 'ad739e8af38ad74e4797be5d22fb09af', 'datetime': datetime.datetime(2024, 5, 22, 23, 24, 59), 'filename': 'DunkTest1_20240522_23_24_59.cnv'}
>>> profile.keys()
['altM', 'CNDC', 'c0mS/cm', 'c0uS/cm', 'density', 'DEPTH', 'descentrate', 'flECO-AFL', 'oxygenvoltage', 'oxygen_ml_L', 'prdM', 'potemperature', 'potemp068C', 'tv290C', 'tv268C', 'turbWETntu0', 'timeS', 'timeM', 'timeH', 'timeJ', 'v0', 'v1', 'v2', 'v3', 'v4', 'v5', 'flag']
>>> profile['DEPTH']
masked_array(data=[-0.077, -0.12 , -0.12 , ..., -0.119, -0.14 , -0.13 ],
mask=False,
fill_value=-9.99e-29)
>>> profile['potemperature']
masked_array(data=[14.3776, 14.3775, 14.3774, ..., 14.3658, 14.3658,
14.366 ],
mask=False,
fill_value=-9.99e-29)DunkTest1_20240522_23_24_59.cnv.zip Note I'll write a transform later for Astrid when she has a file with GPS included after a ship deployment. |
|
Added Astrid (@abruptbathylab) with read access to https://github.com/mbari-org/m3-download so she can pull changes in the future. Note that m3-download uses pymssql to execute the following query, so it doesn't work with postgres: SELECT
a.uuid AS association_uuid,
o.uuid AS observation_uuid,
ir.uuid AS image_reference_uuid,
o.concept,
a.to_concept,
a.link_value,
ir.url,
o.activity,
o.observation_group,
im.recorded_timestamp,
im.elapsed_time_millis,
im.timecode,
vs.name,
v.start_time,
vr.width AS video_reference_width,
vr.height AS video_reference_height
FROM
M3_ANNOTATIONS.dbo.associations a INNER JOIN
M3_ANNOTATIONS.dbo.observations o ON a.observation_uuid = o.uuid INNER JOIN
M3_ANNOTATIONS.dbo.imaged_moments im ON o.imaged_moment_uuid = im.uuid INNER JOIN
M3_VIDEO_ASSETS.dbo.video_references vr ON im.video_reference_uuid = vr.uuid INNER JOIN
M3_VIDEO_ASSETS.dbo.videos v ON vr.video_uuid = v.uuid INNER JOIN
M3_VIDEO_ASSETS.dbo.video_sequences vs ON v.video_sequence_uuid = vs.uuid LEFT JOIN
M3_ANNOTATIONS.dbo.image_references ir ON JSON_VALUE(a.link_value, '$.image_reference_uuid') = ir.uuid
WHERE
a.link_name = 'bounding box'EDIT: I submitted mbari-org/m3-download#2 to investigate removing the SQL dependency so Astrid can download training sets from her data. |
|
Added pythia support to Astrid's docker compose file. I converted the fathomnet meg detector to torch script and renamed it pythia:
image: mbari/pythia
restart: always
ports:
- "9999:8080"
environment:
- YOLO_VERSION=8
- YOLOV5_RESOLUTION=1280
# - YOLOV5_RESOLUTION=640
volumes:
- ${BASE_DIR}/docker/pythia:/opt/models
networks:
- m3
command: run /opt/models/mbari_astrid_osu_yolov8_1280_2024-06-12.torchscript /opt/models/mbari_astrid_osu_yolov8_1280_2024-06-12.names
# command: run /opt/models/fathomnet_megalodon.torchscript /opt/models/fathomnet_megalodon.names |
|
Associations are not show in alphabetical order in the add association compbo box or when they add a quick button. Fix this in VARS |
|
Astrid was testing annotating on MBARI's db across VPN. Annotation works just fine, but Sharktopoda does not connect to VARS when VPN is up. This means Sharktopoda still works as a video player, but you can't localize using Sharktopoda as it can't tell VARS about new localizations. |
Changing annotation systemsOn the VARS Annotation toolbar, select the gear button (settings) and then in the dialog that pops up, the Configuration Server button. MBARI
OSU
|
hohonuuli
added
documentation
Merging seabird dataI added a script It's really a starting point as it will need to be modified when you start capturing position. Maybe Savana can updated it?. It will need to have it def __parse(cnv_file: str, year: int):
profile = fCNV(cnv_file)
n = len(profile['timeJ'])
for i in range(n):
timeJ = profile['timeJ'][i]
altitude = profile["altM"][i]
depth_meters = profile["DEPTH"][i]
temperature = profile["potemperature"][i]
oxygen = profile["oxygen_ml_L"][i]
# https://info.seabird.com/2026_SeaBird_c-mult_c-June-Newsletter_landing-Page-2.html
dt = datetime.datetime(year, 1, 1) + datetime.timedelta(days=(timeJ - 1))
date = dt.strftime("%Y-%m-%dT%H:%M:%S.%fZ")
yield {
# "latitude": latitude,
# "longitude": longitude,
"depth_meters": depth_meters,
"temperature_celsius": temperature,
"oxygen_ml_l": oxygen,
# "salinity": salinity,
"recorded_timestamp": date,
"altitude": altitude,
} |
|
Notes from @lonnylundsten and @kwalz on the model they trained for Astrid: From Kris: From Lonny:
Model conversionFor pythia, the model needs to be converted to torch script and a names files is need. conda activate ultralytics
yolo export model=mbari_astrid_osu_yolov8_1280_2024-06-12.pt format=torchscript imgsz=1280 |
|
Attaching the diff for OSUs install: |
To get started I did the following:
Tasks
m3-quickstart is installed on on
/Users/annotationstation1/workspace/m3-quickstartThe text was updated successfully, but these errors were encountered: