Improved Computer Vision webinar logic / code

.typos.toml

@@ -1,5 +1,14 @@
 [files]
-extend-exclude = ["*.csv", "sign-language-detection-yolov5/*", "orbit-user-analysis/steps/report.py", "customer-satisfaction/pipelines/deployment_pipeline.py", "customer-satisfaction/streamlit_app.py", "nba-pipeline/Building and Using An MLOPs Stack With ZenML.ipynb", "customer-satisfaction/tests/data_test.py"]
+extend-exclude = [
+    "*.csv",
+    "sign-language-detection-yolov5/*",
+    "orbit-user-analysis/steps/report.py",
+    "customer-satisfaction/pipelines/deployment_pipeline.py",
+    "customer-satisfaction/streamlit_app.py",
+    "nba-pipeline/Building and Using An MLOPs Stack With ZenML.ipynb",
+    "customer-satisfaction/tests/data_test.py",
+    "end-to-end-computer-vision/**/*.ipynb"
+]
 
 [default.extend-identifiers]
 #  HashiCorp = "HashiCorp"
@@ -14,6 +23,9 @@ lenght = "lenght"
 preprocesser = "preprocesser"
 Preprocesser = "Preprocesser"
 Implicitly = "Implicitly"
+fo = "fo"
+mapp = "mapp"
+polution = "polution"
 
 [default]
 locale = "en-us"

end-to-end-computer-vision/.gitignore

@@ -3,3 +3,4 @@
 data/
 runs/
 **/tmp*
+runs_dir

end-to-end-computer-vision/README.md

@@ -4,6 +4,8 @@ This is a project that demonstrates an end-to-end computer vision pipeline using
 ZenML. The pipeline is designed to be modular and flexible, allowing for easy
 experimentation and extension.
 
+![diagram.png](_assets/diagram.png)
+
 The project showcases the full lifecycle of a computer vision project, from data
 collection and preprocessing to model training and evaluation. The pipeline also
 incorporates a human-in-the-loop (HITL) component, where human annotators can
@@ -12,36 +14,230 @@ label images to improve the model's performance, as well as feedback using
 
 The project uses the [Ship Detection
 dataset](https://huggingface.co/datasets/datadrivenscience/ship-detection) from
-[DataDrivenScience](https://datadrivenscience.com/) on the Hugging Face Hub, which contains images of ships in
-satellite imagery. The goal is to train a model to detect ships in the images.
-Note that this isn't something that our YOLOv8 model is particularly good at out
-of the box, so it serves as a good example of how to build a pipeline that can
-be extended to other use cases.
+[DataDrivenScience](https://datadrivenscience.com/) on the Hugging Face Hub,
+which contains images of ships in satellite imagery. The goal is to train a
+model to detect ships in the images. Note that this isn't something that our
+YOLOv8 model is particularly good at out of the box, so it serves as a good
+example of how to build a pipeline that can be extended to other use cases.
+
+This project needs some infrastructure and tool setup to work. Here is a list of
+things that you'll need to do.
 
-## Run this pipeline
+## ZenML
 
-### Setup
+We recommend using our [ZenML Cloud offering](https://cloud.zenml.io/) to get a
+deployed instance of zenml:
 
-You'll need to run the following:
+### Set up your environment
 
 ```bash
-zenml integration install label_studio torch gcp mlflow -y
 pip install -r requirements.txt
-pip uninstall wandb
+zenml integration install label_studio torch gcp mlflow -y
+pip uninstall wandb  # This comes in automatically
+```
+
+And to use the Albumentations and annotation plugins in the last step, you'll
+need to install them:
+
+```bash
+fiftyone plugins download https://github.com/jacobmarks/fiftyone-albumentations-plugin
+
+fiftyone plugins download https://github.com/voxel51/fiftyone-plugins --plugin-names @voxel51/annotation
 ```
 
-You can also set the following environment variables:
+You should also set up the following environment variables:
 
 ```bash
 export DATA_UPLOAD_MAX_NUMBER_FILES=1000000
 export WANDB_DISABLED=True
 ```
 
-And to use the Albumentations and annotation plugins, you'll need to install
-them:
+### Connect to your deployed ZenML instance
 
 ```bash
-fiftyone plugins download https://github.com/jacobmarks/fiftyone-albumentations-plugin
+zenml connect --url <INSERT_ZENML_URL_HERE>
+```
 
-fiftyone plugins download https://github.com/voxel51/fiftyone-plugins --plugin-names @voxel51/annotation
+## Cloud Provider
+
+We will use GCP in the commands listed below, but it will work for other cloud
+providers.
+
+### Follow our guide to set up your credential for GCP
+
+[Set up a GCP service
+connector](https://docs.zenml.io/stacks-and-components/auth-management/gcp-service-connector)
+
+### Set up a bucket to persist your training data
+
+### Set up a bucket to use as artifact store within ZenML
+
+[Learn how to set up a GCP artifact store stack component within zenml
+here](https://docs.zenml.io/stacks-and-components/component-guide/artifact-stores)
+### Set up vertex for pipeline orchestration
+
+[Learn how to set up a Vertex orchestrator stack component within zenml
+here](https://docs.zenml.io/stacks-and-components/component-guide/orchestrators/vertex)
+### For training on accelerators like GPUs/TPUs set up Vertex
+
+[Learn how to set up a Vertex step operator stack component within zenml
+here](https://docs.zenml.io/stacks-and-components/component-guide/step-operators/vertex)
+### Set up Container Registry
+
+[Learn how to set up a google cloud container registry component within zenml
+here](https://docs.zenml.io/stacks-and-components/component-guide/container-registries/gcp)
+
+## Label Studio
+
+### [Start Label Studio locally](https://labelstud.io/guide/start)
+### [Follow these ZenML instructions to set up Label Studio as a stack component](https://docs.zenml.io/stacks-and-components/component-guide/annotators/label-studio)
+### Create a project within Label Studio and name it `ship_detection_gcp`
+### [Set up Label Studio to use external storage](https://labelstud.io/guide/storage) 
+use the first bucket that you created to data persistence
+
+## ZenML Stacks
+
+### Local Stack
+
+The local stack should use the `default` orchestrator, a gcp remote artifact
+store that we'll call `gcp_artifact_store` here and a local label-studio
+annotator that we'll refer to as `label_studio_local`.
+
+```bash
+# Make sure to replace the names with the names that you choose for your setup
+zenml stack register <local_stack> -o default -a <gcp_artifact_store> -an <label_studio_local>
+```
+
+### Remote Stack
+
+The remote stack should use the `vertex_orchestrator` , a `gcp_artifact_store`,
+a `gcp_container_registry` and a `vertex_step_operator`.
+
+
+```bash
+# Make sure to replace the names with the names that you choose for your setup
+zenml stack register <gcp_stack> -o <vertex_orchestrator> -a <gcp_artifact_store> -c <gcp_container_registry> -s <vertex_step_operator>
 ```
+
+The project consists of the following pipelines:
+
+## data_ingestion_pipeline
+
+This pipeline downloads the [Ship Detection
+dataset](https://huggingface.co/datasets/datadrivenscience/ship-detection). This
+dataset contains some truly huge images with a few hundred million pixels. In
+order to make these useable, we break down all source images into manageable
+tiles with a maximum height/width of 1000 pixels. After this preprocessing is
+done, the images are uploaded into a cloud bucket and the ground truth
+annotations are uploaded to a local Label Studio instance.
+
+### Configure this pipeline
+The configuration file for this pipeline lives at `./configs/ingest_data.yaml`.
+Make sure in particular to change `data_source` to point at the GCP bucket which
+is dedicated to be the storage location for the data. Also make sure to adjust
+the `ls_project_id` to correspond to the id of your project within Label Studio.
+
+### Run this pipeline
+
+Label Studio should be up and running for the whole duration of this pipeline
+run.
+
+```bash
+zenml stack set <local_stack>
+python run.py --ingest
+```
+
+## data_export_pipeline
+
+This pipeline exports the annotations from Label Studio and loads it into the
+ZenML artifact store to make them accessible to downstream pipelines.
+
+### Configure this pipeline
+The configuration file for this pipeline lives at `./configs/data_export.yaml`.
+Make sure in particular to change `dataset_name` to reflect the name of the
+dataset within Label Studio.
+
+### Run this pipeline
+
+Label Studio should be up and running for the whole duration of this pipeline
+run.
+
+```bash
+zenml stack set <local_stack>
+python run.py --export
+```
+
+## training_pipeline
+
+This pipeline trains a YOLOv8 object detection model. 
+
+### Configure this pipeline
+You can choose to run this pipeline locally or on the cloud. These two options
+use two different configuration files. For local training:
+`./configs/training_pipeline.yaml`. For training on the cloud:
+`./configs/training_pipeline_remote_gpu.yaml`. Make sure `data_source` points to
+your cloud storage bucket.
+
+### Run this pipeline
+
+This pipeline requires the associated model (see the model section of the
+configuration yaml file) to have a version in the `staging` stage. In order to
+promote the model produced by the latest run of the `data_export_pipeline`, run
+the following code:
+
+```bash
+zenml model version update <MODEL_NAME> latest -s staging 
+```
+
+For local training run the following code:
+
+```bash
+zenml stack set <local_stack>
+python run.py --training --local
+```
+
+For remote training run the following code:
+
+```bash
+zenml stack set <remote_stack>
+python run.py --training
+```
+
+## inference_pipeline
+
+This pipeline performs inference on the object detection model.
+
+### Configure this pipeline
+You can configure this pipeline at the following yaml file
+`./configs/inference_pipeline.yaml`.  Make sure `data_source` points to your
+cloud storage bucket that contains images that you want to perform batch
+inference on
+
+### Run this pipeline
+
+This pipeline requires the associated model (see the model section of the
+configuration yaml file) to have a version in the `production` stage. In order
+to promote the model produced by the latest run of the `training_pipeline`, run
+the following code:
+
+```bash
+zenml model version update <MODEL_NAME> staging -s production 
+```
+
+```bash
+zenml stack set <local_stack>
+python run.py --inference
+```
+
+
+## Analyze and Curate your data through FiftyOne
+
+Now to close the loop, we will import the predictions into FiftyOne. All you'll
+need to do is run:
+
+```bash
+python run.py --fiftyone
+```
+
+Within FiftyOne, you can now analyze all the predictions and export them back to
+Label Studio for finetuned labeling and retraining.

end-to-end-computer-vision/configs/data_export_alexej.yaml → end-to-end-computer-vision/configs/data_export.yaml

@@ -2,7 +2,7 @@ enable_cache: False
 
 # pipeline configuration
 parameters:
-  dataset_name: "ship_detection_gcp" # "ship_detection"
+  dataset_name: "ship_detection_gcp"  # This is the name of the dataset in label studio
 
 # Configuration of the Model Control Plane
 model:

end-to-end-computer-vision/configs/cloud_inference.yaml → end-to-end-computer-vision/configs/inference_pipeline.yaml

@@ -19,7 +19,7 @@ steps:
     enable_cache: False
     enable_step_logs: False
     parameters:
-      inference_data_source: "gs://zenml-internal-artifact-store/inference_cv_webinar"
+      inference_data_source:   # Insert your bucket path here where the inference images live e.g. "gs://foo/bar"
 
 # configuration of the Model Control Plane
 model:

end-to-end-computer-vision/configs/ingest_data.yaml

@@ -1,10 +1,11 @@
 
 steps:
-  download_dataset_from_hf:
+  download_and_tile_dataset_from_hf:
     enable_cache: True
+    enable_step_logs: False
     parameters:
       dataset: "datadrivenscience/ship-detection"
-      data_source: "gs://zenml-internal-artifact-store/label_studio_cv_webinar"
+      data_source: # Insert your bucket path here where the training images will live e.g. "gs://foo/bar"
   upload_labels_to_label_studio:
     enable_cache: False
     parameters:

end-to-end-computer-vision/configs/training.yaml → end-to-end-computer-vision/configs/training_pipeline.yaml

@@ -7,10 +7,10 @@ steps:
   train_model:
     enable_cache: False
     parameters:
-      data_source: "gs://zenml-internal-artifact-store/label_studio_cv_webinar"
-      batch_size: 16
-      imgsz: 640
-      epochs: 30
+      data_source: # Insert your bucket path here where the training images lives e.g. "gs://foo/bar"
+      batch_size: 8
+      imgsz: 720
+      epochs: 1
 
 settings:
   docker:

end-to-end-computer-vision/configs/training_gpu.yaml → end-to-end-computer-vision/configs/training_pipeline_remote_gpu.yaml

@@ -21,14 +21,14 @@ steps:
     step_operator: gcp_a100
     enable_step_logs: False
     parameters:
-      data_source: "gs://zenml-internal-artifact-store/label_studio_cv_webinar"
+      data_source:  # Insert your bucket path here where the training images lives e.g. "gs://foo/bar"
       batch_size: 8
-      imgsz: 480
-      epochs: 5000
+      imgsz: 720
+      epochs: 50000
       is_quad_gpu_env: True
     settings:
       step_operator.vertex:
-        accelerator_type: NVIDIA_TESLA_T4 # NVIDIA_TESLA_A100 # see https://cloud.google.com/vertex-ai/docs/reference/rest/v1/MachineSpec#AcceleratorType
+        accelerator_type: NVIDIA_TESLA_T4 # see https://cloud.google.com/vertex-ai/docs/reference/rest/v1/MachineSpec#AcceleratorType
         accelerator_count: 4
         disk_size_gb: 25
       docker:

end-to-end-computer-vision/data/.gitignore

@@ -0,0 +1,4 @@
+# Ignore everything in this directory
+*
+# Except this file
+!.gitignore

end-to-end-computer-vision/data/README.md

@@ -0,0 +1 @@
+This directory serves as a place to store and access temporary datafiles.