diff --git a/INSTALLATION.md b/INSTALLATION.md new file mode 100644 index 00000000..2fbb9337 --- /dev/null +++ b/INSTALLATION.md @@ -0,0 +1,54 @@ +## Installation + +### Prerequisites + +- Python `3.8` | `3.9` | `3.10` +- PyTorch `1.13.0` (recommended) (compatible with: `1.11.x` - `1.13.x`) +- TensorFlow `2.8.0` (recommended) (compatible with: `2.3.x` - `2.8.x`) + +### Install with PyPI (stable) + +```bash +pip install netspresso +``` + +### Install with GitHub + +To install with editable mode, + +```bash +git clone https://github.com/nota-netspresso/pynetspresso.git +cd pynetspresso +pip install -e . +``` + +### Docker with docker-compose + +For the latest information, please check `docker-compose.yml` + +```python +# run command +export TAG=v$(cat netspresso/VERSION) && \ +docker compose run --service-ports --name netspresso-dev netspresso bash +``` + +### Docker image build + +If you run with `docker run` command, follow the image build and run command in the below: + +```python +# build an image +docker build -t netspresso:v$(cat netspresso/VERSION) . +``` + +```python +# docker run command +docker run -it --ipc=host\ + --gpus='"device=0,1,2,3"'\ + -v /PATH/TO/DATA:/DATA/PATH/IN/CONTAINER\ + -v /PATH/TO/CHECKPOINT:/CHECKPOINT/PATH/IN/CONTAINER\ + -p 50001:50001\ + -p 50002:50002\ + -p 50003:50003\ + --name netspresso-dev netspresso:v$(cat netspresso/VERSION) +``` diff --git a/README.md b/README.md index 3f9e4725..b7c4a970 100644 --- a/README.md +++ b/README.md @@ -2,14 +2,13 @@ -# PyNetsPresso -

+ ☀️ NetsPresso Model Zoo ☀️
YOLO Fastest | YOLOX | YOLOv8 @@ -20,11 +19,17 @@

+
+ 🌟 STMicro x NetsPresso 🌟
+ STM32 model zoo +
+
+

- +
@@ -94,85 +99,111 @@ To get started with the PyNetsPresso, you will need to sign up either at [NetsPr

- - - - - - - - - - - - - - - - - - - - - -
StepsTypesDescription
- Train -
- (Model Zoo) - 		-
- Image Classification - PyTorch-CIFAR-Models -
-
- Object Detection - YOLO Fastest
- YOLOX
- YOLOv5
- YOLOv7 -
-
- Semantic Segmentation - PIDNet -
-
- Pose Estimation - YOLOv8 -
- 		Build and train models.
Compressnp.compressorCompress and optimize the user’s pre-trained model.
Convertnp.launcherConvert AI models to run efficiently on the desired hardware and provide easy installation for seamless usage of the converted AI models.
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
StepTypeDescription
Trainnp.trainerBuild and train a model.
+
+ Model Zoo +
+ Image Classification + PyTorch-CIFAR-Models +
+
+ Object Detection + YOLO Fastest
+ YOLOX
+ YOLOv5
+ YOLOv7 +
+
+ Semantic Segmentation + PIDNet +
+
+ Pose Estimation + YOLOv8 +
+
+
Compressnp.compressorCompress and optimize the user’s model.
Convertnp.converterConvert and quantize the user’s model to run efficiently on your device.
Benchmarknp.benchmarkerBenchmark the user's model to measure model inference speed on your device.
- ## Installation -There are two ways you can install the PyNetsPresso: using pip or manually through our project GitHub repository. +### Prerequisites + +- Python `3.8` | `3.9` | `3.10` +- PyTorch `1.13.0` (recommended) (compatible with: `1.11.x` - `1.13.x`) +- TensorFlow `2.8.0` (recommended) (compatible with: `2.3.x` - `2.8.x`) -To install this package, please use Python 3.8 or higher. +### Install with PyPI (stable) -From PyPI (Recommended) ```bash pip install netspresso ``` -From Github -```bash -git clone https://github.com/nota-netspresso/pynetspresso.git -cd pynetspresso -pip install -e . +To use **editable mode** or **docker**, see [INSTALLATION.md](INSTALLATION.md). + +## Getting started + +### Login + +To use the PyNetsPresso, please enter the email and password registered in [NetsPresso]. + +```python +from netspresso import NetsPresso + +netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD") ``` +### Trainer -## Quick Start +#### Train -### ⭐⭐⭐ (New Feature) Train ⭐⭐⭐ +To start training a model, first select a task. + +Then configure the dataset, model specifications, augmentation, and hyperparameters. + +Once setup is finished, enter the GPU number and project name for training. ```python -from loguru import logger -from netspresso.trainer import Trainer, Task +from netspresso.enums import Task +from netspresso.trainer.optimizers import AdamW +from netspresso.trainer.schedulers import CosineAnnealingWarmRestartsWithCustomWarmUp +from netspresso.trainer.augmentations import Resize + -trainer = Trainer(task=Task.OBJECT_DETECTION) -logger.info(trainer.available_models) # ['EfficientFormer', 'YOLOX'] +# 1. Declare trainer +trainer = netspresso.trainer(task=Task.OBJECT_DETECTION) # IMAGE_CLASSIFICATION, OBJECT_DETECTION, SEMANTIC_SEGMENTATION +# 2. Set config for training +# 2-1. Data trainer.set_dataset_config( name="traffic_sign_config_example", root_path="/root/traffic-sign", @@ -182,134 +213,248 @@ trainer.set_dataset_config( valid_label="labels/valid", id_mapping=["prohibitory", "danger", "mandatory", "other"], ) -trainer.set_model_config(model_name="YOLOX") -trainer.set_training_config(epochs=40, batch_size=16, lr=6e-3, opt="adamw", warmup_epochs=10) -trainer.train(gpus="0, 1") -``` - -```python -from netspresso.trainer import Trainer, Task +# 2-2. Model +print(trainer.available_models) # ['EfficientFormer', 'YOLOX-S', 'ResNet', 'MobileNetV3', 'MixNetL', 'MixNetM', 'MixNetS'] +trainer.set_model_config(model_name="YOLOX-S", img_size=512) -trainer = Trainer(task=Task.IMAGE_CLASSIFICATION) +# 2-3. Augmentation +trainer.set_augmentation_config( + train_transforms=[Resize()], + inference_transforms=[Resize()], +) -trainer.set_dataset_config_with_yaml(yaml_path="config/data/beans.yaml") -trainer.set_model_config_with_yaml(yaml_path="config/model/resnet50-classification.yaml") +# 2-4. Training +optimizer = AdamW(lr=6e-3) +scheduler = CosineAnnealingWarmRestartsWithCustomWarmUp(warmup_epochs=10) +trainer.set_training_config( + epochs=40, + batch_size=16, + optimizer=optimizer, + scheduler=scheduler, +) -trainer.train(gpus="0, 1") +# 3. Train +trainer.train(gpus="0, 1", project_name="PROJECT_TRAIN_SAMPLE") ``` -#### Download config folder from netspresso-trainer - -If you want to train the trainer as a yaml file, download the config folder and use it. +#### Retrain -```bash -python tools/github_download.py --repo Nota-NetsPresso/netspresso-trainer --path config -``` +To start retraining a model, use `hparam.yaml` that the original model was trained on. -### Login +Then, enter the compressed model path. -To use the PyNetsPresso, please enter the email and password registered in NetsPresso. +Adjust the training hyperparameters as needed. (See 2-2. for detailed code.) ```python -from netspresso.client import SessionClient +from netspresso.trainer.optimizers import AdamW + +# 1. Declare trainer +trainer = netspresso.trainer(yaml_path="./temp/hparams.yaml") + +# 2. Set config for retraining +# 2-1. FX Model +trainer.set_fx_model(fx_model_path="./temp/FX_MODEL_PATH.pt") + +# 2-2. Training +optimizer = AdamW(lr=6e-3) +trainer.set_training_config( + epochs=30, + batch_size=16, + optimizer=optimizer, +) -session = SessionClient(email='YOUR_EMAIL', password='YOUR_PASSWORD') +# 3. Train +trainer.train(gpus="0, 1", project_name="PROJECT_RETRAIN_SAMPLE") ``` -### Automatic Compression +### Compressor -Automatically compress the model by setting the compression ratio for the model. +#### Compress (Automatic compression) -Enter the ID of the uploaded model, the name and storage path of the compressed model, and the compression ratio. +To start compressing a model, enter the model path to compress and the appropriate compression ratio. + +The compressed model will be saved in the specified output directory (`output_dir`). ```python -from netspresso.compressor import Compressor +# 1. Declare compressor +compressor = netspresso.compressor() -compressor = Compressor(user_session=session) +# 2. Run automatic compression compressed_model = compressor.automatic_compression( - model_name="YOUR_MODEL_NAME", - task=Task.IMAGE_CLASSIFICATION, - framework=Framework.TENSORFLOW_KERAS, - input_shapes="YOUR_MODEL_INPUT_SHAPES", # ex) [{"batch": 1, "channel": 3, "dimension": [32, 32]}] - input_path="YOUR_MODEL_PATH", # ex) "./examples/sample_models/mobilenetv1.h5" - output_path="OUTPUT_PATH", # ex) ./outputs/compressed/compressed_model.h5, + input_shapes=[{"batch": 1, "channel": 3, "dimension": [224, 224]}], + input_model_path="./examples/sample_models/graphmodule.pt", + output_dir="./outputs/compressed/pytorch_automatic_compression", compression_ratio=0.5, ) ``` -### Convert Model and Benchmark the Converted Model -Convert an ONNX model into a TensorRT model, and benchmark the TensorRT model on the Jetson Nano. +### Converter + +#### Convert + +To start converting a model, enter the model path to convert and the target framework and device name. + +For NVIDIA GPUs and Jetson devices, enter the software version additionally due to the jetpack version. + +The converted model will be saved in the specified output directory (`output_dir`). ```python -from loguru import logger -from netspresso.launcher import Converter, Benchmarker, ModelFramework, DeviceName, SoftwareVersion +from netspresso.enums import DeviceName, Framework, SoftwareVersion -converter = Converter(user_session=session) -conversion_task = converter.convert_model( - model_path="YOUR_MODEL_PATH", # ex) "./examples/sample_models/test.onnx" - target_framework=ModelFramework.TENSORRT, - target_device_name=DeviceName.JETSON_AGX_ORIN, - target_software_version=SoftwareVersion.JETPACK_5_0_1, - output_path="CONVERTED_MODEL_PATH" # ex) "./outputs/converted/converted_model.trt" -) -logger.info(conversion_task) +# 1. Declare converter +converter = netspresso.converter() -benchmarker = Benchmarker(user_session=session) -benchmark_task = benchmarker.benchmark_model( - model_path="CONVERTED_MODEL_PATH", # ex) "./outputs/converted/converted_model.trt" +# 2. Run convert +conversion_task = converter.convert_model( + input_model_path="./examples/sample_models/test.onnx", + output_dir="./outputs/converted/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1", + target_framework=Framework.TENSORRT, target_device_name=DeviceName.JETSON_AGX_ORIN, target_software_version=SoftwareVersion.JETPACK_5_0_1, ) -logger.info(f"model inference latency: {benchmark_task.latency} ms") -logger.info(f"model gpu memory footprint: {benchmark_task.memory_footprint_gpu} MB") -logger.info(f"model cpu memory footprint: {benchmark_task.memory_footprint_cpu} MB") ``` -## Available Options for Launcher (Convert, Benchmark) - -### Available Target Frameworks for Conversion with Source Models - -| Target / Source Model | ONNX | TENSORFLOW_KERAS | TENSORFLOW | -| :-------------------- | :--: | :--------------: | :--------: | -| TENSORRT | ✔️ | | | -| DRPAI | ✔️ | | | -| OPENVINO | ✔️ | | | -| TENSORFLOW_LITE | ✔️ | ✔️ | ✔️ | +### Benchmarker +#### Benchmark -### Available Devices for Framework +To start benchmarking a model, enter the model path to benchmark and the target device name. -| Device / Framework | ONNX | TENSORRT | TENSORFLOW_LITE | DRPAI | OPENVINO | -| :------------------- | :--: | :------: | :-------------: | :---: | :------: | -| RASPBERRY_PI_4B | ✔️ | | ✔️ | | | -| RASPBERRY_PI_3B_PLUS | ✔️ | | ✔️ | | | -| RASPBERRY_PI_ZERO_W | ✔️ | | ✔️ | | | -| RASPBERRY_PI_ZERO_2W | ✔️ | | ✔️ | | | -| RENESAS_RZ_V2L | ✔️ | | | ✔️ | | -| RENESAS_RZ_V2M | ✔️ | | | ✔️ | | -| RENESAS_RA8D1 | | | ✔️(only INT8) | | | -| ALIF_ENSEMBLE_E7_DEVKIT_GEN2 | | | ✔️(only INT8) | | | -| JETSON_NANO | ✔️ | ✔️ | | | | -| JETSON_TX2 | ✔️ | ✔️ | | | | -| JETSON_XAVIER | ✔️ | ✔️ | | | | -| JETSON_NX | ✔️ | ✔️ | | | | -| JETSON_AGX_ORIN | ✔️ | ✔️ | | | | -| AWS_T4 | ✔️ | ✔️ | | | | -| Intel_XEON_W_2233 | | | | | ✔️ | +For NVIDIA GPUs and Jetson devices, device name and software version have to be matched with the target device of the conversion. +TensorRT Model has strong dependency with the device type and its jetpack version. -### Available Software Versions for Jetson Devices +```python +from netspresso.enums import DeviceName, SoftwareVersion -Software Versions requires only Jetson Device. If you are using a different device, you do not need to enter it. +# 1. Declare benchmarker +benchmarker = netspresso.benchmarker() -| Software Version / Device | JETSON_NANO | JETSON_TX2 | JETSON_XAVIER | JETSON_NX | JETSON_AGX_ORIN | -| :------------------------ | :---------: | :--------: | :-----------: | :-------: | :-------------: | -| JETPACK_4_4_1 | ✔️ | | | | | -| JETPACK_4_6 | ✔️ | ✔️ | ✔️ | ✔️ | | -| JETPACK_5_0_1 | | | | | ✔️ | -| JETPACK_5_0_2 | | | | ✔️ | | +# 2. Run benchmark +benchmark_task = benchmarker.benchmark_model( + input_model_path="./outputs/converted/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1.trt", + target_device_name=DeviceName.JETSON_AGX_ORIN, + target_software_version=SoftwareVersion.JETPACK_5_0_1, +) +print(f"model inference latency: {benchmark_task['result']['latency']} ms") +print(f"model gpu memory footprint: {benchmark_task['result']['memory_footprint_gpu']} MB") +print(f"model cpu memory footprint: {benchmark_task['result']['memory_footprint_cpu']} MB") +``` +
+ Supported options for Converter & Benchmarker +
+ + ### Frameworks that support conversion for model's framework + + | Target / Source Framework | ONNX | TENSORFLOW_KERAS | TENSORFLOW | + |:--------------------------|:----:|:----------------:|:----------:| + | TENSORRT | ✔️ | | | + | DRPAI | ✔️ | | | + | OPENVINO | ✔️ | | | + | TENSORFLOW_LITE | ✔️ | ✔️ | ✔️ | + + ### Devices that support benchmarks for model's framework + + | Device / Framework | ONNX | TENSORRT | TENSORFLOW_LITE | DRPAI | OPENVINO | + |:-----------------------------|:----:|:--------:|:---------------:|:-----:|:--------:| + | RASPBERRY_PI_5 | ✔️ | | ✔️ | | | + | RASPBERRY_PI_4B | ✔️ | | ✔️ | | | + | RASPBERRY_PI_3B_PLUS | ✔️ | | ✔️ | | | + | RASPBERRY_PI_ZERO_W | ✔️ | | ✔️ | | | + | RASPBERRY_PI_ZERO_2W | ✔️ | | ✔️ | | | + | ARM_ETHOS_U_SERIES | | | ✔️(only INT8) | | | + | ALIF_ENSEMBLE_E7_DEVKIT_GEN2 | | | ✔️(only INT8) | | | + | RENESAS_RA8D1 | | | ✔️(only INT8) | | | + | RENESAS_RZ_V2L | ✔️ | | | ✔️ | | + | RENESAS_RZ_V2M | ✔️ | | | ✔️ | | + | JETSON_NANO | ✔️ | ✔️ | | | | + | JETSON_TX2 | ✔️ | ✔️ | | | | + | JETSON_XAVIER | ✔️ | ✔️ | | | | + | JETSON_NX | ✔️ | ✔️ | | | | + | JETSON_AGX_ORIN | ✔️ | ✔️ | | | | + | AWS_T4 | ✔️ | ✔️ | | | | + | INTEL_XEON_W_2233 | | | | | ✔️ | + + ### Software versions that support conversions and benchmarks for specific devices + + Software Versions requires only Jetson Device. If you are using a different device, you do not need to enter it. + + | Software Version / Device | JETSON_NANO | JETSON_TX2 | JETSON_XAVIER | JETSON_NX | JETSON_AGX_ORIN | + |:--------------------------|:-----------:|:----------:|:-------------:|:---------:|:---------------:| + | JETPACK_4_4_1 | ✔️ | | | | | + | JETPACK_4_6 | ✔️ | ✔️ | ✔️ | ✔️ | | + | JETPACK_5_0_1 | | | | | ✔️ | + | JETPACK_5_0_2 | | | | ✔️ | | + + The code below is an example of using software version. + + ```python + conversion_result = converter.convert_model( + input_model_path=INPUT_MODEL_PATH, + output_dir=OUTPUT_DIR, + target_framework=Framework.TENSORRT, + target_device_name=DeviceName.JETSON_AGX_ORIN, + target_software_version=SoftwareVersion.JETPACK_5_0_1, + ) + benchmark_result = benchmarker.benchmark_model( + input_model_path=CONVERTED_MODEL_PATH, + target_device_name=DeviceName.JETSON_AGX_ORIN, + target_software_version=SoftwareVersion.JETPACK_5_0_1, + ) + ``` + + ### Hardware type that support benchmarks for specific devices + + Benchmark and compare models with and without Arm Helium. + + `RENESAS_RA8D1` and `ALIF_ENSEMBLE_E7_DEVKIT_GEN2` are available for use. + + The benchmark results with Helium can be up to twice as fast as without Helium. + + The code below is an example of using hardware type. + + ```python + benchmark_result = benchmarker.benchmark_model( + input_model_path=CONVERTED_MODEL_PATH, + target_device_name=DeviceName.RENESAS_RA8D1, + target_data_type=DataType.INT8, + target_hardware_type=HardwareType.HELIUM + ) + ``` +
+
+ + +## Guide to Credit Consumption by Module + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModuleFeatureCredit
CompressorAutomatic compression25
Advanced compression50
ConverterConvert50
BenchmarkerBenchmark25
## NetsPresso Model Compressor Best Practice diff --git a/SUPPORT_OPTIONS.md b/SUPPORT_OPTIONS.md new file mode 100644 index 00000000..c480c0f2 --- /dev/null +++ b/SUPPORT_OPTIONS.md @@ -0,0 +1,79 @@ +## Supported options for Converter & Benchmarker + +### Frameworks that support conversion for model's framework + +| Target / Source Framework | ONNX | TENSORFLOW_KERAS | TENSORFLOW | +|:--------------------------|:----:|:----------------:|:----------:| +| TENSORRT | ✔️ | | | +| DRPAI | ✔️ | | | +| OPENVINO | ✔️ | | | +| TENSORFLOW_LITE | ✔️ | ✔️ | ✔️ | + +### Devices that support benchmarks for model's framework + +| Device / Framework | ONNX | TENSORRT | TENSORFLOW_LITE | DRPAI | OPENVINO | +|:-----------------------------|:----:|:--------:|:---------------:|:-----:|:--------:| +| RASPBERRY_PI_5 | ✔️ | | ✔️ | | | +| RASPBERRY_PI_4B | ✔️ | | ✔️ | | | +| RASPBERRY_PI_3B_PLUS | ✔️ | | ✔️ | | | +| RASPBERRY_PI_ZERO_W | ✔️ | | ✔️ | | | +| RASPBERRY_PI_ZERO_2W | ✔️ | | ✔️ | | | +| ARM_ETHOS_U_SERIES | | | ✔️(only INT8) | | | +| ALIF_ENSEMBLE_E7_DEVKIT_GEN2 | | | ✔️(only INT8) | | | +| RENESAS_RA8D1 | | | ✔️(only INT8) | | | +| RENESAS_RZ_V2L | ✔️ | | | ✔️ | | +| RENESAS_RZ_V2M | ✔️ | | | ✔️ | | +| JETSON_NANO | ✔️ | ✔️ | | | | +| JETSON_TX2 | ✔️ | ✔️ | | | | +| JETSON_XAVIER | ✔️ | ✔️ | | | | +| JETSON_NX | ✔️ | ✔️ | | | | +| JETSON_AGX_ORIN | ✔️ | ✔️ | | | | +| AWS_T4 | ✔️ | ✔️ | | | | +| INTEL_XEON_W_2233 | | | | | ✔️ | + +### Software versions that support conversions and benchmarks for specific devices + +Software Versions requires only Jetson Device. If you are using a different device, you do not need to enter it. + +| Software Version / Device | JETSON_NANO | JETSON_TX2 | JETSON_XAVIER | JETSON_NX | JETSON_AGX_ORIN | +|:--------------------------|:-----------:|:----------:|:-------------:|:---------:|:---------------:| +| JETPACK_4_4_1 | ✔️ | | | | | +| JETPACK_4_6 | ✔️ | ✔️ | ✔️ | ✔️ | | +| JETPACK_5_0_1 | | | | | ✔️ | +| JETPACK_5_0_2 | | | | ✔️ | | + +The code below is an example of using software version. + +```python +conversion_result = converter.convert_model( + input_model_path=INPUT_MODEL_PATH, + output_dir=OUTPUT_DIR, + target_framework=Framework.TENSORRT, + target_device_name=DeviceName.JETSON_AGX_ORIN, + target_software_version=SoftwareVersion.JETPACK_5_0_1, +) +benchmark_result = benchmarker.benchmark_model( + input_model_path=CONVERTED_MODEL_PATH, + target_device_name=DeviceName.JETSON_AGX_ORIN, + target_software_version=SoftwareVersion.JETPACK_5_0_1, +) +``` + +### Hardware type that support benchmarks for specific devices + +Benchmark and compare models with and without Arm Helium. + +`RENESAS_RA8D1` and `ALIF_ENSEMBLE_E7_DEVKIT_GEN2` are available for use. + +The benchmark results with Helium can be up to twice as fast as without Helium. + +The code below is an example of using hardware type. + +```python +benchmark_result = benchmarker.benchmark_model( + input_model_path=CONVERTED_MODEL_PATH, + target_device_name=DeviceName.RENESAS_RA8D1, + target_data_type=DataType.INT8, + target_hardware_type=HardwareType.HELIUM +) +``` \ No newline at end of file