Dummy Model Creation

[JoeBatt1989]

Descripton

Create Models of different sizes and configurations so that we can fully test the MWM functionality and performance. There was a mean pixel calc which was used for testing DICOM inout and output in legacy AIDE. The limitation on this model was that the execution was so quick that it did not stress the system. The new models must not have this same limitation.

Acceptance Criteria

1. Models must be configurable to use variable resources. Example resources have been set out in the below table. This is subject to change so must be easily configurable:

Name	CPU Cores	RAM	GPU	Disk Space
Dummy (mean pixel calc)
Small	2	1GB	1GB	2GB
Medium	8	10GB	6GB	15GB
Large	12	16GB	12GB	25GB

2. Must be executed via an Argo workflow
3. Must use system resources (Processing time must equate to the size of the model - randomise)
4. Must be able to create metadata

Original ticket from @laurencejackson - Project-MONAI/monai-deploy#63

[lillie-dae]

I've been looking into Stress testing argo.

Theres a stress image here:

https://hub.docker.com/r/progrium/stress/

i created a template in argo to use it and heres an example monai workflow using that template.

Monai workflow template

{
	"name": "stres workflow",
	"version": "1.0.0",
	"description": "Attempt at making a workflow",
	"informatics_gateway": {
		"ae_title": "MonaiSCU",
		"data_origins": [
			"MY_SCANNER"
		],
		"export_destinations": [
			"PROD_PACS"
		]
	},
	"tasks": [
		{
			"id": "argo-task",
			"description": "trigger simple argo workflow",
			"type": "argo",
			"args": {
				"namespace":"argo",
				"workflow_template_name": "delightful-bear",
				"server_url": "https://localhost:2746",
				"allow_insecure": true,
				"messaging_endpoint": "rabbit-monai",
                "messaging_username" : "admin",
                "messaging_password" : "admin",
                "messaging_topic" : "md.tasks.callback",
                "messaging_exchange" : "monaideploy",
				"messaging_vhost" : "monaideploy",
                "parameters": "{ \"cpu\":\"2\", \"io\":\"2\", \"vm\":\"2\", \"timeout\":\"15s\", \"vmbytes\": \"128M\" }"
			},
			"artifacts": {
				"input": [
					{
						"name": "input_diacom",
						"value": "{{ context.input.dicom }}/input_dicom"
					}
				],
				"output": [
					{
						"name": "report",
						"value": "{{ context.output }}/report",
						"Mandatory": false
					}
				]
			}
		}
	]
}

Argo workflow template

{
  "metadata": {
    "name": "delightful-bear",
    "namespace": "argo",
    "uid": "33721971-6ce8-45ff-ad2e-d468138d8d02",
    "resourceVersion": "1784231",
    "generation": 14,
    "creationTimestamp": "2022-08-30T14:22:05Z",
    "labels": {
      "example": "true",
      "workflows.argoproj.io/creator": "system-serviceaccount-argo-argo-server"
    },
    "managedFields": [
      {
        "manager": "argo",
        "operation": "Update",
        "apiVersion": "argoproj.io/v1alpha1",
        "time": "2022-08-30T14:22:05Z",
        "fieldsType": "FieldsV1",
        "fieldsV1": {
          "f:metadata": {
            "f:labels": {
              ".": {},
              "f:example": {},
              "f:workflows.argoproj.io/creator": {}
            }
          },
          "f:spec": {}
        }
      }
    ]
  },
  "spec": {
    "templates": [
      {
        "name": "stress",
        "inputs": {
          "parameters": [
            {
              "name": "cpu",
              "value": "{{workflow.parameters.cpu}}"
            },
            {
              "name": "io",
              "value": "{{workflow.parameters.io}}"
            },
            {
              "name": "vm",
              "value": "{{workflow.parameters.vm}}"
            },
            {
              "name": "vmbytes",
              "value": "{{workflow.parameters.vmbytes}}"
            },
            {
              "name": "timeout",
              "value": "{{workflow.parameters.timeout}}"
            }
          ]
        },
        "outputs": {},
        "metadata": {},
        "container": {
          "name": "main",
          "image": "progrium/stress",
          "command": [
            "stress"
          ],
          "args": [
            "--cpu",
            "{{inputs.parameters.cpu}}",
            "--io",
            "{{inputs.parameters.io}}",
            "--vm-bytes",
            "{{inputs.parameters.vmbytes}}",
            "--vm",
            "{{inputs.parameters.vm}}",
            "--timeout",
            "{{inputs.parameters.timeout}}"
          ],
          "resources": {}
        }
      }
    ],
    "entrypoint": "stress",
    "arguments": {
      "parameters": [
        {
          "name": "cpu",
          "value": "1"
        },
        {
          "name": "io",
          "value": "1"
        },
        {
          "name": "vm",
          "value": "1"
        },
        {
          "name": "timeout",
          "value": "15s"
        },
        {
          "name": "vmbytes",
          "value": "64M"
        }
      ]
    },
    "ttlStrategy": {
      "secondsAfterCompletion": 300
    },
    "podGC": {
      "strategy": "OnPodCompletion"
    },
    "workflowMetadata": {
      "labels": {
        "example": "true"
      }
    }
  }
}

[woodheadio]

@lillie-dae created a composite Docker image containing both the stress tool mentioned above and a GPU Burn tool. I will migrate the DockerFile for this into the Performance Testing area of the MONAI deploy repo.