Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send the user a message in case deployment fails on how to trouble shoot and possible fixes #85

Open
anirudTT opened this issue Nov 20, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@anirudTT
Copy link
Contributor

nternal Server Error: /docker/deploy/
tt_studio_backend_api | Traceback (most recent call last):
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/api/client.py", line 265, in _raise_for_status
tt_studio_backend_api | response.raise_for_status()
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status
tt_studio_backend_api | raise HTTPError(http_error_msg, response=self)
tt_studio_backend_api | requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.47/containers/create?name=dummy_echo_model_p8001
tt_studio_backend_api |
tt_studio_backend_api | The above exception was the direct cause of the following exception:
tt_studio_backend_api |
tt_studio_backend_api | Traceback (most recent call last):
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/models/containers.py", line 873, in run
tt_studio_backend_api | container = self.create(image=image, command=command,
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/models/containers.py", line 932, in create
tt_studio_backend_api | resp = self.client.api.create_container(**create_kwargs)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/api/container.py", line 439, in create_container
tt_studio_backend_api | return self.create_container_from_config(config, name, platform)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/api/container.py", line 456, in create_container_from_config
tt_studio_backend_api | return self._result(res, True)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/api/client.py", line 271, in _result
tt_studio_backend_api | self._raise_for_status(response)
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/api/client.py", line 267, in _raise_for_status
tt_studio_backend_api | raise create_api_error_from_http_exception(e) from e
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
tt_studio_backend_api | raise cls(e, response=response, explanation=explanation) from e
tt_studio_backend_api | docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.47/containers/create?name=dummy_echo_model_p8001: Not Found ("No such image: dummy_echo_model:v0.0.1")
tt_studio_backend_api |
tt_studio_backend_api | During handling of the above exception, another exception occurred:
tt_studio_backend_api |
tt_studio_backend_api | Traceback (most recent call last):
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/api/client.py", line 265, in _raise_for_status
tt_studio_backend_api | response.raise_for_status()
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status
tt_studio_backend_api | raise HTTPError(http_error_msg, response=self)
tt_studio_backend_api | requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.47/images/create?tag=v0.0.1&fromImage=dummy_echo_model
tt_studio_backend_api |
tt_studio_backend_api | The above exception was the direct cause of the following exception:
tt_studio_backend_api |
tt_studio_backend_api | Traceback (most recent call last):
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
tt_studio_backend_api | response = get_response(request)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/django/core/handlers/base.py", line 197, in _get_response
tt_studio_backend_api | response = wrapped_callback(request, *callback_args, **callback_kwargs)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
tt_studio_backend_api | return view_func(request, *args, **kwargs)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/django/views/generic/base.py", line 104, in view
tt_studio_backend_api | return self.dispatch(request, *args, **kwargs)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/rest_framework/views.py", line 509, in dispatch
tt_studio_backend_api | response = self.handle_exception(exc)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/rest_framework/views.py", line 469, in handle_exception
tt_studio_backend_api | self.raise_uncaught_exception(exc)
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
tt_studio_backend_api | raise exc
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/rest_framework/views.py", line 506, in dispatch
tt_studio_backend_api | response = handler(request, *args, **kwargs)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/api/docker_control/views.py", line 102, in post
tt_studio_backend_api | response = run_container(impl, weights_id)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/api/docker_control/docker_utils.py", line 53, in run_container
tt_studio_backend_api | container = client.containers.run(impl.image_version, **run_kwargs)
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/models/containers.py", line 876, in run
tt_studio_backend_api | self.client.images.pull(image, platform=platform)
tt_studio_frontend | Received Response from the Target: 500 /docker/deploy/
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/models/images.py", line 464, in pull
tt_studio_backend_api | pull_log = self.client.api.pull(
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/api/image.py", line 429, in pull
tt_studio_backend_api | self._raise_for_status(response)
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/api/client.py", line 267, in _raise_for_status
tt_studio_backend_api | raise create_api_error_from_http_exception(e) from e
tt_studio_backend_api | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tt_studio_backend_api | File "/usr/local/lib/python3.12/site-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
tt_studio_backend_api | raise cls(e, response=response, explanation=explanation) from e
tt_studio_backend_api | docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.47/images/create?tag=v0.0.1&fromImage=dummy_echo_model: Not Found ("pull access denied for dummy_echo_model, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

@tstescoTT
Copy link
Contributor

tstescoTT commented Dec 4, 2024

We should capture error stack traces and add them to the logs, then logs should be surfaced to user on failure. e.g. a link to the log file viewer at the correct log file in the GUI.

This would cover the case where the port is occupied by another container that is not managed by the tt-studio backend.

@tstescoTT tstescoTT added the enhancement New feature or request label Dec 4, 2024
@anirudTT
Copy link
Contributor Author

This issue should be tagged along with this #98 to correctly show the users logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants