Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Active Stack as part of configuration YAML #2370

Closed
wants to merge 38 commits into from
Closed
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
60af95f
add active_stack to run_config
kabinja Jan 29, 2024
ac820e5
remove test generated file
kabinja Jan 29, 2024
9f76559
fix typos
kabinja Jan 29, 2024
fd1f482
fix typo
kabinja Jan 29, 2024
63bc189
Merge branch 'develop' into feature/config-active-stack
strickvl Jan 31, 2024
a98112c
Update src/zenml/new/pipelines/pipeline.py
kabinja Feb 2, 2024
c2d7774
Update src/zenml/stack/utils.py
kabinja Feb 2, 2024
8242efb
fix doc string for __exit__ method of stack_context
kabinja Feb 4, 2024
f075c4a
Merge branch 'develop' into feature/config-active-stack
kabinja Feb 4, 2024
fbe12e0
fix linting
kabinja Feb 4, 2024
80a578d
add documentation
kabinja Feb 4, 2024
120885a
Merge branch 'develop' into feature/config-active-stack
strickvl Feb 5, 2024
001a031
Update docs/book/user-guide/advanced-guide/pipelining-features/config…
kabinja Feb 5, 2024
323d33c
Update docs/book/user-guide/advanced-guide/pipelining-features/config…
kabinja Feb 5, 2024
8959f07
Update src/zenml/stack/utils.py
kabinja Feb 5, 2024
dfc3115
Update src/zenml/stack/utils.py
kabinja Feb 5, 2024
f141fed
update TOC (#2406)
strickvl Feb 6, 2024
2cdcfac
fix docstring indentation
kabinja Feb 6, 2024
590b074
Fix GCP service connector login to overwrite existing valid credentia…
stefannica Feb 5, 2024
b64fae9
Update `has_custom_name` for legacy artifacts (#2384)
avishniakov Feb 5, 2024
ce9a34f
Use native VertexAI scheduler capability instead of old GCP official …
francoisserra Feb 6, 2024
c1abd87
HyperAI integration: orchestrator and service connector (#2372)
christianversloot Feb 6, 2024
a0a12e8
Let contributors shield link to ZenML contributors (#2400)
christianversloot Feb 6, 2024
479b292
Prepare release 0.55.2 (#2401)
bcdurak Feb 6, 2024
e51297c
update TOC (#2406)
strickvl Feb 6, 2024
1022574
Merge branch 'develop' into feature/config-active-stack
kabinja Feb 6, 2024
855c96b
Merge branch 'develop' into feature/config-active-stack
strickvl Feb 7, 2024
4c284fe
ubuntu-dind-runners for release workflow
strickvl Feb 20, 2024
9865043
Merge branch 'zenml-io:main' into feature/config-active-stack
kabinja Feb 20, 2024
c34b389
fix bug comparing id in exit method
kabinja Feb 20, 2024
e27df0e
Merge branch 'develop' into feature/config-active-stack
strickvl Feb 23, 2024
876a9fb
rename active_stack to stack in pipeline run configuration
kabinja Feb 26, 2024
d07e90c
replace stack_context to use temporary_active_stack
kabinja Feb 26, 2024
5126923
Merge branch 'develop' into feature/config-active-stack
strickvl Feb 28, 2024
226791c
Merge branch 'develop' into feature/config-active-stack
strickvl Mar 13, 2024
7e72e00
Merge branch 'develop' into feature/config-active-stack
bcdurak Mar 13, 2024
951ea5b
temporary_active_stack
kabinja Mar 24, 2024
597ffbb
Merge branch 'develop' into feature/config-active-stack
strickvl Mar 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ python run.py
<summary>An example of a generated YAML configuration template</summary>

```yaml
active_stack: Optional[str]
build: Union[PipelineBuildBase, UUID, NoneType]
enable_artifact_metadata: Optional[bool]
enable_artifact_visualization: Optional[bool]
Expand Down Expand Up @@ -330,6 +331,12 @@ These are boolean flags for various configurations:
* `enable_cache`: Utilize [caching](../../starter-guide/cache-previous-executions.md) or not.
* `enable_step_logs`: Enable tracking [step logs](managing-steps.md#enable-or-disable-logs-storing).

### `active_stack` name or ID

The name of the UUID of the `active stack` to use for this
kabinja marked this conversation as resolved.
Show resolved Hide resolved
pipeline. If specified the active stack is set for the duration of the pipeline execution and restored upon
kabinja marked this conversation as resolved.
Show resolved Hide resolved
completion. If not specified, the current active stack is used.

### `build` ID

The UUID of the [`build`](../infrastructure-management/containerize-your-pipeline.md) to use for this pipeline. If specified, Docker image building is skipped for remote orchestrators, and the Docker image specified in this build is used.
Expand Down
2 changes: 1 addition & 1 deletion src/zenml/new/pipelines/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -1580,7 +1580,7 @@ def _prepare_if_possible(self) -> None:
def _update_stack_from_config(
self, run_configuration: PipelineRunConfiguration
) -> None:
"""Activate the stack from the pipeline run configuation if one is given.
"""Activate the stack from the pipeline run configuration if one is given.

Args:
run_configuration: The run configuration for this pipeline.
Expand Down
16 changes: 12 additions & 4 deletions src/zenml/stack/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,16 +149,24 @@
"""Constructor for stack_context saves active stack."""
self._default_stack = Client().active_stack

def __enter__(self):
def __enter__(self) -> None:
"""Enters in the stack context."""
pass

def __exit__(
self,
exc_type: Optional[Type[BaseException]],
exc_value: Optional[BaseException],
exception_type: Optional[Type[BaseException]],
exception_value: Optional[BaseException],
traceback: Optional[TracebackType],
) -> None:
"""Restores the original active stack."""
"""Get a stack component flavor by name and type from a ZenStore.

Args:
exception_type: Type of the execption that was raised.

Check warning on line 165 in src/zenml/stack/utils.py

View workflow job for this annotation

GitHub Actions / spellcheck

"execption" should be "exception".
None if no execption.

Check warning on line 166 in src/zenml/stack/utils.py

View workflow job for this annotation

GitHub Actions / spellcheck

"execption" should be "exception".
kabinja marked this conversation as resolved.
Show resolved Hide resolved
exception_value: Type of exception that was raised.
e.g., divide_by_zero error. None if no exception.
traceback: Traceback report. None if no excpetion.

Check warning on line 169 in src/zenml/stack/utils.py

View workflow job for this annotation

GitHub Actions / spellcheck

"excpetion" should be "exception".
kabinja marked this conversation as resolved.
Show resolved Hide resolved
"""
if self._default_stack.id != Client().active_stack:
bcdurak marked this conversation as resolved.
Show resolved Hide resolved
Client().activate_stack(self._default_stack.id)
Loading