Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump Pydantic to version 2 #160

Closed
wants to merge 29 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
6891ade
Bump Pydantic to version 2
ThomasLaPiana Sep 7, 2023
c491758
Add Pydantic 2 to Action Matrix
ThomasLaPiana Sep 8, 2023
153bf2d
checkin
ThomasLaPiana Nov 10, 2023
7693bcb
feat: remove data qualifiers
ThomasLaPiana Nov 10, 2023
3e486b1
feat: more qualifier removals
ThomasLaPiana Nov 10, 2023
7371ff7
Merge branch 'ThomasLaPiana-remove-data-qualifiers' into ThomasLaPian…
ThomasLaPiana Nov 10, 2023
20a0785
fix: mypy and pylint
ThomasLaPiana Nov 10, 2023
d55e284
fix more failing tests
ThomasLaPiana Nov 13, 2023
5f2c86c
fix version and fides_key validation test failures
ThomasLaPiana Nov 13, 2023
a718e17
fix missing defaults
ThomasLaPiana Nov 13, 2023
c329f44
clean up more test failures
ThomasLaPiana Nov 13, 2023
bfb8d3d
fix more errors
ThomasLaPiana Nov 13, 2023
25e5b80
fix parent key validation
ThomasLaPiana Nov 14, 2023
73edf8b
get everything passing (by removing two tests)
ThomasLaPiana Nov 14, 2023
126e4be
feat: update CI checks for new pydantic versions
ThomasLaPiana Nov 14, 2023
cc4d782
update requirements file, remove python 3.8 and add 3.12 to supported…
ThomasLaPiana Nov 14, 2023
ba08cf1
remove python 3.12 from the matrix
ThomasLaPiana Nov 14, 2023
12ef6a8
fix static checks
ThomasLaPiana Nov 14, 2023
63d2795
docs: changelog
ThomasLaPiana Nov 14, 2023
40d4faa
re-add tests and fix privacy declaration checks
ThomasLaPiana Nov 27, 2023
d82bf8c
Merge branch 'fideslang-3' into ThomasLaPiana-bump-pydantic-2
ThomasLaPiana Nov 27, 2023
537537c
fix flexible default test
ThomasLaPiana Nov 27, 2023
191f6df
fix static checks
ThomasLaPiana Nov 27, 2023
9ec615a
fix the validators on our custom fideskey types to be json schema valid
ThomasLaPiana Nov 28, 2023
43abfb7
turned off strict checking for the registry id
ThomasLaPiana Nov 28, 2023
e7de840
add more None defaults to optional types
ThomasLaPiana Nov 30, 2023
a66a4ce
remove all uses of URL since they might cause issues with database st…
ThomasLaPiana Nov 30, 2023
557aebf
remove a validator on System that caused issues when loading from Orm…
ThomasLaPiana Nov 30, 2023
b87a229
remove deprecation tests and update model (root) validators
ThomasLaPiana Nov 30, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/pr_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,9 @@ jobs:
Pytest-Matrix:
strategy:
matrix:
python_version: ["3.8", "3.9", "3.10", "3.11"]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python 3.8 wasn't playing nicely, so I axed it. It is almost EOL and we can only make the matrix so large

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood, how do we generally communicate things like this to customers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than changelog, it'll automatically fail to pip install if they have an older version.

Afaik most people are using Docker containers, this would be most disruptive to customers using the CLI

pydantic_version: ["1.8.2", "1.9.2", "1.10.9"]
pyyaml_version: ["5.4.1", "6.0"]
python_version: ["3.9", "3.10", "3.11"]
pydantic_version: ["2.2.1", "2.3.0", "2.4.2", "2.5.0"]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the most recent ones, but Pydantic moves fast so we'll need to keep an eye on this

pyyaml_version: ["5.4.1", "6.0.1"]
runs-on: ubuntu-latest
continue-on-error: true
steps:
Expand Down
9 changes: 8 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,16 @@ The types of changes are:
- Give Flexible Legal Basis a default of True [#184](https://github.com/ethyca/fideslang/pull/184)


## [3.0.0 Unreleased]

### Changed

- Updated to Pydantic 2.X, which is now the only supported version [#160](https://github.com/ethyca/fideslang/pull/160)
- Removed Python 3.8 from the list of supported versions [#160](https://github.com/ethyca/fideslang/pull/160)

## [2.2.1](https://github.com/ethyca/fideslang/compare/2.2.0...2.2.1)

### Added
### Added

- Added a `System.cookies` property to support `Cookie` records explicitly associated with a `System` generally [#181](https://github.com/ethyca/fideslang/pull/181)
- Added a `System.previous_vendor_id` property to support to associate a `System` record with a "deprecated" vendor record [#182](https://github.com/ethyca/fideslang/pull/182)
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.8-slim-bullseye as base
FROM python:3.9-slim-bullseye as base

# Update pip in the base image since we'll use it everywhere
RUN pip install -U pip
Expand Down
2 changes: 1 addition & 1 deletion dev-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ black==23.3.0
mypy==1.4.0
nox>=2023
packaging>=22.0
pre-commit==2.9.3
pre-commit==3.5.0
pylint==2.10.0
pytest==7.3.1
pytest-cov==2.11.1
Expand Down
7 changes: 4 additions & 3 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
nox.options.sessions = []
nox.options.reuse_existing_virtualenvs = True

TESTED_PYTHON_VERSIONS = ["3.8", "3.9", "3.10", "3.11"]
TESTED_PYDANTIC_VERSIONS = ["1.8.2", "1.9.2", "1.10.9"]
TESTED_PYYAML_VERSIONS = ["5.4.1", "6.0"]
# These should match what is in the `pr_checks.yml` file for CI runs
TESTED_PYTHON_VERSIONS = ["3.9", "3.10", "3.11"]
TESTED_PYDANTIC_VERSIONS = ["2.2.1", "2.3.0", "2.4.2", "2.5.0"]
TESTED_PYYAML_VERSIONS = ["5.4.1", "6.0.1"]


def install_requirements(session: nox.Session) -> None:
Expand Down
3 changes: 1 addition & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,12 @@ name = "fideslang"
description = "Fides Taxonomy Language"
dynamic = ["dependencies", "version"]
readme = "README.md"
requires-python = ">=3.8, <4"
requires-python = ">=3.9, <4"
authors = [{ name = "Ethyca, Inc.", email = "[email protected]" }]
license = { text = "Apache License 2.0" }
classifiers = [
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
pydantic>=1.8.1,<1.11.0
pydantic>=2.2.1,<=2.6.0
pyyaml>=5,<7
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not cross-supporting Pydantic 1.0!

packaging>=20.0
2 changes: 1 addition & 1 deletion src/fideslang/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@
DataSubject,
DataUse,
Evaluation,
FidesCollectionKey,
FidesDatasetReference,
FidesMeta,
FidesModel,
Expand All @@ -46,6 +45,7 @@
System,
Taxonomy,
)
from .validation import FidesCollectionKey

FidesModelType = Union[Type[FidesModel], Type[Evaluation]]
model_map: Dict[str, FidesModelType] = {
Expand Down
2 changes: 1 addition & 1 deletion src/fideslang/default_taxonomy/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@ def default_factory(taxonomy_class: CustomType, **kwargs: Dict) -> CustomType:
# This is the version where we started tracking from, so
# we use it as the default starting point.
kwargs["version_added"] = "2.0.0" # type: ignore[assignment]
item = taxonomy_class.parse_obj(kwargs)
item = taxonomy_class.model_validate(obj=kwargs)
pattisdr marked this conversation as resolved.
Show resolved Hide resolved
return item
16 changes: 8 additions & 8 deletions src/fideslang/gvl/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,16 +50,16 @@ def _load_data() -> None:
) as mapping_file:
data = load(mapping_file)
for raw_purpose in data["purposes"].values():
purpose = Purpose.parse_obj(raw_purpose)
mapped_purpose = MappedPurpose.parse_obj(raw_purpose)
purpose = Purpose.model_validate(raw_purpose)
mapped_purpose = MappedPurpose.model_validate(raw_purpose)
GVL_PURPOSES[purpose.id] = purpose
MAPPED_PURPOSES[mapped_purpose.id] = mapped_purpose
for data_use in mapped_purpose.data_uses:
MAPPED_PURPOSES_BY_DATA_USE[data_use] = mapped_purpose

for raw_special_purpose in data["specialPurposes"].values():
special_purpose = Purpose.parse_obj(raw_special_purpose)
mapped_special_purpose = MappedPurpose.parse_obj(raw_special_purpose)
special_purpose = Purpose.model_validate(raw_special_purpose)
mapped_special_purpose = MappedPurpose.model_validate(raw_special_purpose)
GVL_SPECIAL_PURPOSES[special_purpose.id] = special_purpose
MAPPED_SPECIAL_PURPOSES[mapped_special_purpose.id] = mapped_special_purpose
for data_use in mapped_special_purpose.data_uses:
Expand All @@ -71,12 +71,12 @@ def _load_data() -> None:
feature_data = load(feature_mapping_file)

for raw_feature in feature_data["features"].values():
feature = Feature.parse_obj(raw_feature)
feature = Feature.model_validate(raw_feature)
GVL_FEATURES[feature.id] = feature
FEATURES_BY_NAME[feature.name] = feature

for raw_special_feature in feature_data["specialFeatures"].values():
special_feature = Feature.parse_obj(raw_special_feature)
special_feature = Feature.model_validate(raw_special_feature)
GVL_SPECIAL_FEATURES[special_feature.id] = special_feature
FEATURES_BY_NAME[special_feature.name] = special_feature

Expand All @@ -86,8 +86,8 @@ def _load_data() -> None:
data_category_data = load(data_category_mapping_file)

for raw_data_category in data_category_data.values():
data_category = GVLDataCategory.parse_obj(raw_data_category)
mapped_data_category = MappedDataCategory.parse_obj(raw_data_category)
data_category = GVLDataCategory.model_validate(raw_data_category)
mapped_data_category = MappedDataCategory.model_validate(raw_data_category)
GVL_DATA_CATEGORIES[data_category.id] = data_category
MAPPED_GVL_DATA_CATEGORIES[mapped_data_category.id] = mapped_data_category

Expand Down
Loading
Loading