Skip to content

Commit

Permalink
catalog: add maybeAddConstraintIDs to RestoreChanges
Browse files Browse the repository at this point in the history 
All backups taken in versions 22.1 and later should already have
constraint IDs set. Technically, now that we are in v24.1, we no longer
support restoring from versions older than 22.1, but we still have
tests that restore from old versions. Until those fixtures are updated,
we need to handle the case where constraint IDs are not set already when
restoring a backup.

To prove that this works, I've resurrected a different test that
restores from a v21.1 backup, which was earlier deleted in 8b56efd.

Release note: None
  • Loading branch information
@rafiss
rafiss committed Mar 8, 2024
1 parent d2042ef commit 8cd23eb
Show file tree
Hide file tree
Showing 7 changed files with 132 additions and 96 deletions.
9 changes: 9 additions & 0 deletions pkg/ccl/backupccl/testdata/backup-restore/show_backup
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@
new-cluster name=s1 allow-implicit-access localities=eu-central-1,eu-north-1,us-east-1
----

link-backup cluster=s1 src-path=show_backup_validate,invalidDependOnBy_21.1 dest-path=invalidDependOnBy_21.1
----

# This backup intentionally has a dangling invalid depend on by reference.
query-sql regex=invalid\sdepended-on-by
SELECT * FROM [SHOW BACKUP VALIDATE FROM 'invalidDependOnBy_21.1' IN 'nodelocal://1/'];
----
true

link-backup cluster=s1 src-path=show_backup_validate,valid-22.2 dest-path=valid-22.2
----

Expand Down
Binary file added pkg/ccl/backupccl/testdata/show_backup_validate/invalidDependOnBy_21.1/BACKUP_MANIFEST
View file
Binary file not shown.
1 change: 1 addition & 0 deletions ...l/backupccl/testdata/show_backup_validate/invalidDependOnBy_21.1/BACKUP_MANIFEST-CHECKSUM
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
'���
54 changes: 6 additions & 48 deletions pkg/cli/testdata/doctor/test_examine_zipdir
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,54 +3,12 @@ debug doctor examine zipdir testdata/doctor/debugzip-wth
debug doctor examine zipdir testdata/doctor/debugzip 21.1-52
WARNING: errors occurred during the production of system.jobs.txt, contents may be missing or incomplete.
Examining 37 descriptors and 42 namespace entries...
ParentID 1, ParentSchemaID 29: relation "descriptor" (3): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "users" (4): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "zones" (5): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "settings" (6): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "tenants" (8): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "lease" (11): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "eventlog" (12): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "rangelog" (13): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "ui" (14): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "jobs" (15): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "web_sessions" (19): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "table_statistics" (20): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "locations" (21): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "role_members" (23): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "comments" (24): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "replication_constraint_stats" (25): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "replication_critical_localities" (26): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "replication_stats" (27): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "reports_meta" (28): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "namespace" (30): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "protected_ts_meta" (31): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "protected_ts_meta" (31): constraint ID was missing for constraint "check_singleton"
ParentID 1, ParentSchemaID 29: relation "protected_ts_meta" (31): constraint ID 0 in constraint "check_singleton" already in use by "primary"
ParentID 1, ParentSchemaID 29: relation "protected_ts_records" (32): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "role_options" (33): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "statement_bundle_chunks" (34): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "statement_diagnostics_requests" (35): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "statement_diagnostics" (36): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "scheduled_jobs" (37): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "sqlliveness" (39): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "users" (53): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "vehicles" (54): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "vehicles" (54): constraint ID was missing for constraint "fk_city_ref_users"
ParentID 52, ParentSchemaID 29: relation "vehicles" (54): constraint ID 0 in constraint "fk_city_ref_users" already in use by "primary"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID was missing for constraint "check_vehicle_city_city"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID 0 in constraint "check_vehicle_city_city" already in use by "primary"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID was missing for constraint "fk_city_ref_users"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID 0 in constraint "fk_city_ref_users" already in use by "check_vehicle_city_city"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID was missing for constraint "fk_vehicle_city_ref_vehicles"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID 0 in constraint "fk_vehicle_city_ref_vehicles" already in use by "fk_city_ref_users"
ParentID 52, ParentSchemaID 29: relation "vehicle_location_histories" (56): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "vehicle_location_histories" (56): constraint ID was missing for constraint "fk_city_ref_rides"
ParentID 52, ParentSchemaID 29: relation "vehicle_location_histories" (56): constraint ID 0 in constraint "fk_city_ref_rides" already in use by "primary"
ParentID 52, ParentSchemaID 29: relation "promo_codes" (57): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "user_promo_codes" (58): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "user_promo_codes" (58): constraint ID was missing for constraint "fk_city_ref_users"
ParentID 52, ParentSchemaID 29: relation "user_promo_codes" (58): constraint ID 0 in constraint "fk_city_ref_users" already in use by "primary"
ParentID 52, ParentSchemaID 29: relation "users" (53): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "vehicles" (54): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "rides" (55): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "vehicle_location_histories" (56): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "promo_codes" (57): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "user_promo_codes" (58): referenced database ID 52: referenced descriptor not found
ParentID 0, ParentSchemaID 0: namespace entry "movr" (52): referenced schema ID 52: referenced descriptor not found
Examining 2 jobs...
job 587337426984566785: running schema change GC refers to missing table descriptor(s) [59]; existing descriptors that still need to be dropped []; job safe to delete: true.
Expand Down
54 changes: 6 additions & 48 deletions pkg/cli/testdata/doctor/test_examine_zipdir_verbose
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,89 +7,47 @@ WARNING: errors occurred during the production of system.jobs.txt, contents may
reading testdata/doctor/debugzip/system.jobs.txt
Examining 37 descriptors and 42 namespace entries...
ParentID 0, ParentSchemaID 0: database "system" (1): processed
ParentID 1, ParentSchemaID 29: relation "descriptor" (3): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "descriptor" (3): processed
ParentID 1, ParentSchemaID 29: relation "users" (4): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "users" (4): processed
ParentID 1, ParentSchemaID 29: relation "zones" (5): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "zones" (5): processed
ParentID 1, ParentSchemaID 29: relation "settings" (6): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "settings" (6): processed
ParentID 1, ParentSchemaID 29: relation "tenants" (8): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "tenants" (8): processed
ParentID 1, ParentSchemaID 29: relation "lease" (11): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "lease" (11): processed
ParentID 1, ParentSchemaID 29: relation "eventlog" (12): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "eventlog" (12): processed
ParentID 1, ParentSchemaID 29: relation "rangelog" (13): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "rangelog" (13): processed
ParentID 1, ParentSchemaID 29: relation "ui" (14): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "ui" (14): processed
ParentID 1, ParentSchemaID 29: relation "jobs" (15): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "jobs" (15): processed
ParentID 1, ParentSchemaID 29: relation "web_sessions" (19): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "web_sessions" (19): processed
ParentID 1, ParentSchemaID 29: relation "table_statistics" (20): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "table_statistics" (20): processed
ParentID 1, ParentSchemaID 29: relation "locations" (21): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "locations" (21): processed
ParentID 1, ParentSchemaID 29: relation "role_members" (23): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "role_members" (23): processed
ParentID 1, ParentSchemaID 29: relation "comments" (24): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "comments" (24): processed
ParentID 1, ParentSchemaID 29: relation "replication_constraint_stats" (25): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "replication_constraint_stats" (25): processed
ParentID 1, ParentSchemaID 29: relation "replication_critical_localities" (26): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "replication_critical_localities" (26): processed
ParentID 1, ParentSchemaID 29: relation "replication_stats" (27): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "replication_stats" (27): processed
ParentID 1, ParentSchemaID 29: relation "reports_meta" (28): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "reports_meta" (28): processed
ParentID 1, ParentSchemaID 29: relation "namespace" (30): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "namespace" (30): processed
ParentID 1, ParentSchemaID 29: relation "protected_ts_meta" (31): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "protected_ts_meta" (31): constraint ID was missing for constraint "check_singleton"
ParentID 1, ParentSchemaID 29: relation "protected_ts_meta" (31): constraint ID 0 in constraint "check_singleton" already in use by "primary"
ParentID 1, ParentSchemaID 29: relation "protected_ts_meta" (31): processed
ParentID 1, ParentSchemaID 29: relation "protected_ts_records" (32): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "protected_ts_records" (32): processed
ParentID 1, ParentSchemaID 29: relation "role_options" (33): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "role_options" (33): processed
ParentID 1, ParentSchemaID 29: relation "statement_bundle_chunks" (34): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "statement_bundle_chunks" (34): processed
ParentID 1, ParentSchemaID 29: relation "statement_diagnostics_requests" (35): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "statement_diagnostics_requests" (35): processed
ParentID 1, ParentSchemaID 29: relation "statement_diagnostics" (36): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "statement_diagnostics" (36): processed
ParentID 1, ParentSchemaID 29: relation "scheduled_jobs" (37): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "scheduled_jobs" (37): processed
ParentID 1, ParentSchemaID 29: relation "sqlliveness" (39): constraint ID was missing for constraint "primary"
ParentID 1, ParentSchemaID 29: relation "sqlliveness" (39): processed
ParentID 0, ParentSchemaID 0: database "defaultdb" (50): processed
ParentID 0, ParentSchemaID 0: database "postgres" (51): processed
ParentID 52, ParentSchemaID 29: relation "users" (53): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "users" (53): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "users" (53): processed
ParentID 52, ParentSchemaID 29: relation "vehicles" (54): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "vehicles" (54): constraint ID was missing for constraint "fk_city_ref_users"
ParentID 52, ParentSchemaID 29: relation "vehicles" (54): constraint ID 0 in constraint "fk_city_ref_users" already in use by "primary"
ParentID 52, ParentSchemaID 29: relation "vehicles" (54): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "vehicles" (54): processed
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID was missing for constraint "check_vehicle_city_city"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID 0 in constraint "check_vehicle_city_city" already in use by "primary"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID was missing for constraint "fk_city_ref_users"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID 0 in constraint "fk_city_ref_users" already in use by "check_vehicle_city_city"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID was missing for constraint "fk_vehicle_city_ref_vehicles"
ParentID 52, ParentSchemaID 29: relation "rides" (55): constraint ID 0 in constraint "fk_vehicle_city_ref_vehicles" already in use by "fk_city_ref_users"
ParentID 52, ParentSchemaID 29: relation "rides" (55): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "rides" (55): processed
ParentID 52, ParentSchemaID 29: relation "vehicle_location_histories" (56): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "vehicle_location_histories" (56): constraint ID was missing for constraint "fk_city_ref_rides"
ParentID 52, ParentSchemaID 29: relation "vehicle_location_histories" (56): constraint ID 0 in constraint "fk_city_ref_rides" already in use by "primary"
ParentID 52, ParentSchemaID 29: relation "vehicle_location_histories" (56): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "vehicle_location_histories" (56): processed
ParentID 52, ParentSchemaID 29: relation "promo_codes" (57): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "promo_codes" (57): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "promo_codes" (57): processed
ParentID 52, ParentSchemaID 29: relation "user_promo_codes" (58): constraint ID was missing for constraint "primary"
ParentID 52, ParentSchemaID 29: relation "user_promo_codes" (58): constraint ID was missing for constraint "fk_city_ref_users"
ParentID 52, ParentSchemaID 29: relation "user_promo_codes" (58): constraint ID 0 in constraint "fk_city_ref_users" already in use by "primary"
ParentID 52, ParentSchemaID 29: relation "user_promo_codes" (58): referenced database ID 52: referenced descriptor not found
ParentID 52, ParentSchemaID 29: relation "user_promo_codes" (58): processed
ParentID 0, ParentSchemaID 0: namespace entry "defaultdb" (50): processed
ParentID 0, ParentSchemaID 0: namespace entry "movr" (52): referenced schema ID 52: referenced descriptor not found
Expand Down
4 changes: 4 additions & 0 deletions pkg/sql/catalog/post_deserialization_changes.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ const (
// references.
RemovedDuplicateIDsInRefs

// AddedConstraintIDs indicates that table descriptors had constraint ID
// added.
AddedConstraintIDs

// RemovedSelfEntryInSchemas corresponds to a change which occurred in
// database descriptors to recover from an earlier bug whereby when
// dropping a schema, we'd mark the database itself as though it was the
Expand Down
Loading

0 comments on commit 8cd23eb

Please sign in to comment.