Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sentry: alter_table.go:1262: identity column × of relation × has 0 sequences instead of 1 (1) Wraps: (2) assertion failure Wraps: (3) attached stack trace -- stack trace: | github.com/cockroachd... #138925

Open
cockroach-sentry opened this issue Jan 13, 2025 · 5 comments
Open

Sentry: alter_table.go:1262: identity column × of relation × has 0 sequences instead of 1 (1) Wraps: (2) assertion failure Wraps: (3) attached stack trace -- stack trace: | github.com/cockroachd... #138925

cockroach-sentry opened this issue Jan 13, 2025 · 5 comments
Assignees
@rafiss
Labels
branch-release-24.3 Used to mark GA and release blockers, technical advisories, and bugs for 24.3 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) X-blathers-triaged blathers was able to find an owner

Comments

@cockroach-sentry
Copy link
Collaborator

cockroach-sentry commented Jan 13, 2025 •
edited by cockroach-jira-scripts
Loading

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/6216371960/?referrer=webhooks_plugin

Panic Message:

alter_table.go:1262: identity column × of relation × has 0 sequences instead of 1
(1)
Wraps: (2) assertion failure
Wraps: (3) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/sql.applyColumnMutation
  | 	pkg/sql/alter_table.go:1262
  | github.com/cockroachdb/cockroach/pkg/sql.(*alterTableNode).startExec
  | 	pkg/sql/alter_table.go:641
  | github.com/cockroachdb/cockroach/pkg/sql.startExec.func2
  | 	pkg/sql/plan.go:525
  | github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal.func1
  | 	pkg/sql/walk.go:107
  | github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal
  | 	pkg/sql/walk.go:294
  | github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visit
  | 	pkg/sql/walk.go:74
  | github.com/cockroachdb/cockroach/pkg/sql.walkPlan
  | 	pkg/sql/walk.go:38
  | github.com/cockroachdb/cockroach/pkg/sql.startExec
  | 	pkg/sql/plan.go:528
  | github.com/cockroachdb/cockroach/pkg/sql.(*planNodeToRowSource).Start
  | 	pkg/sql/plan_node_to_row_source.go:166
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Columnarizer).Init
  | 	pkg/sql/colexec/columnarizer.go:178
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).init
  | 	pkg/sql/colflow/stats.go:89
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
  | 	pkg/sql/colexecerror/error.go:162
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).Init
  | 	pkg/sql/colflow/stats.go:98
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Materializer).Start.func1
  | 	pkg/sql/colexec/materializer.go:228
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
  | 	pkg/sql/colexecerror/error.go:162
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Materializer).Start
  | 	pkg/sql/colexec/materializer.go:227
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).Start.func1
  | 	pkg/sql/colflow/flow_coordinator.go:111
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
  | 	pkg/sql/colexecerror/error.go:162
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).Start
  | 	pkg/sql/colflow/flow_coordinator.go:110
  | github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run
  | 	pkg/sql/execinfra/processorsbase.go:726
  | github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Run
  | 	pkg/sql/flowinfra/flow.go:574
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlow).Run
  | 	pkg/sql/colflow/vectorized_flow.go:300
  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run
  | 	pkg/sql/distsql_running.go:924
  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun
  | 	pkg/sql/distsql_running.go:2017
  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll.func3
  | 	pkg/sql/distsql_running.go:1731
  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll
  | 	pkg/sql/distsql_running.go:1734
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine
  | 	pkg/sql/conn_executor_exec.go:2435
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine
  | 	pkg/sql/conn_executor_exec.go:1981
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState
  | 	pkg/sql/conn_executor_exec.go:1187
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt.func1
  | 	pkg/sql/conn_executor_exec.go:141
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithProfiling
  | 	pkg/sql/conn_executor_exec.go:3451
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt
  | 	pkg/sql/conn_executor_exec.go:140
Wraps: (4) identity column × of relation × has 0 sequences instead of 1
Error types: (1) *colexecerror.notInternalError (2) *assert.withAssertionFailure (3) *withstack.withStack (4) *errutil.leafError
-- report composition:
*errutil.leafError: identity column × of relation × has 0 sequences instead of 1
alter_table.go:1262: *withstack.withStack (top exception)
*assert.withAssertionFailure
*colexecerror.notInternalError
Stacktrace (expand for inline code snippets):

pkg/sql/conn_executor_exec.go#L139-L141
pkg/sql/conn_executor_exec.go#L3450-L3452
pkg/sql/conn_executor_exec.go#L140-L142
pkg/sql/conn_executor_exec.go#L1186-L1188
pkg/sql/conn_executor_exec.go#L1980-L1982
pkg/sql/conn_executor_exec.go#L2434-L2436
pkg/sql/distsql_running.go#L1733-L1735
pkg/sql/distsql_running.go#L1730-L1732
pkg/sql/distsql_running.go#L2016-L2018
pkg/sql/distsql_running.go#L923-L925
pkg/sql/colflow/vectorized_flow.go#L299-L301
pkg/sql/flowinfra/flow.go#L573-L575
pkg/sql/execinfra/processorsbase.go#L725-L727
pkg/sql/colflow/flow_coordinator.go#L109-L111
pkg/sql/colexecerror/error.go#L161-L163
pkg/sql/colflow/flow_coordinator.go#L110-L112
pkg/sql/colexec/materializer.go#L226-L228
pkg/sql/colexecerror/error.go#L161-L163
pkg/sql/colexec/materializer.go#L227-L229
pkg/sql/colflow/stats.go#L97-L99
pkg/sql/colexecerror/error.go#L161-L163
pkg/sql/colflow/stats.go#L88-L90
pkg/sql/colexec/columnarizer.go#L177-L179
pkg/sql/plan_node_to_row_source.go#L165-L167
pkg/sql/plan.go#L527-L529
pkg/sql/walk.go#L37-L39
pkg/sql/walk.go#L73-L75
pkg/sql/walk.go#L293-L295
pkg/sql/walk.go#L106-L108
pkg/sql/plan.go#L524-L526
pkg/sql/alter_table.go#L640-L642
pkg/sql/alter_table.go#L1261-L1263

pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt at line 140
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execWithProfiling at line 3451
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt.func1 at line 141
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmtInOpenState at line 1187
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).dispatchToExecutionEngine at line 1981
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execWithDistSQLEngine at line 2435
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRunAll at line 1734
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRunAll.func3 at line 1731
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRun at line 2017
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).Run at line 924
pkg/sql/colflow/vectorized_flow.go in pkg/sql/colflow.(*vectorizedFlow).Run at line 300
pkg/sql/flowinfra/flow.go in pkg/sql/flowinfra.(*FlowBase).Run at line 574
pkg/sql/execinfra/processorsbase.go in pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run at line 726
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*FlowCoordinator).Start at line 110
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 162
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*FlowCoordinator).Start.func1 at line 111
pkg/sql/colexec/materializer.go in pkg/sql/colexec.(*Materializer).Start at line 227
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 162
pkg/sql/colexec/materializer.go in pkg/sql/colexec.(*Materializer).Start.func1 at line 228
pkg/sql/colflow/stats.go in pkg/sql/colflow.(*batchInfoCollector).Init at line 98
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 162
pkg/sql/colflow/stats.go in pkg/sql/colflow.(*batchInfoCollector).init at line 89
pkg/sql/colexec/columnarizer.go in pkg/sql/colexec.(*Columnarizer).Init at line 178
pkg/sql/plan_node_to_row_source.go in pkg/sql.(*planNodeToRowSource).Start at line 166
pkg/sql/plan.go in pkg/sql.startExec at line 528
pkg/sql/walk.go in pkg/sql.walkPlan at line 38
pkg/sql/walk.go in pkg/sql.(*planVisitor).visit at line 74
pkg/sql/walk.go in pkg/sql.(*planVisitor).visitInternal at line 294
pkg/sql/walk.go in pkg/sql.(*planVisitor).visitInternal.func1 at line 107
pkg/sql/plan.go in pkg/sql.startExec.func2 at line 525
pkg/sql/alter_table.go in pkg/sql.(*alterTableNode).startExec at line 641
pkg/sql/alter_table.go in pkg/sql.applyColumnMutation at line 1262

Tags

Tag Value
Command mt start-sql
Environment v24.3.2
Go Version go1.22.8 X:nocoverageredesign
Platform linux amd64
Distribution CCL
Cockroach Release v24.3.2
Cockroach SHA 2c5477e
# of CPUs 32
# of Goroutines 389

Jira issue: CRDB-46428
@cockroach-sentry cockroach-sentry added branch-release-24.3 Used to mark GA and release blockers, technical advisories, and bugs for 24.3 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. labels Jan 13, 2025
@blathers-crl blathers[crl]
Copy link

blathers-crl bot commented Jan 13, 2025

CC'ing via the CODEOWNERS-based sentry heuristic:

  • @cockroachdb/sql-foundations

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@blathers-crl blathers-crl bot added T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) X-blathers-triaged blathers was able to find an owner labels Jan 13, 2025
@rafiss
Copy link
Collaborator

rafiss commented Jan 14, 2025

I'll try to search in Datadog to see if I can identify which cluster this came from.

@exalate-issue-sync exalate-issue-sync bot changed the title Sentry: alter_table.go:1262: identity column × of relation × has 0 sequences instead of 1 (1) Wraps: (2) assertion failure Wraps: (3) attached stack trace -- stack trace: | github.com/cockroachd... Sentry: alter_table.go:1262: identity column × of relation × has 0 sequences instead of 1 (1) Wraps: (2) assertion failure Wraps: (3) attached stack trace -- stack trace: | github.com/cockroachd... Jan 14, 2025
@rafiss
Copy link
Collaborator

rafiss commented Jan 14, 2025

Logs are here: https://us5.datadoghq.com/logs?query=source%3Acockroachdb%20env%3Aproduction%20%40friendly_id%3Acrl-prod-j77%20%40tenant_id%3A6198%20%40channel%3ATELEMETRY&agg_m=count&agg_m_source=base&agg_t=count&clustering_pattern_field_path=message&cols=host%2Cservice&event=AwAAAZRgMzPwrnp16AAAABhBWlJnTXpQd0FBQk1JMDBsaVJacnhnQUEAAAAkMDE5NDYwMzMtNGQ4OS00Zjg1LWI4YzgtNDJhNzNhNmY0YjUxAAHMtw&fromUser=true&messageDisplay=expanded-lg&refresh_mode=paused&storage=flex_tier&stream_sort=desc&viz=stream&from_ts=1736780640000&to_ts=1736780760000&live=false

friendly_id: crl-prod-j77
tenant_id: 6198
statement: ALTER TABLE <table> ALTER COLUMN <column> DROP IDENTITY IF EXISTS

@rafiss
Copy link
Collaborator

rafiss commented Jan 14, 2025

This column looks strange. Since it has GENERATED_BY_DEFAULT, it should have a default expression that refers to a sequence. Using unique_rowid() shouldn't be allowed here.

            {
                "defaultExpr": "unique_rowid()",
                "generatedAsIdentitySequenceOption": "",
                "generatedAsIdentityType": "GENERATED_BY_DEFAULT",
                "id": 1,
                "name": "transaction_id",
                "type": {
                    "family": "IntFamily",
                    "oid": 20,
                    "width": 64
                }
            },

I tracked down the create statement here: https://us5.datadoghq.com/logs?query=source%3Acockroachdb%20env%3Aproduction%20%40friendly_id%3Acrl-prod-j77%20%40tenant_id%3A6198%20%40channel%3ATELEMETRY%20%2A%3A%22fl_fms.fms.transaction%22&agg_m=count&agg_m_source=base&agg_t=count&clustering_pattern_field_path=message&cols=host%2Cservice&event=AwAAAZPP3URiVg9XmQAAABhBWlBQM1VSaUFBQVgzaUtoTUxSbGpRQUEAAAAkMDE5M2NmZGQtZTM2Ni00OTRiLWE1N2ItMWU1ZGI4OTAyYWE4AAbuTg&fromUser=true&messageDisplay=expanded-lg&refresh_mode=paused&storage=flex_tier&stream_sort=desc&viz=stream&from_ts=1734220800000&to_ts=1734479999999&live=false

That helped me come up with a simple repro. The following is allowed in 24.2 and earlier, but it shouldn't be:

> SET serial_normalization = rowid; -- this is the default value of the setting.

> create table t (a serial generated always as identity, b int);

In 24.3 we block this, and it fails with:

ERROR: relation "t" (105): column is identity without sequence references "a"

This was referenced Jan 14, 2025
Open
Merged
craig bot pushed a commit that referenced this issue Jan 15, 2025
@rafiss
Merge #139084
0b4d620 
139084: sql: relax validation for IDENTITY columns r=rafiss a=rafiss

The validations added in 4f99f9c were
too strict. This patch makes it so we can continue having IDENTITY
columns that are not backed by a sequence. This can happen if a SERIAL
column is created with IDENTITY, and serial_normalization=rowid.

informs #138925
informs #138972

Release note (bug fix): Fixed a bug where columns created with GENERATED ... BY IDENTITY with the SERIAL type could incorrectly fail internal validations.

Co-authored-by: Rafi Shamim <[email protected]>
@blathers-crl blathers-crl bot mentioned this issue Jan 15, 2025
Open
@rafiss
Copy link
Collaborator

rafiss commented Jan 15, 2025

After more discussion, we've decided to continue with the 24.2 behavior of allowing this kind of column. Removing the IDENTITY property would be too destructive a change, since one property of those columns is that they cannot be written to. Instead, we will make the validations less strict, so that an IDENTITY column can be backed by unique_rowid() similar to how it was in 24.2. The validation in #135955 was added with the intention of making sure OwnsSequenceIDs matches UsesSequenceIDs, so we will allow IDENTITY columns to have either 1 or 0 elements in both of those slices.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rafiss rafiss

Labels
branch-release-24.3 Used to mark GA and release blockers, technical advisories, and bugs for 24.3 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) X-blathers-triaged blathers was able to find an owner
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rafiss @cockroach-sentry