Sentry: datum.go:5341: comparison of two different versions of enum %!s(func() string=0x14c7a00) oid ×: versions 105 and 106 (1) Wraps: (2) Wraps: (3) assertion failure Wraps: (4) attached stack trac...

[cockroach-sentry]

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/6167530672/?referrer=webhooks_plugin

Panic Message:

datum.go:5341: comparison of two different versions of enum %!s(func() string=0x14c7a00) oid ×: versions 105 and 106
(1)
Wraps: (2)
Wraps: (3) assertion failure
Wraps: (4) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/sql/sem/tree.(*DEnum).Compare
  | 	github.com/cockroachdb/cockroach/pkg/sql/sem/tree/datum.go:5341
  | github.com/cockroachdb/cockroach/pkg/sql/sem/eval.(*evaluator).EvalCompareScalarOp
  | 	github.com/cockroachdb/cockroach/pkg/sql/sem/eval/binary_op.go:157
  | github.com/cockroachdb/cockroach/pkg/sql/sem/tree.(*CompareScalarOp).Eval
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/sql/sem/tree/eval_op_generated.go:301
  | github.com/cockroachdb/cockroach/pkg/sql/sem/eval.(*evaluator).EvalComparisonExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/sem/eval/expr.go:262
  | github.com/cockroachdb/cockroach/pkg/sql/sem/tree.(*ComparisonExpr).Eval
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/sql/sem/tree/eval_expr_generated.go:117
  | github.com/cockroachdb/cockroach/pkg/sql/sem/eval.(*evaluator).EvalAndExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/sem/eval/expr.go:35
  | github.com/cockroachdb/cockroach/pkg/sql/sem/tree.(*AndExpr).Eval
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/sql/sem/tree/eval_expr_generated.go:67
  | github.com/cockroachdb/cockroach/pkg/sql/sem/eval.Expr
  | 	github.com/cockroachdb/cockroach/pkg/sql/sem/eval/expr.go:21
  | github.com/cockroachdb/cockroach/pkg/sql/execinfrapb.RunFilter
  | 	github.com/cockroachdb/cockroach/pkg/sql/execinfrapb/pkg/sql/execinfrapb/expr.go:43
  | github.com/cockroachdb/cockroach/pkg/sql/execinfrapb.(*exprHelper).evalFilter
  | 	github.com/cockroachdb/cockroach/pkg/sql/execinfrapb/pkg/sql/execinfrapb/expr.go:286
  | github.com/cockroachdb/cockroach/pkg/sql/execinfrapb.(*ExprHelper).EvalFilter
  | 	github.com/cockroachdb/cockroach/pkg/sql/execinfrapb/pkg/sql/execinfrapb/expr.go:96
  | github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*joinerBase).render
  | 	github.com/cockroachdb/cockroach/pkg/sql/rowexec/joinerbase.go:231
  | github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*joinReaderNoOrderingStrategy).nextRowToEmit
  | 	github.com/cockroachdb/cockroach/pkg/sql/rowexec/joinreader_strategies.go:293
  | github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*joinReader).emitRow
  | 	github.com/cockroachdb/cockroach/pkg/sql/rowexec/joinreader.go:1173
  | github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*joinReader).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/rowexec/joinreader.go:781
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Columnarizer).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/columnarizer.go:234
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).exec
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:514
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:305
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:197
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:162
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:195
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*isNullSelOp).Next
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/sql/colexec/is_null_ops.eg.go:213
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*HashRouter).processNextBatch
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/routers.go:662
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*HashRouter).Run.func1.1
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/routers.go:586
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:162
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*HashRouter).Run.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/routers.go:624
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:162
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*HashRouter).Run
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/routers.go:581
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreator).setupRouter.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:820
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreator).setupRouter.(*vectorizedFlowCreator).accumulateAsyncComponent.func2.1
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:723
  | runtime.goexit
  | 	src/runtime/asm_amd64.s:1695
Wraps: (5) comparison of two different versions of enum %!s(func() string=0x14c7a00) oid ×: versions 105 and 106
Error types: (1) *colexecerror.notInternalError (2) *colexecerror.notInternalError (3) *assert.withAssertionFailure (4) *withstack.withStack (5) *errutil.leafError
-- report composition:
*errutil.leafError: comparison of two different versions of enum %!s(func() string=0x14c7a00) oid ×: versions 105 and 106
datum.go:5341: *withstack.withStack (top exception)
*assert.withAssertionFailure
*colexecerror.notInternalError
*colexecerror.notInternalError

Stacktrace (expand for inline code snippets):

src/runtime/asm_amd64.s#L1694-L1696

cockroach/pkg/sql/colflow/vectorized_flow.go

Lines 722 to 724 in 74f6082

	defer wg.Done()
	run(ctx, flowCtxCancel)
	}()

cockroach/pkg/sql/colflow/vectorized_flow.go

Lines 819 to 821 in 74f6082

	runRouter := func(ctx context.Context, _ context.CancelFunc) {
	router.Run(logtags.AddTag(ctx, "hashRouterID", streamIDs))
	}

cockroach/pkg/sql/colflow/routers.go

Lines 580 to 582 in 74f6082

	// well for more fine-grained control of error propagation.
	if err := colexecerror.CatchVectorizedRuntimeError(func() {
	r.Input.Init(ctx)

cockroach/pkg/sql/colexecerror/error.go

Lines 161 to 163 in 74f6082

	}()
	operation()
	return retErr

cockroach/pkg/sql/colflow/routers.go

Lines 623 to 625 in 74f6082

	if err := colexecerror.CatchVectorizedRuntimeError(processNextBatch); err != nil {
	r.cancelOutputs(ctx, err)

cockroach/pkg/sql/colexecerror/error.go

Lines 161 to 163 in 74f6082

	}()
	operation()
	return retErr

cockroach/pkg/sql/colflow/routers.go

Lines 585 to 587 in 74f6082

	processNextBatch := func() {
	done = r.processNextBatch(ctx)
	}

cockroach/pkg/sql/colflow/routers.go

Lines 661 to 663 in 74f6082

	func (r *HashRouter) processNextBatch(ctx context.Context) bool {
	b := r.Input.Next()
	n := b.Length()

https://github.com/cockroachdb/cockroach/blob/74f60821a82c570ab58b8d7716c1d74b85e87df0/bazel-out/k8-opt/bin/pkg/sql/colexec/is_null_ops.eg.go#L212-L214

cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go

Lines 194 to 196 in 74f6082

	var batch coldata.Batch
	if err := colexecerror.CatchVectorizedRuntimeError(
	func() {

cockroach/pkg/sql/colexecerror/error.go

Lines 161 to 163 in 74f6082

	}()
	operation()
	return retErr

cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go

Lines 196 to 198 in 74f6082

	func() {
	batch = d.inMemoryOp.Next()
	},

cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go

Lines 304 to 306 in 74f6082

	case hjProbing:
	output := hj.exec()
	if output.Length() == 0 {

cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go

Lines 513 to 515 in 74f6082

	for {
	batch := hj.InputOne.Next()
	batchSize := batch.Length()

cockroach/pkg/sql/colexec/columnarizer.go

Lines 233 to 235 in 74f6082

	for batchDone := false; !batchDone; {
	row, meta := c.input.Next()
	if meta != nil {

cockroach/pkg/sql/rowexec/joinreader.go

Lines 780 to 782 in 74f6082

	case jrEmittingRows:
	jr.runningState, row, meta = jr.emitRow()
	case jrReadyToDrain:

cockroach/pkg/sql/rowexec/joinreader.go

Lines 1172 to 1174 in 74f6082

	) {
	rowToEmit, nextState, err := jr.strategy.nextRowToEmit(jr.Ctx())
	if err != nil {

cockroach/pkg/sql/rowexec/joinreader_strategies.go

Lines 292 to 294 in 74f6082

	// Render the output row, this also evaluates the ON condition.
	outputRow, err := s.render(inputRow, s.emitState.lookedUpRow)
	if err != nil {

cockroach/pkg/sql/rowexec/joinerbase.go

Lines 230 to 232 in 74f6082

	}
	res, err := jb.onCond.EvalFilter(jb.Ctx(), combinedRow)
	if !res || err != nil {

https://github.com/cockroachdb/cockroach/blob/74f60821a82c570ab58b8d7716c1d74b85e87df0/pkg/sql/execinfrapb/pkg/sql/execinfrapb/expr.go#L95-L97
https://github.com/cockroachdb/cockroach/blob/74f60821a82c570ab58b8d7716c1d74b85e87df0/pkg/sql/execinfrapb/pkg/sql/execinfrapb/expr.go#L285-L287
https://github.com/cockroachdb/cockroach/blob/74f60821a82c570ab58b8d7716c1d74b85e87df0/pkg/sql/execinfrapb/pkg/sql/execinfrapb/expr.go#L42-L44

cockroach/pkg/sql/sem/eval/expr.go

Lines 20 to 22 in 74f6082

	func Expr(ctx context.Context, evalCtx *Context, n tree.TypedExpr) (tree.Datum, error) {
	return n.Eval(ctx, (*evaluator)(evalCtx))
	}

https://github.com/cockroachdb/cockroach/blob/74f60821a82c570ab58b8d7716c1d74b85e87df0/bazel-out/k8-opt/bin/pkg/sql/sem/tree/eval_expr_generated.go#L66-L68

cockroach/pkg/sql/sem/eval/expr.go

Lines 34 to 36 in 74f6082

	func (e *evaluator) EvalAndExpr(ctx context.Context, expr *tree.AndExpr) (tree.Datum, error) {
	left, err := expr.Left.(tree.TypedExpr).Eval(ctx, e)
	if err != nil {

https://github.com/cockroachdb/cockroach/blob/74f60821a82c570ab58b8d7716c1d74b85e87df0/bazel-out/k8-opt/bin/pkg/sql/sem/tree/eval_expr_generated.go#L116-L118

cockroach/pkg/sql/sem/eval/expr.go

Lines 261 to 263 in 74f6082

	}
	d, err := expr.Op.EvalOp.Eval(ctx, e, newLeft, newRight)
	if d == tree.DNull || err != nil {

https://github.com/cockroachdb/cockroach/blob/74f60821a82c570ab58b8d7716c1d74b85e87df0/bazel-out/k8-opt/bin/pkg/sql/sem/tree/eval_op_generated.go#L300-L302

cockroach/pkg/sql/sem/eval/binary_op.go

Lines 156 to 158 in 74f6082

	}
	cmp, err := left.Compare(ctx, e.ctx(), right)
	if err != nil {

cockroach/pkg/sql/sem/tree/datum.go

Lines 5340 to 5342 in 74f6082

	if v.EnumTyp.TypeMeta.Version != d.EnumTyp.TypeMeta.Version {
	panic(errors.AssertionFailedf(
	"comparison of two different versions of enum %s oid %d: versions %d and %d",

src/runtime/asm_amd64.s in runtime.goexit at line 1695
pkg/sql/colflow/vectorized_flow.go in pkg/sql/colflow.(*vectorizedFlowCreator).setupRouter.(*vectorizedFlowCreator).accumulateAsyncComponent.func2.1 at line 723
pkg/sql/colflow/vectorized_flow.go in pkg/sql/colflow.(*vectorizedFlowCreator).setupRouter.func1 at line 820
pkg/sql/colflow/routers.go in pkg/sql/colflow.(*HashRouter).Run at line 581
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 162
pkg/sql/colflow/routers.go in pkg/sql/colflow.(*HashRouter).Run.func1 at line 624
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 162
pkg/sql/colflow/routers.go in pkg/sql/colflow.(*HashRouter).Run.func1.1 at line 586
pkg/sql/colflow/routers.go in pkg/sql/colflow.(*HashRouter).processNextBatch at line 662
bazel-out/k8-opt/bin/pkg/sql/colexec/is_null_ops.eg.go in pkg/sql/colexec.(*isNullSelOp).Next at line 213
pkg/sql/colexec/colexecdisk/disk_spiller.go in pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next at line 195
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 162
pkg/sql/colexec/colexecdisk/disk_spiller.go in pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next.func1 at line 197
pkg/sql/colexec/colexecjoin/hashjoiner.go in pkg/sql/colexec/colexecjoin.(*hashJoiner).Next at line 305
pkg/sql/colexec/colexecjoin/hashjoiner.go in pkg/sql/colexec/colexecjoin.(*hashJoiner).exec at line 514
pkg/sql/colexec/columnarizer.go in pkg/sql/colexec.(*Columnarizer).Next at line 234
pkg/sql/rowexec/joinreader.go in pkg/sql/rowexec.(*joinReader).Next at line 781
pkg/sql/rowexec/joinreader.go in pkg/sql/rowexec.(*joinReader).emitRow at line 1173
pkg/sql/rowexec/joinreader_strategies.go in pkg/sql/rowexec.(*joinReaderNoOrderingStrategy).nextRowToEmit at line 293
pkg/sql/rowexec/joinerbase.go in pkg/sql/rowexec.(*joinerBase).render at line 231
pkg/sql/execinfrapb/pkg/sql/execinfrapb/expr.go in pkg/sql/execinfrapb.(*ExprHelper).EvalFilter at line 96
pkg/sql/execinfrapb/pkg/sql/execinfrapb/expr.go in pkg/sql/execinfrapb.(*exprHelper).evalFilter at line 286
pkg/sql/execinfrapb/pkg/sql/execinfrapb/expr.go in pkg/sql/execinfrapb.RunFilter at line 43
pkg/sql/sem/eval/expr.go in pkg/sql/sem/eval.Expr at line 21
bazel-out/k8-opt/bin/pkg/sql/sem/tree/eval_expr_generated.go in pkg/sql/sem/tree.(*AndExpr).Eval at line 67
pkg/sql/sem/eval/expr.go in pkg/sql/sem/eval.(*evaluator).EvalAndExpr at line 35
bazel-out/k8-opt/bin/pkg/sql/sem/tree/eval_expr_generated.go in pkg/sql/sem/tree.(*ComparisonExpr).Eval at line 117
pkg/sql/sem/eval/expr.go in pkg/sql/sem/eval.(*evaluator).EvalComparisonExpr at line 262
bazel-out/k8-opt/bin/pkg/sql/sem/tree/eval_op_generated.go in pkg/sql/sem/tree.(*CompareScalarOp).Eval at line 301
pkg/sql/sem/eval/binary_op.go in pkg/sql/sem/eval.(*evaluator).EvalCompareScalarOp at line 157
pkg/sql/sem/tree/datum.go in pkg/sql/sem/tree.(*DEnum).Compare at line 5341

Tags

Tag	Value
Command	server
Environment	v24.2.6
Go Version	go1.22.5 X:nocoverageredesign
Platform	linux amd64
Distribution	CCL
Cockroach Release	v24.2.6
Cockroach SHA	74f6082
# of CPUs	8
# of Goroutines	991

Jira issue: CRDB-45796

[blathers-crl]

CC'ing via the CODEOWNERS-based sentry heuristic:

• @cockroachdb/sql-foundations

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}