23.6.0

Various fixes & improvements

• ref(DRS): Deprecate dataset experimental flag (#4321) by @enochtangg
• Revert "fix: Temporarily revert dlq topic changes" (#4339) by @lynnagara
• fix(iam-policy): Don't build google cloud API client in debug and test environments (#4344) by @davidtsuk
• feat(prod-queries): Add preliminary UI for the production queries tool (#4345) by @davidtsuk
• ref(iam-policy): Replace user emails with groups in iam_policy (#4346) by @davidtsuk
• feat(spans): Add processors to extract millisecond granularity data (#4313) by @nikhars
• ref(gen_met): Add mapping optimizer to generic metrics (#4336) by @evanh
• Add optional Redis SSL support (#4343) by @frank-m
• feat(spans): Add migration to create start_ms and end_ms columns (#4308) by @nikhars
• ref(admin): Give team-sns google group full access to snuba admin (#4337) by @davidtsuk
• feat(CapMan): Multiple Allocation Policies on a Storage [backend] (#4294) by @rahul-kumar-saini
• chore(spans): Allow spans ingestion based on project sampling (#4214) by @nikhars
• fix: Temporarily revert dlq topic changes (#4338) by @lynnagara
• fix(tests): Skip flaky optimize tests (#4315) by @nikhars
• ref(dlq): Simplify the way DLQ topic is configured (#4333) by @lynnagara
• fix: All consumers have unique DLQ topics (#4335) by @lynnagara
• feat(dlq): DLQ topics API for admin (#4334) by @lynnagara
• Use Google Groups API for authorization (#4306) by @davidtsuk
• fix(dlq): Pass for to configure_metrics (#4329) by @lynnagara
• ref: Upgrade arroyo, and remove now-duplicated rdkafka.total_queue_size metric (#4330) by @untitaker
• fix(gen-metrics): Add tags hash column to the generic metrics dist ta… (#4324) by @nikhars
• ref: Set compression in build_kafka_producer_configuration() function (#4220) by @lynnagara
• fix(dlq): ExitAfterNMessages strategy should be first step not last (#4318) by @lynnagara
• feat(spans): Allow spans migrations to run locally (#4323) by @enochtangg

Plus 40 more

23.5.2

Various fixes & improvements

• feat(CapMan): Spans Allocation Policy (#4261) by @rahul-kumar-saini
• feat(CapMan): Allocation Policy UI tests (#4200) by @rahul-kumar-saini
• fix(CapMan): organzation_id -> organization_id (#4267) by @rahul-kumar-saini
• fix(DC): Outcomes Raw Entity was under wrong Dataset (#4263) by @rahul-kumar-saini
• reject search issues bad column (#4260) by @volokluev
• Final bypass for subscriptions (#4253) by @fpacifici
• feat(search-issues): expose and process profile_id and replay_id columns (#4164) by @barkbarkimashark
• fix: Fix flaky optimize test (#4252) by @untitaker
• fix(spans): use system instead of platform (#4244) by @dbanda
• promote profiles and functions to complete (#4238) by @enochtangg
• ref: bump sentry-kafka-schemas to 0.1.10 (#4250) by @getsentry-bot
• feat(CapMan): Most throttled Orgs visibility (#4248) by @rahul-kumar-saini
• feat(DRS): Enable readiness state for storages (#4240) by @enochtangg
• ref(dashboard): Add a setting that restricts the tools visible in the dashboard (#4174) by @evanh
• fix: Add querylog max.message.bytes to topic configuration (#4221) by @lynnagara
• fix(logs) : use structlog in migrations (#4236) by @dbanda
• feat(dlq): Consumer builder can build a DLQ strategy (#4235) by @lynnagara
• ref: bump sentry-arroyo to 2.11.4 (#4243) by @getsentry-bot
• ref: bump sentry-arroyo to 2.11.3 (#4241) by @getsentry-bot
• fix(parsing): Update error message for missing datetime column comparison condition (#4189) by @davidtsuk
• fix: Querylog producer (#4219) by @lynnagara
• fix: Fix attribution producer (#4218) by @lynnagara
• test(dlq): Add a small test for ExitAfterNMessages strategy (#4234) by @lynnagara
• feat(dlq): Add is_valid() method to DlqInstruction class (#4233) by @lynnagara

Plus 12 more

23.5.1

Various fixes & improvements

• ref(process_message): Tag all metrics by consumer_group (#4226) by @untitaker
• feat: Default join timeout for all consumers of 5 seconds (#4222) by @lynnagara
• fix(clickhouse): Add a check for the max supported clickhouse version (#4158) by @davidtsuk
• fix: Fix flaky optimize test (#4217) by @lynnagara
• fix: Fix DLQ producer config (#4216) by @lynnagara
• ref: bump sentry-arroyo to 2.11.2 (#4215) by @getsentry-bot
• feat: Move DLQ configuration into consumer configuration (#4206) by @lynnagara
• feat: Python and Rust consumers share common cluster resolution code (#4205) by @lynnagara
• Fix slack notifications for when allocation policy changes (#4204) by @volokluev
• ref: bump sentry-kafka-schemas to 0.1.9 (#4207) by @getsentry-bot
• feat(admin): add amount of rows in result set to tracing tool (#4167) by @volokluev
• feat(capman): Allow overriding policy defaults in configuration (#4201) by @volokluev
• fix: All consumers run with --no-strict-offset-reset in dev (#4177) by @lynnagara
• fix(CapMan): Validate Tenant IDs (#4176) by @rahul-kumar-saini
• ref: Use RetryingRedisCluster from sentry-redis-tools (#4197) by @untitaker
• ref(consumer): pass sentry_received_timestamp again (#4198) by @MeredithAnya
• feat(dlq): Add the DLQ instruction mechanism (#4199) by @lynnagara
• fix(capman): create auditlog notifications for allocation policy updates (#4193) by @volokluev
• feat(schema-validation): Validate all messages 😱 (#4194) by @lynnagara
• Revert "ref(consumer): Pass sentry_received_timestamp for e2e latency (#4052)" (0d1b0cf) by @getsentry-bot
• ref(consumer): Pass sentry_received_timestamp for e2e latency (#4052) by @ayirr7
• feat(profiling): Create profiling datasets by default in self-hosted (#4195) by @phacops
• feat: Add metric to count validation failures [SNS-2279] (#4182) by @untitaker
• ref: bump sentry-arroyo to 2.11.1 (#4190) by @getsentry-bot

Plus 10 more

23.5.0

Various fixes & improvements

• Pin action-github-commit (#4175) by @chadwhitacre
• ref(tech-debt) Column validator also checks mapped columns (#4116) by @evanh
• Make kafka consumer max poll time configurable (#4165) by @nikhars
• ref: bump sentry-redis-tools to 0.1.5 (#4170) by @getsentry-bot
• ref: bump sentry-kafka-schemas to 0.1.7 (#4168) by @getsentry-bot
• fix: Fix another reference to unavailable secret (#4161) by @untitaker
• Revert "drop old tables (#3896)" (ee62ee0) by @getsentry-bot
• drop old tables (#3896) by @barkbarkimashark
• fix(consumer): Remove DLQ policy for generic metrics consumers config (#4166) by @ayirr7
• fix(consumers): Add timeout for http batch writes (#4160) by @nikhars
• chore(arroyo): Bump version to 2.10.4 (#4162) by @nikhars
• feat(CapMan): AllocationPolicy Config API (#4025) by @rahul-kumar-saini
• feat(functions): Process new functions message (#4056) by @Zylphrex
• fix: Fix permissions of bump-version.yml workflow (#4157) by @untitaker
• feat(CapMan): Basic Snuba Admin UI (#4055) by @rahul-kumar-saini
• ref: Arroyo 2.10.3 (#4153) by @untitaker
• add test referrer to single thread referrers (#4154) by @volokluev
• fix: Unknown functions shouldn't count against the SLO (#4150) by @evanh
• feat(ci): allow skip check migrations (#4111) by @dbanda
• Revert "turn on policy, add user report referrer (#4151)" (21864c3) by @getsentry-bot
• turn on policy, add user report referrer (#4151) by @volokluev
• ref(admin): Give more users access to the admin tool (#4149) by @evanh
• build: sentry-kafka-schemas 0.1.6 (#4147) by @lynnagara
• test: Fix flaky optimize test (#4143) by @lynnagara

Plus 89 more

23.4.0

Various fixes & improvements

• fix(CI): add migrations check to main pipeline (attempt 2) (#4019) by @dbanda
• fix: Add killswitch to disable raising of InvalidMessage (#4023) by @untitaker
• add a manually triggered migrations stage to prod gocd (#4014) by @dbanda
• feat(search-issues): process transaction_duration values (#4008) by @barkbarkimashark
• Revert "feat(CI): add migrations check to main pipeline (#4013)" (b59bb7e) by @getsentry-bot
• build: Arroyo 2.10.1 (#4017) by @untitaker
• feat(CI): add migrations check to main pipeline (#4013) by @dbanda
• feat(CapMan): Errors Allocation Policy V0 (fixed) (#4016) by @volokluev
• ref: Make all errors replacements class-based (#4009) by @untitaker
• Revert "feat(CapMan): AllocationPolicy for Errors (#3999)" (#4015) by @volokluev
• add optional transaction_duration column (#4007) by @barkbarkimashark
• feat(CapMan): AllocationPolicy for Errors (#3999) by @volokluev
• build: sentry-kafka-schemas 0.0.28 (#4005) by @lynnagara
• feat: DLQ for querylog (#4006) by @lynnagara
• fix(search-issues): make resource_id, subtitle, culprit, and level fields available in entity (#3995) by @barkbarkimashark
• build(deps): bump h2 from 0.3.16 to 0.3.17 in /rust_snuba (#4010) by @dependabot
• fix(admin) Fix regex to allow SelectExecutor tracing lines (#3983) by @evanh
• feat: Use new DLQ from Arroyo (#4001) by @lynnagara
• ref: Make delete_groups replacement class-based (#3996) by @untitaker
• build: Arroyo 2.10.0 (#4003) by @lynnagara
• fix: update validation from org_countries for granularity (#4002) by @andriisoldatenko
• fix: Fix type issue (#4000) by @lynnagara
• ref(ci): add auto labeling for migrations (#3961) by @MeredithAnya
• build(deps): bump sentry-arroyo[json] from 2.8.0 to 2.9.1 (#3988) by @dependabot

Plus 90 more

23.3.1

Various fixes & improvements

• add file for self hosted settings (#3889) by @enochtangg
• ci: Run Rust linter (#3884) by @lynnagara
• Add sns gcloud project GoCD pipeline (#3850) by @dbanda
• rust: add simple transform and produce strategy example (#3796) by @dbanda
• ref: Rename positions to offsets everywhere (#3888) by @lynnagara
• feat(rust-arroyo): Split the message interface to support batching (#3885) by @lynnagara
• feat(tx_processor): write replay_id as top level column (#3854) by @JoshFerge
• feat(rust-arroyo): Get rid of Position (#3883) by @lynnagara
• Remove filtering only local nodes for tracing tool (#3876) by @enochtangg
• fix: Add sentry-compatible alias to install-python-dependencies (#3875) by @untitaker
• fix(replays): set transaction as empty string instead of null (#3878) by @JoshFerge
• feat(schemas): Partially type the data in querylog producer (#3858) by @lynnagara
• feat: The Rust consumer calls Python message processors (#3871) by @untitaker
• feat(rust-consumer): Avoid config files overwriting each other (#3873) by @lynnagara
• ref: Bump sentry-kafka-schemas 0.0.9 (#3882) by @marandaneto
• ref(admin): add CreateTableQuery predefined queries (#3844) by @MeredithAnya
• feat(rust-consumer): Add python processor info to consumer config (#3874) by @lynnagara
• feat(rust-consumer): Remove option to build raw config (#3870) by @lynnagara
• feat: Start building the config for Rust consumer (#3869) by @lynnagara
• Revert "use redis username from env (#3862)" (a03fb44) by @getsentry-bot
• use redis username from env (#3862) by @HydrofinLoewenherz
• feat(rust-consumer): Multistorage consumer (#3865) by @lynnagara
• fix(rust-consumer): Remove auto-initialize feature (#3868) by @lynnagara
• feat(Querylog): Organization ID from tenant_ids again (#3867) by @rahul-kumar-saini

Plus 2 more

23.3.0

Various fixes & improvements

• Revert "feat(Querylog): Organization ID from tenant_ids (#3857)" (a555678) by @getsentry-bot
• feat(Querylog): Organization ID from tenant_ids (#3857) by @rahul-kumar-saini
• feat(rust-consumer): Actually parse Python settings in Rust (#3860) by @lynnagara
• feat(rust-consumer): Parse Python settings in Rust (#3859) by @lynnagara
• ref(admin): Log when a connection error happens (#3834) by @evanh
• feat: Share Python settings with Rust and add Rust consumer entrypoint (#3856) by @lynnagara
• use correct connection for multi-node clusters (#3846) by @dbanda
• meta: Update post merge hook to warn about updated deps (#3851) by @lynnagara
• ref(admin): dont use tracing user for querylog and remove readonly=2 (#3849) by @MeredithAnya
• ref(schema) Update to 0.0.6 (#3843) by @evanh
• fix(querylog): enable readonly 2 for querylog (#3847) by @MeredithAnya
• fix(test): Add extra query editor test (#3841) by @john-z-yang
• fix(migrations): add replay_id and exception_main_thread to errors dist ro (#3840) by @MeredithAnya
• test: Run all consumers' message processors against examples (#3842) by @untitaker
• ref: Add typing to all messages in errors replacer (#3835) by @untitaker
• feat(replays): Default event_hash when segment_id is null (#3759) by @cmanallen
• remove dead code from web/query (#3832) by @volokluev
• ref: Bump sentry-kafka-schemas to 0.0.5 (#3839) by @untitaker
• ref(admin) enable events/transactions migration groups (#3838) by @MeredithAnya
• ref: Update mypy to 1.1.1 (#3836) by @untitaker
• feat(querylog): Support for query interpolation in Querylog query editor (#3803) by @john-z-yang
• feat: Add new statuses for SLO (#3747) by @evanh
• ref: Start using python types from sentry-kafka-schemas (#3812) by @untitaker
• fix(CapMan): Ensure tenant_ids metric is accurate (#3831) by @rahul-kumar-saini

Plus 59 more

23.2.0

Various fixes & improvements

• fix: Reject improperly typed tag conditions as invalid queries (#3727) by @evanh
• ref(batch-writer): Remove unused terminate() method (#3744) by @lynnagara
• allow strings in select statements on system queries (#3745) by @volokluev
• ref: InsertBatchWriter and ReplacementBatchWriter don't subclass ProcessingStrategy (#3743) by @lynnagara
• add support for enum column type (#3729) by @enochtangg
• change search_issues migrations to non-blocking (#3739) by @barkbarkimashark
• fix: Rename consumer group tag (#3740) by @lynnagara
• fix ui bug where we dont properly refresh table after running (#3733) by @dbanda
• change log error to warn (#3734) by @dbanda
• ref: bump timeout slightly (#3738) by @asottile-sentry
• ref: force ipv4 localhost (#3736) by @asottile-sentry
• add generic metric consumers to freight (#3737) by @enochtangg
• fix(admin): allow querying dist nodes too (#3731) by @dbanda
• Revert "feat(admin): allow querylog to be queried by more threads (#3718)" (#3732) by @volokluev
• migrations: add custom search issues role (#3724) by @dbanda
• docs(clickhouse): describe basic Sentry CH usage (#3701) by @onewland
• ref(migrations): Make ADMIN_ALLOWED_MIGRATION_GROUPS a set (#3720) by @MeredithAnya
• feat(admin): allow querylog to be queried by more threads (#3718) by @volokluev
• feat(replays): add replay_id to transactions table (#3721) by @JoshFerge
• feat(replays): add replay_id to errors table (#3722) by @JoshFerge
• Remove experimental flag from replays (#3726) by @cmanallen
• fix: Fix typing of multistorage consumer (#3719) by @lynnagara
• ref(consumer): Build strategy factory function in multistorage consumer (#3714) by @ayirr7
• fix(cli): Allow more options for datasets and entities (#3715) by @nikhars

Plus 64 more

23.1.1

Various fixes & improvements

• ref(DC): Remove Generic Metrics Python (#3650) by @rahul-kumar-saini
• ref(config): Create Replays Entity yaml file (#3643) by @enochtangg
• ref(config) Create Profiles Entity yaml file (#3642) by @enochtangg
• feat(DC): create events entity, expand testing, fix transactions yaml (#3645) by @volokluev
• refDC): Extract session translation mappers (#3636) by @volokluev
• ref: remove yarn apt list after installation (#3648) by @asottile-sentry
• feat(api-abuse): Add triggered rate limiter name to Querylog stats (#3625) by @enochtangg
• migrations: make querylog match SaaS (#3611) by @dbanda
• build: Arroyo 2.5.0 (#3635) by @lynnagara
• feat: Clean up unused topic (#3634) by @lynnagara
• ref(DC): Add FixedString field to configuration (#3630) by @volokluev
• feat(config): Migrate GroupedMessage and GroupAssignee Entities to YAML (#3616) by @evanh
• ref(migrations): Simplfy policies and status checks (#3606) by @MeredithAnya
• fix(outcomes): Add TTL to outcomes dataset (#3615) by @nikhars
• ref(copy-tables): fix regex (#3631) by @MeredithAnya
• feat(profiling): Add profile id to transactions table (#3607) by @Zylphrex
• ref(config) Migrate Outcomes and OutcomesRaw entities to YAML (#3632) by @evanh
• feat: Tag invalid messages for DLQ with invalid_message: true (#3633) by @lynnagara
• ref: Remove invalid TODO (#3624) by @lynnagara
• feat: Remove ability to override scheduling mode (#3623) by @lynnagara
• feat(api-abuse): Add timing metrics for table_concurrent (#3628) by @enochtangg
• ref: Remove debugging code (#3621) by @lynnagara
• add settings for codecov (#3627) by @volokluev
• migrations: make errors schema match SaaS (#3604) by @dbanda

Plus 18 more

23.1.0

Various fixes & improvements

• feat(DC): Add simple Dataset configs (#3586) by @rahul-kumar-saini
• bug(DC): Add config built storage keys to all_storages dictionary in factory (#3502) by @enochtangg
• bump search_issues storage occurrence_type_id int size (#3593) by @enochtangg
• feat: Change settings.TOPIC_PARTITION_COUNTS to use logical topic name (#3589) by @lynnagara
• feat: Arroyo 2.4.0 (#3585) by @lynnagara
• feat(admin): implement basic admin user roles (#3522) by @MeredithAnya
• fix: Change DefaultNoneColumnMapper to use a normal set (#3580) by @evanh
• update migrations doc to reflect new style (#3584) by @dbanda
• fix(DC): Bring EntityContainsColumnsValidator back to config entities (#3579) by @volokluev
• feat(slices): Make Subscription Scheduler filter by slice ID (#3338) by @ayirr7
• Revert "fix(DC): Bring EntityContainsColumnsValidator back to config entities (#3540)" (#3578) by @volokluev
• Pick up self-hosted CI bugfix (#3577) by @chadwhitacre
• fix(DC): Bring EntityContainsColumnsValidator back to config entities (#3540) by @volokluev
• feat(oncall): add the request_id to the query breadcrumbs (#3561) by @volokluev
• feat(DC): Add Storage selector support for Entity configuration (#3545) by @enochtangg
• feat(issue-platform): increase occurrence_type_id column size from UInt8 to UInt16 (#3575) by @barkbarkimashark
• feat(issue-platform): process extra fields from event (#3571) by @barkbarkimashark
• docs(schema): why do we promote tags? (#3574) by @onewland
• Remove sorting experiment on sql conditions (#3567) by @enochtangg
• ref: Add docstring about topic_name vs get_physical_topic_name. (#3568) by @lynnagara
• feat(ops): allow disabled entities for storage sets that are not in use (#3572) by @onewland
• test(consumer): Add preliminary unit tests to ConsumerBuilder (#3389) by @ayirr7
• docs(slicing): Add mega-cluster and basic consumer setup (#3433) by @ayirr7
• feat(consumer): Remove parallel collect option (#3555) by @lynnagara

Plus 39 more