feat(eap): Add timestamp_ns column to eap_items_1 table #7671
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a new UInt16 column `timestamp_ns` to the eap_items_1 table for storing sub-second timestamp precision. The column is added after the existing `timestamp` column and included in the sort key. Co-Authored-By: Claude <noreply@anthropic.com>
|
This PR has a migration; here is the generated SQL for -- start migrations
-- forward migration events_analytics_platform : 0052_add_timestamp_ns_column
Local op:
ALTER TABLE eap_items_1_local
ADD COLUMN IF NOT EXISTS timestamp_ns UInt16 AFTER timestamp,
MODIFY ORDER BY (organization_id, project_id, item_type, timestamp, timestamp_ns, trace_id, item_id)
Distributed op: ALTER TABLE eap_items_1_dist ON CLUSTER 'cluster_one_sh' ADD COLUMN IF NOT EXISTS timestamp_ns UInt16 AFTER timestamp;
-- end forward migration events_analytics_platform : 0052_add_timestamp_ns_column
-- backward migration events_analytics_platform : 0052_add_timestamp_ns_column
-- end backward migration events_analytics_platform : 0052_add_timestamp_ns_column |
volokluev
reviewed
Jan 26, 2026
snuba/snuba_migrations/events_analytics_platform/0052_add_timestamp_ns_column.py
Outdated
Show resolved
Hide resolved
volokluev
reviewed
Jan 28, 2026
Comment on lines
44
to
57
| operations.DropColumn( | ||
| storage_set=self.storage_set_key, | ||
| table_name=self.dist_table_name, | ||
| column_name="timestamp_ns", | ||
| target=OperationTarget.DISTRIBUTED, | ||
| ), | ||
| operations.RunSql( | ||
| storage_set=self.storage_set_key, | ||
| statement=f""" | ||
| ALTER TABLE {self.local_table_name} | ||
| MODIFY ORDER BY (organization_id, project_id, item_type, timestamp, trace_id, item_id) | ||
| """, | ||
| target=OperationTarget.LOCAL, | ||
| ), |
Member
There was a problem hiding this comment.
would this actually work? if you drop the column, the order by would break maybe?
Contributor
Author
There was a problem hiding this comment.
We can't modify the order by again since we'd modify it with columns that contain data and it might change the order of things on disk, the actual backwards operation here would be to stop inserting in timestamp_ns.
I removed the backwards operation.
phacops
force-pushed
the
feat/add-timestamp-ns-column
branch
2 times, most recently
from
5735acb to
51b6b0c
Compare
Replace RunSql with AddColumn operation for adding the timestamp_ns column to the distributed table. This uses the standard migration operation instead of raw SQL. Note: This migration is non-reversible due to ClickHouse limitations. ClickHouse does not allow removing a column from the middle of a sorting key via ALTER TABLE MODIFY ORDER BY. Co-Authored-By: Claude <noreply@anthropic.com>
phacops
force-pushed
the
feat/add-timestamp-ns-column
branch
from
51b6b0c to
e0ac45e
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add a new
timestamp_nscolumn of typeUInt16to theeap_items_1table for storing sub-second timestamp precision.The migration:
timestamp_nscolumn after the existingtimestampcolumntimestamp_nsaftertimestamp:(organization_id, project_id, item_type, timestamp, timestamp_ns, trace_id, item_id)