-
Notifications
You must be signed in to change notification settings - Fork 572
DecompressedSize added to Resource table #5274
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
src/Microsoft.Health.Fhir.SqlServer/Features/Schema/Sql/Sprocs/MergeResources.sql
Outdated
Show resolved
Hide resolved
src/Microsoft.Health.Fhir.SqlServer/Features/Schema/Sql/Sprocs/MergeResources.sql
Outdated
Show resolved
Hide resolved
src/Microsoft.Health.Fhir.SqlServer/Features/Schema/Migrations/102.diff.sql
Show resolved
Hide resolved
src/Microsoft.Health.Fhir.SqlServer/Features/Schema/Migrations/102.diff.sql
Outdated
Show resolved
Hide resolved
src/Microsoft.Health.Fhir.SqlServer/Features/Schema/Sql/Tables/Resource.sql
Outdated
Show resolved
Hide resolved
src/Microsoft.Health.Fhir.SqlServer/Features/Schema/Sql/Types/ResourceList_Temp.sql
Outdated
Show resolved
Hide resolved
src/Microsoft.Health.Fhir.SqlServer/Features/Schema/SchemaVersionConstants.cs
Outdated
Show resolved
Hide resolved
|
@rajithaalurims I think this PR is a prerequisite to address issue https://microsofthealth.visualstudio.com/Health/_workitems/edit/156583 |
src/Microsoft.Health.Fhir.SqlServer/Features/Schema/SchemaVersionConstants.cs
Outdated
Show resolved
Hide resolved
| ,IsRawResourceMetaSet bit NOT NULL | ||
| ,RequestMethod varchar(10) NULL | ||
| ,SearchParamHash varchar(64) NULL | ||
| ,DecompressedLength INT NULL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lower case
| Parameter table entries: | ||
| - FHIR_TotalDataSize: Stores ( total ingested data size + total index size) in GB | ||
| - FHIR_TotalIndexSize: Stores total index size in GB | ||
| - Both entries include timestamp of last calculation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Can we rename the entries to FHIR_TotalDataSizeInGB and FHIR_TotalIndexSizeInGB
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
| ### Consequences | ||
| - Background job adds periodic database load every 4 hours | ||
| - Failure in job does not impact core FHIR server functionality | ||
| - Falure in job results in stale data size metrics until next successful run |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume we would emit failure logs from the background job. Would we need to setup any alerting/monitoring if the background job fails?
Do you think if its useful to add a section for alerts/monitors if needed at all for this design spec.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great question. I have added alerting and monitoring in the ADR I put in place on WP side. If there are no metrics being emitted as expected we do get ICMs.
| TransactionId bigint NULL, | ||
| HistoryTransactionId bigint NULL | ||
| HistoryTransactionId bigint NULL, | ||
| DecompressedLength int NULL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand this is nullable for backward compatibility. Do we have a test coverage to ensure we don't pass the null value/0 value for these changes, if it makes sense?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For iteration1 null and 0 are acceptable. After iteration 2, null values are not acceptable. Once all three iterations are done, we should not have any null values in that table. I can add test coverage in iteration 2 or 3 to make sure there are no null values.
Description
A new nullable column DecompressedSize added to Resource table.
A new table type is created to support backward and forward compatibility.
Stored procedures are changed to use data from either old or new data type depending on schema version supported on the instance.
Related issues
Addresses [issue #156583].
Testing
Describe how this change was tested.
FHIR Team Checklist
Semver Change (docs)
Patch|Skip|Feature|Breaking (reason)