Fix parquet-reader outputting invalid json #72
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Example output without this:
{ "FileName": "/home/ubuntu/ClickHouse/tests/queries/0_stateless/data_parquet/68131.parquet", "Version": "1.0", "CreatedBy": "parquet-mr version 1.12.2 (build 77e30c8093386ec52c3cfa6c34b7ef3321322c94)", "TotalRows": "1", "NumberOfRowGroups": "1", "NumberOfRealColumns": "1", "NumberOfColumns": "1", "Columns": [ { "Id": "0", "Name": "f", "PhysicalType": "INT32", "ConvertedType": "NONE", "LogicalType": {"Type": "None"} } ], "RowGroups": [ { "Id": "0", "TotalBytes": "43", "TotalCompressedBytes": "43", "Rows": "1", "ColumnChunks": [ {"Id": "0", "Values": "2", "StatsSet": "True", "Stats": {"NumNulls": "0", "Max": "2", "Min": "1" }, "Compression": "UNCOMPRESSED", "Encodings": "PLAIN", "UncompressedSize": "43", "CompressedSize": "43", ColumnIndex {"offset": "47", "length": "23"}", OffsetIndex {"offset": "70", "length": "10"}" } ] } ] }It goes off the rails starting at
ColumnIndex.ClickHouse uses this tool in test
00900_long_parquet_load(ran manually to get table schemas).