docs(datafusion): add user documentation for DataFusion integration #2029
+352
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add comprehensive user documentation for the DataFusion integration that covers SQL-based table operations, catalog integration, and query optimization features. Changes: - Add datafusion.md documentation page with setup, SQL operations, metadata tables, partitioned tables, and configuration options - Add datafusion_integration.rs example with annotated code sections - Update SUMMARY.md to include new documentation page - Add required dependencies to examples crate Closes apache#2027
xbattlax
force-pushed
the
docs_datafusion_integration
branch
from
853219a to
4830a74
Compare
- Change catalog name from "iceberg" to "my_catalog" to clarify it has no special meaning - Move external table code to separate example file - Remove sections not suited for end-users: External Tables, Table Provider Types, Creating Partitioned Tables (Rust API), Query Optimization - Add clarification that table properties must be set via Iceberg catalog API
Contributor
Author
xbattlax
force-pushed
the
docs_datafusion_integration
branch
3 times, most recently
from
68f9f95 to
6cb8fbd
Compare
xbattlax
force-pushed
the
docs_datafusion_integration
branch
from
6cb8fbd to
7f6bfda
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add comprehensive user documentation for the DataFusion integration, addressing issue #2027.
Changes
Add
website/src/datafusion.mddocumentation page covering:IcebergCatalogProviderCREATE TABLE,INSERT INTO,SELECT$snapshots,$manifests)IcebergTableProviderFactoryAdd
crates/examples/src/datafusion_integration.rsworking example with annotated code sections for documentation includesUpdate
website/src/SUMMARY.mdto include new documentation pageAdd required dependencies to
crates/examples/Cargo.tomlNotes
This addresses the documentation request from #2027, which noted substantial progress in DataFusion integration (per epic #1382). The documentation covers all major features merged including:
CREATE TABLEfor DataFusion #1972)insert_intoforIcebergTableProvider#1600, feat(datafusion): SupportINSERT INTOpartitioned tables #1827)Closes #2027