Skip to content

Comments

Add support for BM25 search#6

Merged
dimamik merged 4 commits intomainfrom
feat/bm-25
Feb 5, 2026
Merged

Add support for BM25 search#6
dimamik merged 4 commits intomainfrom
feat/bm-25

Conversation

@dimamik
Copy link
Owner

@dimamik dimamik commented Feb 4, 2026

This pull request introduces BM25 full-text search support to the project, leveraging the pg_textsearch extension for PostgreSQL 17+. It adds the new Torus.bm25/5 macro for state-of-the-art relevance ranking, updates documentation and tests, and ensures the CI pipeline and test environment are compatible with the new requirements. The most important changes are grouped below.

BM25 Full-Text Search Feature:

  • Added the Torus.bm25/5 macro in lib/torus.ex for BM25-ranked full-text search, including comprehensive documentation, usage examples, and migration guidance.
  • Implemented the BM25 query logic in the new module lib/torus/search/bm25.ex, supporting options like ordering, score selection, pre-filtering, and score thresholding.

Documentation and Guides:

  • Updated README.md and CHANGELOG.md to document the new BM25 search capability, its requirements, usage scenarios, and how it compares to existing full-text search. [1] [2] [3] [4]
  • Bumped the project version to 0.6.0 in mix.exs and included the changelog in the documentation extras. [1] [2]

CI and Test Environment Updates:

  • Upgraded the CI pipeline to use PostgreSQL 17 and Elixir 1.19.5/OTP 28.3, and added installation of the pg_textsearch extension for testing. [1] [2] [3]
  • Added a migration for test environments to install pg_textsearch and create BM25 indexes on relevant columns.
  • Updated test exclusion tags to support new tests that require CI to be skipped.

Testing and Validation:

  • Added compile-time validation tests for the bm25/5 macro to ensure correct option usage, especially around score_threshold and index_name.

@xu-chris
Copy link

xu-chris commented Feb 5, 2026

Thinking Elixir fans jump on for support, brilliant! 🙏
I simply support this effort!

@dimamik dimamik merged commit a72d9f8 into main Feb 5, 2026
1 check passed
@dimamik dimamik deleted the feat/bm-25 branch February 5, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants