Skip to content

feat: Add correlation threshold filtering and maximum limit for retrieved memories#1006

Merged
CaralHsi merged 51 commits intoMemTensor:dev-20260202-v2.0.5from
hijzy:dev
Feb 6, 2026
Merged

feat: Add correlation threshold filtering and maximum limit for retrieved memories#1006
CaralHsi merged 51 commits intoMemTensor:dev-20260202-v2.0.5from
hijzy:dev

Conversation

@hijzy
Copy link
Collaborator

@hijzy hijzy commented Feb 4, 2026

Description

Correlation Threshold Filtering:
Add a configurable correlation threshold parameter (float, default: 0.0)
If threshold > 0: filter out memories with correlation scores below the threshold
If threshold = 0: return all memories without filtering (backward compatible)

Maximum Retrieval Limit:
Add a configurable maximum number of memories to return (int, default: unlimited/very high)
When retrieving memories, return at most this number of highest-scoring memories

Related Issue (Required): #1005

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

  • Test Script Or Test Steps (please provide)
data = {
  "query": "我国庆想出去玩,帮我推荐个没去过的城市,以及没住过的酒店品牌",
  "user_id": "memos_user_123",
  "conversation_id": "0928",
  "relativity": 0.7
}
headers = {
  "Content-Type": "application/json",
  "Authorization": f"Token {os.environ['MEMOS_API_KEY']}"
}
url = f"{os.environ['MEMOS_BASE_URL']}/search/memory"

res = requests.post(url=url, headers=headers, data=json.dumps(data))

print(f"result: {res.json()}")

Checklist

  • I have performed a self-review of my own code | 我已自行检查了自己的代码
  • I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释
  • I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常
  • I have created related documentation issue/PR in MemOS-Docs (if applicable) | 我已在 MemOS-Docs 中创建了相关的文档 issue/PR(如果适用)
  • I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR(如果适用)
  • I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人

Reviewer Checklist

  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Made sure Checks passed
  • Tests have been provided

@hijzy hijzy marked this pull request as ready for review February 4, 2026 03:02
@CaralHsi CaralHsi merged commit 2550b93 into MemTensor:dev-20260202-v2.0.5 Feb 6, 2026
16 checks passed
anatolykoptev added a commit to anatolykoptev/MemOSina that referenced this pull request Feb 8, 2026
…xt similarity helpers

Adds:
- `relativity` field on APISearchRequest (default 0 for PolarDB compat —
  PolarDB scores are (cosine+1)/2 ≈ 0.5, upstream default 0.57 would
  filter everything)
- `_apply_relativity_threshold()` — post-search quality filter
- `include_embedding` parameter pushed down to tree.py search()
  (cleaner than creating new searcher in handler)
- `_dice_similarity()`, `_bigram_similarity()`, `_tfidf_similarity()`
  static methods for text dedup
- Fix: pref_buckets key "preference" → "pref_mem"
- Similarity threshold 0.9 → 0.92

Cherry-picked from 2550b93 (MemTensor#1006)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
anatolykoptev added a commit to anatolykoptev/MemOSina that referenced this pull request Feb 8, 2026
- Remove duplicate _dice/_bigram/_tfidf_similarity methods from
  search_handler.py (97 lines, artifact of MemTensor#1006 cherry-pick)
- Remove unused escape_sql_string() from polardb/helpers.py
- Remove unused seach_by_keywords_* typo aliases from polardb/search.py
- Clean up drop_database() unreachable code in polardb/maintenance.py
- Remove commented-out sleep blocks in polardb/connection.py
- Make heavy ML imports optional in evaluation scripts (tiktoken,
  sentence_transformers, bert_score)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants