feat: implement v2.0 cognitive layers (Phases 10.5-14) #10
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Phase 13: Outbox Index Ingestion - Standard stack identified (memory-indexing crate) - Architecture patterns documented (unified consumer, checkpoints) - Pitfalls catalogued (blocking async, checkpoint timing) - Code examples from existing codebase patterns - 4 plan breakdown recommended Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Phase 13: Outbox Index Ingestion - 4 plans in 3 waves - Wave 1: Outbox consumer infrastructure (13-01) - Wave 2: Incremental updates (13-02) and rebuild command (13-03) in parallel - Wave 3: Scheduler integration (13-04) - Ready for execution Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Phase 15: Configuration Wizard Skills - Standard stack identified (AskUserQuestion, SKILL.md patterns) - Architecture patterns documented (state detection, flag modes, output formatting) - Pitfalls catalogued (state detection, platform paths, header length) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
## Overview This PR implements the complete v2.0 feature set for Agent Memory, adding 5 new crates and 578 tests. The implementation follows the 6-layer cognitive architecture where indexes are accelerators, not dependencies. ## New Crates (11,208 LOC) ### memory-search (1,470 LOC) - Tantivy BM25 full-text search integration - Document schema for TOC nodes and grips - Configurable tokenization and ranking ### memory-embeddings (571 LOC) - Candle ML framework integration - all-MiniLM-L6-v2 model (384-dim embeddings) - Model caching in ~/.cache/memory-embeddings ### memory-vector (1,381 LOC) - usearch HNSW vector index wrapper - Configurable M, ef_construction, ef_search params - Thread-safe RwLock-protected index access ### memory-indexing (2,584 LOC) - Unified outbox consumer pipeline - Checkpoint-based crash recovery - BM25 and vector index updaters - Full rebuild support via admin command ### memory-topics (5,202 LOC) - HDBSCAN density-based clustering - LLM labeling with TF-IDF keyword fallback - Exponential time-decay importance scoring (30-day half-life) - Topic relationships: SIMILAR, PARENT, CHILD, SEQUENTIAL - Lifecycle management: pruning, resurrection ## Phase Breakdown ### Phase 10.5: Agentic TOC Search - SearchNode RPC: search within a single node - SearchChildren RPC: recursive child search - term_overlap_score() for index-free matching - CLI: `query search --node <id> --query <terms>` ### Phase 11: BM25 Teleport (Tantivy) - TeleportSearch RPC: keyword search with BM25 ranking - GetTeleportStatus RPC: index availability check - Background index commit job - CLI: `query teleport-search --query <keywords>` ### Phase 12: Vector Teleport (HNSW) - VectorTeleport RPC: semantic similarity search - HybridSearch RPC: RRF fusion (k=60) of BM25+vector - GetVectorIndexStatus RPC: index stats - CLI: `query vector-teleport`, `query hybrid-search` ### Phase 13: Outbox Index Ingestion - IndexingPipeline: processes outbox entries - Per-index checkpoints for crash recovery - Incremental updates (add/update documents) - Admin command: `admin rebuild-indexes` - Scheduled background job via Phase 10 scheduler ### Phase 14: Topic Graph Memory - HDBSCAN clustering on TOC node embeddings - LLM-generated labels with keyword fallback - GetTopicGraphStatus, GetTopicsByQuery RPCs - GetRelatedTopics, GetTopTopics RPCs - Lifecycle: pruning dormant topics, resurrection - CLI: `query topics`, `admin topic-prune` ## Proto Changes (+364 lines) New RPCs added to MemoryService: - SearchNode, SearchChildren - TeleportSearch - VectorTeleport, HybridSearch, GetVectorIndexStatus - GetTopicGraphStatus, GetTopicsByQuery - GetRelatedTopics, GetTopTopics New enums: - SearchField (TITLE, SUMMARY, BULLETS, KEYWORDS) - TeleportDocType (TOC_NODE, GRIP) - VectorTargetType - TopicRelationshipType ## CI/CD Added GitHub Actions workflows: - ci.yml: format, clippy, test, build, doc checks - release.yml: multi-platform release builds ## Documentation Updates - ARCHITECTURE.md: Added new crate structure and dependency graph - API.md: Added all new gRPC RPCs with examples - COGNITIVE_ARCHITECTURE.md: Marked all layers complete - Wiki synchronized with GitHub wiki ## Test Summary - 578 tests total - All crates have comprehensive unit tests - Integration tests for cross-crate functionality - Doc tests for public APIs Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The action is dtolnay/rust-toolchain, not dtolnay/rust-action. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Helps avoid runner queue congestion on ubuntu-latest. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements the complete v2.0 feature set for Agent Memory, adding 5 new crates and 578 tests. The implementation follows the 6-layer cognitive architecture where indexes are accelerators, not dependencies.
New Crates (11,208 LOC total)
memory-searchmemory-embeddingsmemory-vectormemory-indexingmemory-topicsPhases Implemented
Key Features
Proto Changes (+364 lines)
New RPCs: SearchNode, SearchChildren, TeleportSearch, VectorTeleport, HybridSearch, GetVectorIndexStatus, GetTopicGraphStatus, GetTopicsByQuery, GetRelatedTopics, GetTopTopics
CI/CD
ci.yml: format, clippy, test, build, doc checksrelease.yml: multi-platform release buildsTest Plan
cargo test --workspace)cargo clippy --workspace --all-targets --all-features -- -D warnings)Documentation
🤖 Generated with Claude Code