Skip to content

Auto-Ingestion Watch Daemon #7

@ashu17706

Description

@ashu17706

What

A `smriti watch` command that monitors agent log directories via `fs.watch()` and auto-ingests new/changed sessions in real-time.

Why

Currently ingestion is manual (`smriti ingest claude`). Developers forget to run it, or run it too late after context is cold. Auto-ingestion means Smriti always has the latest session data available for search and recall.

Tasks

  • Implement `smriti watch` CLI command with graceful start/stop
  • Use `fs.watch()` (or Bun's equivalent) to monitor `~/.claude/projects/` and other agent log dirs
  • Debounce file change events (JSONL files get appended to frequently during active sessions)
  • Incremental ingestion: track file size/mtime, only re-parse appended content
  • Handle session file rotation (new session creates new file)
  • PID file at `~/.smriti/watch.pid` for single-instance enforcement
  • `smriti watch --daemon` for background mode (detached process)
  • `smriti watch --stop` to kill running daemon
  • `smriti watch --status` to check if daemon is running
  • Optional auto-embed: trigger embedding generation after ingestion
  • Optional auto-categorize: trigger categorization after ingestion
  • Configurable debounce interval via `SMRITI_WATCH_DEBOUNCE_MS` (default: 2000)

Files

  • `src/watch.ts` — new Watch daemon implementation
  • `src/index.ts` — Add `watch` command to CLI
  • `src/config.ts` — Add watch-related config vars
  • `test/watch.test.ts` — new Watch daemon tests (using temp directories)

Acceptance Criteria

  • `smriti watch` starts monitoring and logs ingestion events
  • New Claude sessions appear in `smriti search` within seconds of creation
  • Appending to existing session files triggers incremental re-ingestion
  • Only one watch daemon runs at a time (PID file enforcement)
  • `smriti watch --stop` cleanly terminates the daemon
  • CPU usage stays below 1% when idle (no busy polling)
  • Handles agent log directory not existing (waits for creation)

Testing

bun test test/watch.test.ts    # Unit tests with temp dirs
smriti watch                   # Manual: start watching
# In another terminal, use Claude Code — sessions should auto-ingest
smriti watch --status          # Check daemon status
smriti watch --stop            # Stop cleanly

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestphase-3Phase 3: Auto-ingestion & search

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions