Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: CI

on:
push:
branches: [main, portfolio/*]
pull_request:
branches: [main]

permissions:
contents: read

jobs:
smoke-test:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: Download NLTK data
run: |
python -c "import nltk; nltk.download('punkt', quiet=True); nltk.download('punkt_tab', quiet=True)"

- name: Run smoke tests
run: |
python -m pytest tests/smoke_test.py -v

- name: Run demo script
run: |
python demo/run_summary_demo.py
142 changes: 118 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,42 +1,54 @@
# Arrowhead 🏹

[![Python](https://img.shields.io/badge/Python-3.8+-blue.svg)](https://python.org)
[![Python](https://img.shields.io/badge/Python-3.12+-blue.svg)](https://python.org)
[![UV](https://img.shields.io/badge/UV-Fast%20Python%20Package%20Manager-orange.svg)](https://docs.astral.sh/uv/)
[![Ollama](https://img.shields.io/badge/Ollama-Local%20LLMs-green.svg)](https://ollama.ai)
[![DSPy](https://img.shields.io/badge/DSPy-Declarative%20LLM%20Programming-purple.svg)](https://github.com/stanfordnlp/dspy-ai)
[![CI](https://github.com/edgarbc/arrowhead/actions/workflows/ci.yml/badge.svg)](https://github.com/edgarbc/arrowhead/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Tests](https://img.shields.io/badge/Tests-Passing-brightgreen.svg)](https://github.com/yourusername/arrowhead/actions)

> **Obsidian Weekly Hashtag Summarizer** - Automate weekly retrospectives by summarizing journal entries tagged with specific hashtags using local LLMs.
> **TL;DR:** Arrowhead is an LLM-powered CLI tool that automatically summarizes your Obsidian journal entries by hashtag. Point it at your vault, specify a hashtag like `#meeting` or `#work`, and get weekly summaries powered by local LLMs (Ollama) or cloud APIs (OpenAI).

---

## 🎯 Problem Statement

Keeping weekly retrospectives and summaries up-to-date is tedious. If you use Obsidian for daily journaling with hashtags to categorize entries (e.g., `#meeting`, `#work`, `#learning`), you probably spend time manually reviewing and consolidating notes at the end of each week.

**Arrowhead solves this** by automatically scanning your vault, filtering entries by hashtag and date range, and generating well-structured summaries using LLMs—all while keeping your data private with local models.

## 🎯 Overview

Arrowhead is a CLI tool that automates the repetitive task of creating weekly summaries from your Obsidian vault. It scans your journal entries, filters by hashtags and date ranges, and generates consolidated summaries using local LLMs via Ollama.

![Arrowhead Demo](docs/screenshot-placeholder.png)
<!-- Screenshot placeholder - see docs/SCREENSHOT_README.md for instructions -->

### ✨ Features

- **🔍 Smart Vault Scanning** - Discovers markdown files while excluding Obsidian-specific directories
- **��️ Hashtag Filtering** - Filter entries by specific hashtags (e.g., `#meeting`, `#work`)
- **🏷️ Hashtag Filtering** - Filter entries by specific hashtags (e.g., `#meeting`, `#work`)
- **📅 Date Range Support** - Focus on specific weeks or date ranges
- **🤖 Local LLM Integration** - Uses Ollama for cost-effective, privacy-focused summarization
- **☁️ Cloud API Support** - Optional OpenAI integration for cloud-based summarization
- **📦 Intelligent Batching** - Groups entries efficiently to respect token limits
- **📝 Structured Output** - Generates well-formatted summaries with metadata
- **💻 Chat with your notes** - Chat with your summaries using retrieval-augmented generation.
- **💻 Chat with your notes** - Chat with your summaries using retrieval-augmented generation
- **⚡ Fast & Lightweight** - Built with UV for rapid development and deployment
- **🧪 CI-Ready Demo** - Includes deterministic offline summarizer for testing

## 🚀 Quick Start

### Prerequisites

- **Python 3.8+**
- **UV** (Fast Python package manager)
- **Ollama** (Local LLM runtime)
- **Python 3.12+**
- **UV** (Fast Python package manager) - [Install UV](https://docs.astral.sh/uv/getting-started/installation/)
- **Ollama** (Local LLM runtime) - [Install Ollama](https://ollama.ai)

### Installation

```bash
# Clone the repository
git clone https://github.com/yourusername/arrowhead.git
git clone https://github.com/edgarbc/arrowhead.git
cd arrowhead

# Install dependencies with UV
Expand All @@ -46,14 +58,90 @@ uv sync
uv pip install -e .
```

### Configure Local LLM (Ollama)

```bash
# Install Ollama (macOS)
brew install ollama

# Or download from https://ollama.ai

# Start Ollama service
ollama serve

# Pull a model (llama2 recommended for summarization)
ollama pull llama2

# Verify it's working
ollama list
```

### Environment Variables

```bash
# Optional: Configure Ollama endpoint (default: http://localhost:11434)
export OLLAMA_HOST="http://localhost:11434"
export OLLAMA_MODEL="llama2"

# Optional: For OpenAI integration
export OPENAI_API_KEY="your-api-key-here"
export OPENAI_MODEL="gpt-3.5-turbo"
```

### Run the Demo (No LLM Required)

```bash
# Install demo dependencies
pip install -r requirements.txt

# Download NLTK data
python -c "import nltk; nltk.download('punkt'); nltk.download('punkt_tab')"

# Run the demo (uses deterministic TextRank summarizer)
python demo/run_summary_demo.py
```

**Expected Output:**
```
============================================================
Arrowhead Demo - Obsidian Note Summarizer
============================================================

📄 Reading sample note...
Loaded 1758 characters from sample_note.md

------------------------------------------------------------
📝 Original Note Preview (first 500 chars):
------------------------------------------------------------
# Daily Journal - 2024-12-02

## Morning Standup #meeting
...

------------------------------------------------------------
🎯 Generating Summary (using TextRank algorithm)...
------------------------------------------------------------

📋 Summary:

Had a productive morning standup with the team.
The team agreed that we need to prioritize the dashboard work...
Key takeaway: Understanding trade-offs between consistency...

============================================================
✅ Demo completed successfully!
============================================================
```

### Testing
The project both unit tests and integration tests. Integration tests require a local Ollama instance running.

The project has both unit tests and integration tests. Integration tests require a local Ollama instance running.

```bash
# Run all unit tests (no external dependencies)
uv run pytest tests/ -v
# Run smoke tests (no external dependencies)
python -m pytest tests/smoke_test.py -v

# Run only unit tests (excludes integration tests)
# Run all unit tests (no external dependencies)
uv run pytest tests/ -v -m "not integration"

# Run integration tests (requires Ollama)
Expand Down Expand Up @@ -100,8 +188,8 @@ arrowhead scan /path/to/vault --hashtag meeting
```bash
arrowhead/
├── README.md # Project overview and setup instructions
├── pyproject.toml # Dependency management (or setup.py)
├── src/
├── pyproject.toml # Dependency management
├── requirements.txt # Demo/CI dependencies
├── src/
│ └── arrowhead/
│ ├── __init__.py # Package initialization
Expand All @@ -113,23 +201,29 @@ arrowhead/
│ ├── writer.py # Summary aggregation and note writing
│ ├── utils.py # Helper functions (date parsing, logging)
│ └── rag.py # RAG system for chatting with summaries
├── demo/ # Demo files for portfolio/CI
│ ├── sample_note.md # Sample Obsidian note for demo
│ └── run_summary_demo.py # Deterministic demo summarizer
├── tests/ # Unit and integration tests
│ ├── conftest.py # Pytest configuration
│ ├── smoke_test.py # CI smoke tests
│ ├── test_scanner.py
│ ├── test_parser.py
│ ├── test_batcher.py
│ ├── test_summarizer.py
│ └── test_writer.py
│ └── ...
├── examples/ # Sample vault and usage examples
│ └── journal/
│ ├── 2024-12-02.md # Example journal entry markdown file
│ └── 2024-12-03.md
├── Summaries/ # Output folder for generated summaries
├── docs/ # Additional documentation
│ └── usage.md # Usage guide and FAQs
│ └── 2024-12-02.md # Example journal entry
├── .github/
│ └── workflows/
│ └── ci.yml # GitHub Actions CI workflow
└── .gitignore # Ignore venv, __pycache__, etc.

```

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---

**Made with ❤️ for the Obsidian community**
Loading