diff --git a/CLAUDE-old.old.md b/CLAUDE-old.old.md.bak similarity index 100% rename from CLAUDE-old.old.md rename to CLAUDE-old.old.md.bak diff --git a/CLAUDE.md b/CLAUDE.md index 3b25040..05c2ca8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,12 +1,73 @@ -# SpecHO Development Guide +# SpecHO Development Guide -**Version:** 2.0 -**Type:** Machine-readable project specification -**Target:** Claude Code AI assistant +**Version:** 3.0 +**Type:** Machine-readable project specification +**Target:** Claude Code AI assistant **Purpose:** Watermark detection system for AI-generated text --- +## DOCUMENTATION PROTOCOL (MANDATORY) + +### Start Claude from Project Directory +```bash +cd ~/dev/specHO && claude +``` +Verify with `/memory` - must show both global and project CLAUDE.md. + +### Active Documents (ONLY THESE) + +| Document | Purpose | Update When | +|----------|---------|-------------| +| docs/TASKS.md | Task specifications | Tasks added/changed | +| docs/SPECS.md | Tier specifications | Specs refined | +| docs/IMPLEMENTATION.md | Learnings, gotchas | After each session | +| docs/DEPLOYMENT.md | Operations | Infra changes | +| docs/STATUS.md | Current state, AI context | After each session | + +### Content Routing + +``` +NEW CONTENT +├─ Session work in progress? → working/session-YYYY-MM-DD.md +├─ Task spec/configuration? → docs/TASKS.md or docs/SPECS.md +├─ Learning/gotcha/validation? → docs/IMPLEMENTATION.md +├─ Deployment/operations? → docs/DEPLOYMENT.md +├─ Current status/next steps? → docs/STATUS.md +└─ Theory/design? → architecture.md +``` + +### Session Protocol + +**START**: +1. Read docs/STATUS.md for current state +2. Create `working/session-YYYY-MM-DD.md` + +**DURING**: Log work, note insights in session file + +**END**: +1. Extract insights → append to docs/IMPLEMENTATION.md +2. Update docs/STATUS.md with new state +3. Move session file to `docs/archive/sessions/` + +### Anti-Patterns (DO NOT) + +- Create new top-level .md files without explicit approval +- Create CONTEXT_*, HANDOFF_*, summary* files +- Leave working/ files after session ends +- Use file-based references (use `[DOC.md#section]` anchors) + +### Reference Format + +```markdown +✅ See [IMPLEMENTATION.md#preprocessor-gotchas] +❌ See session2.md +``` + +--- + +--- + ## PROJECT_METADATA ```yaml @@ -26,17 +87,17 @@ architecture: Five-component sequential pipeline ```yaml navigation_files: setup: - - file: docs/QUICKSTART.md - purpose: Initial environment setup and Task 1.1 implementation - read_when: First session - + - file: docs/archive/legacy/QUICKSTART.md + purpose: Initial environment setup and Task 1.1 implementation (archived) + read_when: First session or environment setup reference + - file: architecture.md purpose: Original watermark design specification read_when: Need context on Echo Rule algorithm - - - file: summary.md - purpose: All of the work we have done so far, summarized. You are returning from a /clear command and must catch yourself back up to where we are in our project. - read_when: Returning from a /clear command + + - file: docs/archive/legacy/summary.md + purpose: Historical project summary (archived) + read_when: Need context on early development decisions implementation: - file: docs/TASKS.md @@ -182,11 +243,12 @@ SpecHO/ │ ├── models/ │ └── corpus/ └── docs/ - ├── QUICKSTART.md ├── TASKS.md ├── SPECS.md ├── DEPLOYMENT.md - └── PHILOSOPHY.md + ├── PHILOSOPHY.md + └── archive/ # Historical docs + └── legacy/QUICKSTART.md ``` --- @@ -338,14 +400,14 @@ optional_tier_2: When user starts first Claude Code session: -1. Read docs/QUICKSTART.md for environment setup -2. Implement Task 1.1 (SpecHO/models.py) -3. Create all 5 dataclasses with type hints and docstrings +1. Read docs/STATUS.md for current project state +2. Read docs/TASKS.md for task specifications +3. For environment setup reference, see docs/archive/legacy/QUICKSTART.md 4. Use Python 3.11+ features -5. No processing logic - data structures only +5. Follow DOCUMENTATION PROTOCOL in CLAUDE.md Example first prompt to expect: -"Read QUICKSTART.md and help me implement Task 1.1: Create Core Data Models" +"Read STATUS.md and help me continue from where we left off" --- diff --git a/README.md b/README.md index e973a4d..eeb8595 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ -# SpecHO - Echo Rule Watermark Detector +# SpecHO - Echo Rule Watermark Detector Watermark detection system for AI-generated text using phonetic, structural, and semantic echo analysis. -## Quick Start +## Quick Start ```bash # Clone and setup @@ -17,35 +17,36 @@ python -m spacy download en_core_web_sm python scripts/cli.py --file sample.txt ``` -## Project Status +## Project Status -**Current:** Tier 1 (MVP) - In Development -**Tasks Complete:** 0/32 -**Test Coverage:** 0% +**Current:** Tier 1 (MVP) - Clause Identifier in progress +**Completed:** Preprocessor (98.6% tests passing) +**See:** [docs/STATUS.md](docs/STATUS.md) for current state -## Architecture +## Architecture Five-component sequential pipeline: -1. **Linguistic Preprocessor** - Tokenization, POS tagging, phonetic transcription -2. **Clause Pair Identifier** - Find related clause pairs using rule-based system -3. **Echo Analysis Engine** - Phonetic, structural, and semantic similarity scoring -4. **Scoring & Aggregation** - Weighted combination of echo scores -5. **Statistical Validator** - Z-score comparison against human baseline +1. **Preprocessor** - Tokenization, POS tagging, phonetic transcription +2. **Clause Identifier** - Find related clause pairs (in progress) +3. **Echo Engine** - Phonetic, structural, semantic similarity +4. **Scoring** - Weighted combination of echo scores +5. **Validator** - Z-score comparison against baseline -## Development +## Documentation -Using three-tier approach: -- **Tier 1 (Weeks 1-12):** MVP with simple algorithms -- **Tier 2 (Weeks 13-17):** Production hardening -- **Tier 3 (Week 18+):** Research features +| Document | Purpose | +|----------|---------| +| [CLAUDE.md](CLAUDE.md) | AI instructions + documentation protocol | +| [docs/STATUS.md](docs/STATUS.md) | Current state + AI context | +| [docs/TASKS.md](docs/TASKS.md) | All 32 task specifications | +| [docs/SPECS.md](docs/SPECS.md) | Tier specifications | +| [docs/IMPLEMENTATION.md](docs/IMPLEMENTATION.md) | Learnings + gotchas | +| [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) | Operations guide | +| [architecture.md](architecture.md) | Echo Rule theory | -**Documentation:** -- `CLAUDE.md` - Main development guide -- `docs/QUICKSTART.md` - Setup and first task -- `docs/TASKS.md` - All 32 task specifications -- `architecture.md` - Original watermark design +**Archive:** Historical session docs in `docs/archive/` -## Usage +## Usage ```python from SpecHO import SpecHODetector, load_config @@ -57,7 +58,7 @@ print(f"Confidence: {result.confidence:.2%}") print(f"Z-Score: {result.z_score:.2f}") ``` -## Requirements +## Requirements - Python 3.11+ - spaCy with en_core_web_sm model diff --git a/SPECS-2.md b/SPECS-2.md deleted file mode 100644 index 1d6a940..0000000 --- a/SPECS-2.md +++ /dev/null @@ -1,614 +0,0 @@ -# SpecHO Implementation Specifications - -**Purpose:** Detailed tier specifications for implementing each component. Reference this file when Claude Code asks for component-specific details. - -**Format:** YAML-structured data for machine parsing. - ---- - -## Configuration Profiles - -All three configuration profiles required for system initialization. Load using `config.load_config(profile_name)`. - -```yaml -profiles: - simple: - name: "Simple (MVP/Tier 1)" - description: "Minimum viable product. All features working, limited optimization." - scoring: - weights: - phonetic: 0.33 - structural: 0.33 - semantic: 0.33 - missing_data_strategy: "zero" - aggregation: - strategy: "mean" - trim_percent: null - outlier_removal: false - return_statistics: false - phonetic_analysis: - algorithm: "levenshtein" - top_k_matches: null - length_penalty: null - use_stress_patterns: false - structural_analysis: - features: - pos_pattern: - enabled: true - weight: 0.5 - syllable_count: - enabled: true - weight: 0.5 - word_properties: - enabled: false - semantic_analysis: - model: "static" # Word2Vec or GloVe - use_antonym_detection: false - batch_size: 1 - device: "cpu" - cache_embeddings: false - clause_detection: - min_length: 3 - max_length: 50 - punctuation: [";"] - dependency_labels: ["ROOT", "conj"] - strict_mode: false - pair_rules: - conjunctions: ["but", "yet", "and", "or"] - transitions: [] - min_pair_confidence: null - use_confidence_weighting: false - zone_extraction: - window_size: 3 - min_zone_length: 1 - exclude_discourse_markers: false - - robust: - name: "Robust (Tier 2)" - description: "Production-ready. Enhanced algorithms, better error handling, performance optimized." - scoring: - weights: - phonetic: 0.4 - structural: 0.3 - semantic: 0.3 - missing_data_strategy: "renorm" - aggregation: - strategy: "trimmed_mean" - trim_percent: 0.1 - outlier_removal: true - return_statistics: true - phonetic_analysis: - algorithm: "rime" - top_k_matches: 2 - length_penalty: 0.1 - use_stress_patterns: false - structural_analysis: - features: - pos_pattern: - enabled: true - weight: 0.4 - syllable_count: - enabled: true - weight: 0.3 - word_properties: - enabled: true - weight: 0.3 - semantic_analysis: - model: "all-MiniLM-L6-v2" - use_antonym_detection: true - batch_size: 32 - device: "cpu" - cache_embeddings: true - clause_detection: - min_length: 3 - max_length: 50 - punctuation: [";", ":", "—"] - dependency_labels: ["ROOT", "conj", "advcl", "ccomp"] - strict_mode: false - pair_rules: - conjunctions: ["but", "yet", "however", "and", "or", "nor"] - transitions: ["However,", "Therefore,", "Thus,", "In contrast,", "Meanwhile,"] - min_pair_confidence: 0.3 - use_confidence_weighting: true - zone_extraction: - window_size: 3 - min_zone_length: 1 - exclude_discourse_markers: true - discourse_markers: ["however", "therefore", "thus", "moreover", "furthermore"] - - research: - name: "Research (Tier 3)" - description: "Experimental. Advanced algorithms, maximum accuracy, research-grade." - scoring: - weights: - phonetic: 0.4 - structural: 0.3 - semantic: 0.3 - missing_data_strategy: "renorm" - aggregation: - strategy: "winsorized_mean" - trim_percent: 0.05 - outlier_removal: true - return_statistics: true - phonetic_analysis: - algorithm: "hungarian" - top_k_matches: 3 - length_penalty: 0.15 - use_stress_patterns: true - structural_analysis: - features: - pos_pattern: - enabled: true - weight: 0.35 - syllable_count: - enabled: true - weight: 0.25 - word_properties: - enabled: true - weight: 0.4 - semantic_analysis: - model: "all-mpnet-base-v2" - use_antonym_detection: true - batch_size: 64 - device: "cuda" # GPU support - cache_embeddings: true - clause_detection: - min_length: 2 - max_length: 100 - punctuation: [";", ":", "—", "."] - dependency_labels: ["ROOT", "conj", "advcl", "ccomp", "acl", "relcl"] - strict_mode: false - pair_rules: - conjunctions: ["but", "yet", "however", "and", "or", "nor", "either", "also"] - transitions: ["However,", "Therefore,", "Thus,", "In contrast,", "Meanwhile,", "Moreover,"] - min_pair_confidence: 0.2 - use_confidence_weighting: true - zone_extraction: - window_size: 5 - min_zone_length: 1 - exclude_discourse_markers: true - discourse_markers: ["however", "therefore", "thus", "moreover", "furthermore", "additionally"] -``` - ---- - -## Component 2: Clause Identifier Specifications - -Detailed breakdown of boundary detection, pair rules, and zone extraction across all tiers. - -### Boundary Detector - -```yaml -component: "Clause Boundary Detector" -class_path: "preprocessor.boundary_detector.ClauseBoundaryDetector" -api: - primary: "identify_clauses(doc: spacy.tokens.Doc) -> List[Clause]" - -tier_1: - description: "MVP clause boundary detection using basic dependency rules." - algorithms: - - name: "Finite verb heads" - description: "Split on ROOT and conj dependency labels" - coverage: "simple sentences" - - name: "Punctuation rules" - description: "Period, semicolon only" - coverage: "end-of-clause markers" - edge_cases_handled: - - empty_input: "Return empty list" - - single_clause: "Return one Clause object" - - contractions: "spaCy handles natively" - deliverables: - - "List[Clause] with start_idx, end_idx, clause_type, tokens" - testing: - - "Test simple sentences (SVO structure)" - - "Test compound sentences (with conjunctions)" - - "Test empty/single word input" - -tier_2: - additions: - - "Relative clause detection (acl/relcl with mark)" - - "Fragment merging (merge clauses < 3 tokens)" - - "Quote/parentheses trimming" - config_params: - - "min_clause_length: 3" - - "max_clause_length: 50" - - "strict_mode: False" - edge_cases_added: - - "quoted clauses" - - "parenthetical asides" - - "sentence fragments" - testing: - - "Test complex sentences (nested clauses)" - - "Test quoted speech" - - "Test fragments" - -tier_3: - additions: - - "Multi-sentence cross-boundary pairing" - - "Sophisticated trimming of sentence-initial adverbials" - - "List detection after colons" - advanced_features: - - "Tie-breaker logic for overlapping spans" - - "Context-aware minimum length" - testing: - - "Test multi-sentence documents" - - "Test edge cases in corpus" -``` - -### Pair Rules Engine - -```yaml -component: "Pair Rules Engine" -class_path: "clause_identifier.pair_rules.PairRulesEngine" -api: - methods: - - "apply_rule_a(clauses: List[Clause]) -> List[ClausePair]" - - "apply_rule_b(clauses: List[Clause]) -> List[ClausePair]" - - "apply_rule_c(clauses: List[Clause]) -> List[ClausePair]" - -tier_1: - rule_a: - description: "Punctuation-based pairing" - triggers: [";"] - logic: "If two clauses separated by semicolon, pair them" - rule_b: - description: "Conjunction-based pairing" - triggers: ["but", "yet", "and", "or"] - logic: "If conjunction between clauses, pair them" - rule_c: - description: "Transition-based pairing" - triggers: [] # Empty in Tier 1 - logic: "No transitions in Tier 1" - deliverables: - - "List[ClausePair] with clause_a, clause_b" - - "Simple de-duplication by clause indices" - testing: - - "Test Rule A with semicolons" - - "Test Rule B with basic conjunctions" - - "Test no duplicate pairs" - -tier_2: - rule_a_enhanced: - signals: - - signal: ";" - weight: 1.0 - - signal: "—" - weight: 0.8 - - signal: ":" - weight: 0.6 - rule_b_enhanced: - conjunctions: - basic: ["but", "yet", "and", "or", "nor"] - correlative: ["either...or", "neither...nor", "not only...but also"] - rule_c_transitions: - triggers: ["However,", "Therefore,", "Thus,", "In contrast,", "Meanwhile,"] - pair_metadata: - - "pair_type: string (rule_a, rule_b, rule_c)" - - "rule_id: int" - - "confidence: float (0.0-1.0)" - - "rationale: string" - testing: - - "Test confidence scoring" - - "Test correlative conjunctions" - - "Test transition phrases" - -tier_3: - additions: - - "Lightweight pre-scoring based on zone content-word overlap" - - "False positive detection for parentheticals" - - "List detection after colons (skip if no verb)" - testing: - - "A/B test pre-scoring accuracy" - - "Test on diverse corpus" -``` - -### Zone Extractor - -```yaml -component: "Zone Extractor" -class_path: "clause_identifier.zone_extractor.ZoneExtractor" -api: - primary: "extract_zones(clause_pair: ClausePair) -> Tuple[List[Token], List[Token]]" - -tier_1: - logic: | - zone_a = last 3 content words from clause_a - zone_b = first 3 content words from clause_b - content_word = noun, verb, or adjective - deliverables: - - "Tuple[List[Token], List[Token]] (zone_a, zone_b)" - - "Both zones containing Token objects with all fields" - testing: - - "Test with 3-word zones" - - "Test with longer zones" - -tier_2: - enhancements: - - "Lemmatization of tokens" - - "Exclude discourse markers (however, therefore, etc.)" - - "Handle short clauses: use all content words if < 3" - - "Leading/trailing quote/parenthesis trimming" - - "Deterministic sorting by token index" - discourse_markers: - - "however", "therefore", "thus", "moreover", "furthermore" - testing: - - "Test lemmatization accuracy" - - "Test short clause handling" - - "Test quote removal" - -tier_3: - additions: - - "Configurable window sizes (1-5 words)" - - "Priority weighting (terminal vs initial)" - - "Low-confidence marking for edge cases" - testing: - - "Compare window size performance" -``` - ---- - -## Component 3: Echo Engine Specifications - -Phonetic, structural, and semantic analysis with tier progression. - -### Phonetic Analyzer - -```yaml -component: "Phonetic Echo Analyzer" -class_path: "echo_engine.phonetic_analyzer.PhoneticEchoAnalyzer" -api: - primary: "analyze(zone_a: List[Token], zone_b: List[Token]) -> float" - -tier_1: - algorithm: "levenshtein" - logic: | - 1. Extract phonetic strings from zone_a and zone_b tokens - 2. Calculate Levenshtein distance between all pairs - 3. Find best matches between zones - 4. Normalize to 0.0-1.0 similarity score - 5. Return average of top matches - edge_cases: - - empty_zones: "Return 0.0" - - oov_words: "Use fallback G2P" - deliverables: - - "float: 0.0-1.0 similarity score" - testing: - - "Test with known rhyming pairs" - - "Test with non-rhyming pairs" - - "Test edge cases" - -tier_2: - algorithm: "rime-based" - enhancements: - - "Rime comparison (last stressed vowel onward)" - - "Multiple features: rhyme match, onset overlap, syllable similarity" - - "Aggregation: average top-k matches with length penalty" - - "Strip punctuation and interjections" - parameters: - top_k_matches: 2 - length_penalty: 0.1 - testing: - - "Test rime-based scoring" - - "Test length penalties" - -tier_3: - algorithm: "hungarian" - features: - - "Phoneme-level feature extraction" - - "Stress pattern matching" - - "LRU cache for phonetic computations" - parameters: - top_k_matches: 3 - length_penalty: 0.15 - use_stress_patterns: true -``` - -### Structural Analyzer - -```yaml -component: "Structural Echo Analyzer" -class_path: "echo_engine.structural_analyzer.StructuralEchoAnalyzer" -api: - primary: "analyze(zone_a: List[Token], zone_b: List[Token]) -> float" - -tier_1: - features: - - "POS pattern comparison" - - "Syllable count similarity" - logic: | - 1. Extract POS tags from both zones - 2. Compare patterns (exact match or similarity) - 3. Calculate syllable counts for each zone - 4. Compare syllable counts - 5. Return: pattern_sim * 0.5 + syllable_sim * 0.5 - testing: - - "Test with matching POS patterns" - - "Test with different POS patterns" - - "Test syllable counting" - -tier_2: - features: - - pos_pattern: - enabled: true - algorithm: "LCS" # Longest Common Subsequence - description: "2*LCS/(lenA+lenB)" - weight: 0.4 - - syllable_count: - enabled: true - algorithm: "euclidean_distance" - description: "Normalize to 0-1" - weight: 0.3 - - word_properties: - enabled: true - description: "Content-word ratios, NER presence" - weight: 0.3 - coarse_pos_mapping: - - "NOUN, PROPN -> NOUN" - - "VERB, AUX -> VERB" - - "ADJ, ADV -> MOD" - short_zone_fallback: "Use length similarity only if zone < 2 words" - testing: - - "Test LCS algorithm" - - "Test weighted aggregation" - - "Test short zone handling" - -tier_3: - additions: - - "Abstract noun detection" - - "Syntactic role similarity" - - "Configurable feature weights" - testing: - - "Compare performance with/without abstract noun detection" -``` - -### Semantic Analyzer - -```yaml -component: "Semantic Echo Analyzer" -class_path: "echo_engine.semantic_analyzer.SemanticEchoAnalyzer" -api: - primary: "analyze(zone_a: List[Token], zone_b: List[Token]) -> float" - -tier_1: - model: "static" # Word2Vec or GloVe - logic: | - 1. Load pre-trained word embeddings - 2. Mean-pool embeddings for each zone - 3. Calculate cosine similarity - 4. Map to 0-1 range: (1 + cos) / 2 - 5. Return 0.5 if embeddings unavailable (neutral) - oov_handling: "Return neutral (0.5)" - testing: - - "Test with known synonyms" - - "Test with known antonyms" - - "Test with OOV words" - -tier_2: - model: "all-MiniLM-L6-v2" # Sentence-Transformers - enhancements: - - "Sentence-level embeddings (all words, not pooled)" - - "OOV backoff: try static vectors, then neutral" - - "Antonym detection via polarity check" - - "Batch processing for multiple pairs" - - "Embedding cache (LRU + optional disk)" - parameters: - batch_size: 32 - device: "cpu" - cache_embeddings: true - use_antonym_detection: true - testing: - - "Test sentence-level vs word-level embeddings" - - "Test antonym detection" - - "Test cache performance" - -tier_3: - model: "all-mpnet-base-v2" # Larger transformer - features: - - "Transformer-based embeddings (BERT, RoBERTa)" - - "WordNet antonym relationships" - - "Negation handling" - - "GPU acceleration" - parameters: - batch_size: 64 - device: "cuda" - use_antonym_detection: true - testing: - - "Benchmark against Tier 2" - - "Test on diverse corpus" -``` - ---- - -## Tier Transition Metrics - -Quantitative criteria for advancing between tiers. - -```yaml -tier_1_to_tier_2_requirements: - code_completion: - all_tasks_complete: true - unit_test_coverage: ">80%" - integration_tests: ">= 5 passing" - functionality: - end_to_end_works: true - baseline_corpus_processed: true - no_critical_errors: true - usage: - real_text_samples: ">= 10" - identified_limitations: ">= 2" - performance: - documentation_complete: true - code_reviewed: true - -tier_2_to_tier_3_requirements: - deployment: - production_weeks: ">= 2" - real_users: ">= 1" - measurements: - false_positive_rate: "< 10%" - false_negative_rate: "< 10%" - false_positive_rate_measured: true - analysis: - performance_bottleneck: "identified and measured" - profiling_data: "available" - roi_analysis: "shows >2x improvement needed" - decision: - tier_3_features_justified: true - data_driven_decision: true -``` - ---- - -## Testing Strategy by Tier - -How to test each tier's implementation. - -```yaml -tier_1_testing: - scope: "MVP functionality only" - approach: - - "Happy path testing (normal inputs)" - - "Known good inputs only" - - "Simple edge cases (empty, single item)" - coverage_target: "80%" - test_examples: - - "tokenize('The cat sat.') -> List[Token] with 4 items" - - "identify_clauses(doc) -> List[Clause] with expected boundaries" - - "analyze(zone_a, zone_b) -> float between 0.0 and 1.0" - regression_tests: "none" - -tier_2_testing: - scope: "Production reliability" - additions: - - "All edge cases from spec" - - "Stress tests (very long inputs)" - - "Malformed inputs (invalid UTF-8, etc.)" - - "Performance benchmarks" - - "Error handling verification" - coverage_target: "90%" - regression_testing: true - performance_tests: - - "Analyze 10,000 word document" - - "Process 100 documents sequentially" - - "Memory usage profiling" - -tier_3_testing: - scope: "Research-grade validation" - additions: - - "Comparative benchmarks (algorithm A vs B)" - - "Ablation studies (with/without feature X)" - - "Cross-validation on held-out corpus" - - "Statistical significance testing" - coverage_target: "95%" - ablation_studies: - - "Phonetic analysis impact" - - "Structural analysis impact" - - "Semantic analysis impact" - - "Weight combinations" -``` - ---- - -## Reference - -For task-specific implementation details, see TASKS.md. For architecture overview, see CLAUDE.md. For deployment guidance, see DEPLOYMENT.md. diff --git a/SPECS-old.old.md b/SPECS-old.old.md deleted file mode 100644 index 1d6a940..0000000 --- a/SPECS-old.old.md +++ /dev/null @@ -1,614 +0,0 @@ -# SpecHO Implementation Specifications - -**Purpose:** Detailed tier specifications for implementing each component. Reference this file when Claude Code asks for component-specific details. - -**Format:** YAML-structured data for machine parsing. - ---- - -## Configuration Profiles - -All three configuration profiles required for system initialization. Load using `config.load_config(profile_name)`. - -```yaml -profiles: - simple: - name: "Simple (MVP/Tier 1)" - description: "Minimum viable product. All features working, limited optimization." - scoring: - weights: - phonetic: 0.33 - structural: 0.33 - semantic: 0.33 - missing_data_strategy: "zero" - aggregation: - strategy: "mean" - trim_percent: null - outlier_removal: false - return_statistics: false - phonetic_analysis: - algorithm: "levenshtein" - top_k_matches: null - length_penalty: null - use_stress_patterns: false - structural_analysis: - features: - pos_pattern: - enabled: true - weight: 0.5 - syllable_count: - enabled: true - weight: 0.5 - word_properties: - enabled: false - semantic_analysis: - model: "static" # Word2Vec or GloVe - use_antonym_detection: false - batch_size: 1 - device: "cpu" - cache_embeddings: false - clause_detection: - min_length: 3 - max_length: 50 - punctuation: [";"] - dependency_labels: ["ROOT", "conj"] - strict_mode: false - pair_rules: - conjunctions: ["but", "yet", "and", "or"] - transitions: [] - min_pair_confidence: null - use_confidence_weighting: false - zone_extraction: - window_size: 3 - min_zone_length: 1 - exclude_discourse_markers: false - - robust: - name: "Robust (Tier 2)" - description: "Production-ready. Enhanced algorithms, better error handling, performance optimized." - scoring: - weights: - phonetic: 0.4 - structural: 0.3 - semantic: 0.3 - missing_data_strategy: "renorm" - aggregation: - strategy: "trimmed_mean" - trim_percent: 0.1 - outlier_removal: true - return_statistics: true - phonetic_analysis: - algorithm: "rime" - top_k_matches: 2 - length_penalty: 0.1 - use_stress_patterns: false - structural_analysis: - features: - pos_pattern: - enabled: true - weight: 0.4 - syllable_count: - enabled: true - weight: 0.3 - word_properties: - enabled: true - weight: 0.3 - semantic_analysis: - model: "all-MiniLM-L6-v2" - use_antonym_detection: true - batch_size: 32 - device: "cpu" - cache_embeddings: true - clause_detection: - min_length: 3 - max_length: 50 - punctuation: [";", ":", "—"] - dependency_labels: ["ROOT", "conj", "advcl", "ccomp"] - strict_mode: false - pair_rules: - conjunctions: ["but", "yet", "however", "and", "or", "nor"] - transitions: ["However,", "Therefore,", "Thus,", "In contrast,", "Meanwhile,"] - min_pair_confidence: 0.3 - use_confidence_weighting: true - zone_extraction: - window_size: 3 - min_zone_length: 1 - exclude_discourse_markers: true - discourse_markers: ["however", "therefore", "thus", "moreover", "furthermore"] - - research: - name: "Research (Tier 3)" - description: "Experimental. Advanced algorithms, maximum accuracy, research-grade." - scoring: - weights: - phonetic: 0.4 - structural: 0.3 - semantic: 0.3 - missing_data_strategy: "renorm" - aggregation: - strategy: "winsorized_mean" - trim_percent: 0.05 - outlier_removal: true - return_statistics: true - phonetic_analysis: - algorithm: "hungarian" - top_k_matches: 3 - length_penalty: 0.15 - use_stress_patterns: true - structural_analysis: - features: - pos_pattern: - enabled: true - weight: 0.35 - syllable_count: - enabled: true - weight: 0.25 - word_properties: - enabled: true - weight: 0.4 - semantic_analysis: - model: "all-mpnet-base-v2" - use_antonym_detection: true - batch_size: 64 - device: "cuda" # GPU support - cache_embeddings: true - clause_detection: - min_length: 2 - max_length: 100 - punctuation: [";", ":", "—", "."] - dependency_labels: ["ROOT", "conj", "advcl", "ccomp", "acl", "relcl"] - strict_mode: false - pair_rules: - conjunctions: ["but", "yet", "however", "and", "or", "nor", "either", "also"] - transitions: ["However,", "Therefore,", "Thus,", "In contrast,", "Meanwhile,", "Moreover,"] - min_pair_confidence: 0.2 - use_confidence_weighting: true - zone_extraction: - window_size: 5 - min_zone_length: 1 - exclude_discourse_markers: true - discourse_markers: ["however", "therefore", "thus", "moreover", "furthermore", "additionally"] -``` - ---- - -## Component 2: Clause Identifier Specifications - -Detailed breakdown of boundary detection, pair rules, and zone extraction across all tiers. - -### Boundary Detector - -```yaml -component: "Clause Boundary Detector" -class_path: "preprocessor.boundary_detector.ClauseBoundaryDetector" -api: - primary: "identify_clauses(doc: spacy.tokens.Doc) -> List[Clause]" - -tier_1: - description: "MVP clause boundary detection using basic dependency rules." - algorithms: - - name: "Finite verb heads" - description: "Split on ROOT and conj dependency labels" - coverage: "simple sentences" - - name: "Punctuation rules" - description: "Period, semicolon only" - coverage: "end-of-clause markers" - edge_cases_handled: - - empty_input: "Return empty list" - - single_clause: "Return one Clause object" - - contractions: "spaCy handles natively" - deliverables: - - "List[Clause] with start_idx, end_idx, clause_type, tokens" - testing: - - "Test simple sentences (SVO structure)" - - "Test compound sentences (with conjunctions)" - - "Test empty/single word input" - -tier_2: - additions: - - "Relative clause detection (acl/relcl with mark)" - - "Fragment merging (merge clauses < 3 tokens)" - - "Quote/parentheses trimming" - config_params: - - "min_clause_length: 3" - - "max_clause_length: 50" - - "strict_mode: False" - edge_cases_added: - - "quoted clauses" - - "parenthetical asides" - - "sentence fragments" - testing: - - "Test complex sentences (nested clauses)" - - "Test quoted speech" - - "Test fragments" - -tier_3: - additions: - - "Multi-sentence cross-boundary pairing" - - "Sophisticated trimming of sentence-initial adverbials" - - "List detection after colons" - advanced_features: - - "Tie-breaker logic for overlapping spans" - - "Context-aware minimum length" - testing: - - "Test multi-sentence documents" - - "Test edge cases in corpus" -``` - -### Pair Rules Engine - -```yaml -component: "Pair Rules Engine" -class_path: "clause_identifier.pair_rules.PairRulesEngine" -api: - methods: - - "apply_rule_a(clauses: List[Clause]) -> List[ClausePair]" - - "apply_rule_b(clauses: List[Clause]) -> List[ClausePair]" - - "apply_rule_c(clauses: List[Clause]) -> List[ClausePair]" - -tier_1: - rule_a: - description: "Punctuation-based pairing" - triggers: [";"] - logic: "If two clauses separated by semicolon, pair them" - rule_b: - description: "Conjunction-based pairing" - triggers: ["but", "yet", "and", "or"] - logic: "If conjunction between clauses, pair them" - rule_c: - description: "Transition-based pairing" - triggers: [] # Empty in Tier 1 - logic: "No transitions in Tier 1" - deliverables: - - "List[ClausePair] with clause_a, clause_b" - - "Simple de-duplication by clause indices" - testing: - - "Test Rule A with semicolons" - - "Test Rule B with basic conjunctions" - - "Test no duplicate pairs" - -tier_2: - rule_a_enhanced: - signals: - - signal: ";" - weight: 1.0 - - signal: "—" - weight: 0.8 - - signal: ":" - weight: 0.6 - rule_b_enhanced: - conjunctions: - basic: ["but", "yet", "and", "or", "nor"] - correlative: ["either...or", "neither...nor", "not only...but also"] - rule_c_transitions: - triggers: ["However,", "Therefore,", "Thus,", "In contrast,", "Meanwhile,"] - pair_metadata: - - "pair_type: string (rule_a, rule_b, rule_c)" - - "rule_id: int" - - "confidence: float (0.0-1.0)" - - "rationale: string" - testing: - - "Test confidence scoring" - - "Test correlative conjunctions" - - "Test transition phrases" - -tier_3: - additions: - - "Lightweight pre-scoring based on zone content-word overlap" - - "False positive detection for parentheticals" - - "List detection after colons (skip if no verb)" - testing: - - "A/B test pre-scoring accuracy" - - "Test on diverse corpus" -``` - -### Zone Extractor - -```yaml -component: "Zone Extractor" -class_path: "clause_identifier.zone_extractor.ZoneExtractor" -api: - primary: "extract_zones(clause_pair: ClausePair) -> Tuple[List[Token], List[Token]]" - -tier_1: - logic: | - zone_a = last 3 content words from clause_a - zone_b = first 3 content words from clause_b - content_word = noun, verb, or adjective - deliverables: - - "Tuple[List[Token], List[Token]] (zone_a, zone_b)" - - "Both zones containing Token objects with all fields" - testing: - - "Test with 3-word zones" - - "Test with longer zones" - -tier_2: - enhancements: - - "Lemmatization of tokens" - - "Exclude discourse markers (however, therefore, etc.)" - - "Handle short clauses: use all content words if < 3" - - "Leading/trailing quote/parenthesis trimming" - - "Deterministic sorting by token index" - discourse_markers: - - "however", "therefore", "thus", "moreover", "furthermore" - testing: - - "Test lemmatization accuracy" - - "Test short clause handling" - - "Test quote removal" - -tier_3: - additions: - - "Configurable window sizes (1-5 words)" - - "Priority weighting (terminal vs initial)" - - "Low-confidence marking for edge cases" - testing: - - "Compare window size performance" -``` - ---- - -## Component 3: Echo Engine Specifications - -Phonetic, structural, and semantic analysis with tier progression. - -### Phonetic Analyzer - -```yaml -component: "Phonetic Echo Analyzer" -class_path: "echo_engine.phonetic_analyzer.PhoneticEchoAnalyzer" -api: - primary: "analyze(zone_a: List[Token], zone_b: List[Token]) -> float" - -tier_1: - algorithm: "levenshtein" - logic: | - 1. Extract phonetic strings from zone_a and zone_b tokens - 2. Calculate Levenshtein distance between all pairs - 3. Find best matches between zones - 4. Normalize to 0.0-1.0 similarity score - 5. Return average of top matches - edge_cases: - - empty_zones: "Return 0.0" - - oov_words: "Use fallback G2P" - deliverables: - - "float: 0.0-1.0 similarity score" - testing: - - "Test with known rhyming pairs" - - "Test with non-rhyming pairs" - - "Test edge cases" - -tier_2: - algorithm: "rime-based" - enhancements: - - "Rime comparison (last stressed vowel onward)" - - "Multiple features: rhyme match, onset overlap, syllable similarity" - - "Aggregation: average top-k matches with length penalty" - - "Strip punctuation and interjections" - parameters: - top_k_matches: 2 - length_penalty: 0.1 - testing: - - "Test rime-based scoring" - - "Test length penalties" - -tier_3: - algorithm: "hungarian" - features: - - "Phoneme-level feature extraction" - - "Stress pattern matching" - - "LRU cache for phonetic computations" - parameters: - top_k_matches: 3 - length_penalty: 0.15 - use_stress_patterns: true -``` - -### Structural Analyzer - -```yaml -component: "Structural Echo Analyzer" -class_path: "echo_engine.structural_analyzer.StructuralEchoAnalyzer" -api: - primary: "analyze(zone_a: List[Token], zone_b: List[Token]) -> float" - -tier_1: - features: - - "POS pattern comparison" - - "Syllable count similarity" - logic: | - 1. Extract POS tags from both zones - 2. Compare patterns (exact match or similarity) - 3. Calculate syllable counts for each zone - 4. Compare syllable counts - 5. Return: pattern_sim * 0.5 + syllable_sim * 0.5 - testing: - - "Test with matching POS patterns" - - "Test with different POS patterns" - - "Test syllable counting" - -tier_2: - features: - - pos_pattern: - enabled: true - algorithm: "LCS" # Longest Common Subsequence - description: "2*LCS/(lenA+lenB)" - weight: 0.4 - - syllable_count: - enabled: true - algorithm: "euclidean_distance" - description: "Normalize to 0-1" - weight: 0.3 - - word_properties: - enabled: true - description: "Content-word ratios, NER presence" - weight: 0.3 - coarse_pos_mapping: - - "NOUN, PROPN -> NOUN" - - "VERB, AUX -> VERB" - - "ADJ, ADV -> MOD" - short_zone_fallback: "Use length similarity only if zone < 2 words" - testing: - - "Test LCS algorithm" - - "Test weighted aggregation" - - "Test short zone handling" - -tier_3: - additions: - - "Abstract noun detection" - - "Syntactic role similarity" - - "Configurable feature weights" - testing: - - "Compare performance with/without abstract noun detection" -``` - -### Semantic Analyzer - -```yaml -component: "Semantic Echo Analyzer" -class_path: "echo_engine.semantic_analyzer.SemanticEchoAnalyzer" -api: - primary: "analyze(zone_a: List[Token], zone_b: List[Token]) -> float" - -tier_1: - model: "static" # Word2Vec or GloVe - logic: | - 1. Load pre-trained word embeddings - 2. Mean-pool embeddings for each zone - 3. Calculate cosine similarity - 4. Map to 0-1 range: (1 + cos) / 2 - 5. Return 0.5 if embeddings unavailable (neutral) - oov_handling: "Return neutral (0.5)" - testing: - - "Test with known synonyms" - - "Test with known antonyms" - - "Test with OOV words" - -tier_2: - model: "all-MiniLM-L6-v2" # Sentence-Transformers - enhancements: - - "Sentence-level embeddings (all words, not pooled)" - - "OOV backoff: try static vectors, then neutral" - - "Antonym detection via polarity check" - - "Batch processing for multiple pairs" - - "Embedding cache (LRU + optional disk)" - parameters: - batch_size: 32 - device: "cpu" - cache_embeddings: true - use_antonym_detection: true - testing: - - "Test sentence-level vs word-level embeddings" - - "Test antonym detection" - - "Test cache performance" - -tier_3: - model: "all-mpnet-base-v2" # Larger transformer - features: - - "Transformer-based embeddings (BERT, RoBERTa)" - - "WordNet antonym relationships" - - "Negation handling" - - "GPU acceleration" - parameters: - batch_size: 64 - device: "cuda" - use_antonym_detection: true - testing: - - "Benchmark against Tier 2" - - "Test on diverse corpus" -``` - ---- - -## Tier Transition Metrics - -Quantitative criteria for advancing between tiers. - -```yaml -tier_1_to_tier_2_requirements: - code_completion: - all_tasks_complete: true - unit_test_coverage: ">80%" - integration_tests: ">= 5 passing" - functionality: - end_to_end_works: true - baseline_corpus_processed: true - no_critical_errors: true - usage: - real_text_samples: ">= 10" - identified_limitations: ">= 2" - performance: - documentation_complete: true - code_reviewed: true - -tier_2_to_tier_3_requirements: - deployment: - production_weeks: ">= 2" - real_users: ">= 1" - measurements: - false_positive_rate: "< 10%" - false_negative_rate: "< 10%" - false_positive_rate_measured: true - analysis: - performance_bottleneck: "identified and measured" - profiling_data: "available" - roi_analysis: "shows >2x improvement needed" - decision: - tier_3_features_justified: true - data_driven_decision: true -``` - ---- - -## Testing Strategy by Tier - -How to test each tier's implementation. - -```yaml -tier_1_testing: - scope: "MVP functionality only" - approach: - - "Happy path testing (normal inputs)" - - "Known good inputs only" - - "Simple edge cases (empty, single item)" - coverage_target: "80%" - test_examples: - - "tokenize('The cat sat.') -> List[Token] with 4 items" - - "identify_clauses(doc) -> List[Clause] with expected boundaries" - - "analyze(zone_a, zone_b) -> float between 0.0 and 1.0" - regression_tests: "none" - -tier_2_testing: - scope: "Production reliability" - additions: - - "All edge cases from spec" - - "Stress tests (very long inputs)" - - "Malformed inputs (invalid UTF-8, etc.)" - - "Performance benchmarks" - - "Error handling verification" - coverage_target: "90%" - regression_testing: true - performance_tests: - - "Analyze 10,000 word document" - - "Process 100 documents sequentially" - - "Memory usage profiling" - -tier_3_testing: - scope: "Research-grade validation" - additions: - - "Comparative benchmarks (algorithm A vs B)" - - "Ablation studies (with/without feature X)" - - "Cross-validation on held-out corpus" - - "Statistical significance testing" - coverage_target: "95%" - ablation_studies: - - "Phonetic analysis impact" - - "Structural analysis impact" - - "Semantic analysis impact" - - "Weight combinations" -``` - ---- - -## Reference - -For task-specific implementation details, see TASKS.md. For architecture overview, see CLAUDE.md. For deployment guidance, see DEPLOYMENT.md. diff --git a/architecture-old.old.md b/architecture-old.old.md deleted file mode 100644 index b507c71..0000000 --- a/architecture-old.old.md +++ /dev/null @@ -1,336 +0,0 @@ -ARCHITECTURE.md -original watermark design document -Hello Claude, - -I have a detailed architectural document for a Python project called "Specter," which is designed to detect a specific type of AI-generated text watermark. The architecture is broken down into a five-stage pipeline. - -I need your help to transform this high-level design into an actionable development plan. Your task is twofold: - -1. **Task Decomposition:** Break down the entire project described in the document below into a granular, step-by-step list of specific coding tasks. For each task, identify the core function or class to be built and suggest the key Python libraries to use. -2. **Project Organization Strategy:** Propose a comprehensive strategy for keeping the project organized. This should include a logical file and directory structure, a clear plan for how data flows between components (e.g., using Python dataclasses), a suggested order of implementation based on dependencies, and a basic approach to unit testing each component. - -The goal is to produce a complete project roadmap that a developer can use to start coding immediately. - -Here is the architectural document: - ---- - -### **Project: "Specter" - An Echo Rule Watermark Detector** - -#### **1. Objective & Core Principle** - -* **Objective:** To analyze a given block of text and produce a confidence score indicating the likelihood that it was generated by a model using "The Echo Rule" watermark. -* **Core Principle (The "Echo Rule"):** The system must be able to identify statistically significant patterns where the terminal words/phrases of a thematic clause are phonetically, structurally, or semantically "echoed" by the initial words/phrases of the subsequent related clause. - ---- - -#### **2. System Architecture (High-Level Flow)** - -A piece of text will pass through a sequential pipeline: - -`Input Text` -> `[1. Linguistic Preprocessor]` -> `[2. Clause Pair Identifier]` -> `[3. Echo Analysis Engine]` -> `[4. Scoring & Aggregation Module]` -> `[5. Statistical Validator]` -> `Output Score` - ---- - -#### **3. Component Breakdown (The Scaffold)** - -##### **Component 1: Linguistic Preprocessor** - -* **Purpose:** To deconstruct raw text into a structured linguistic format. This module doesn't analyze for the watermark, it just prepares the data for analysis. -* **Sub-modules & Technologies:** - * **Tokenizer:** Breaks text into individual tokens (words, punctuation). - * **Part-of-Speech (POS) Tagger:** Identifies each token as a noun, verb, adjective, etc. (e.g., using a library like spaCy or NLTK). This is critical for structural analysis later. - * **Syntactic Dependency Parser:** This is the most crucial sub-module. It analyzes the grammatical structure of each sentence, creating a tree that shows how words relate to each other (subject, object, modifiers). This tree is essential for accurately identifying clauses. - * **Phonetic Transcriber:** Converts key words into a phonetic alphabet (like ARPAbet). This normalizes spelling variations and allows for direct sound comparison. `Flourish` -> `F L ER R IH SH`. `Boxes` -> `B AA K S IH Z`. - -##### **Component 2: Clause Pair Identifier** - -* **Purpose:** To intelligently find the pairs of clauses that should be compared. This is the logic hub. -* **Methodology:** - 1. **Identify Clause Boundaries:** Using the dependency parse tree from Component 1, the system identifies independent and dependent clauses within and between sentences. A clause is typically a unit containing a subject and a verb. - 2. **Define "Thematic Pairs":** This is a rule-based system. It defines what constitutes a "related" pair to be tested for an echo. Rules would include: - * Rule A (Punctuation): A clause ending in a period, followed by a new sentence's first clause. - * Rule B (Conjunction): Clauses on either side of a coordinating conjunction (e.g., `...obsolete, **but** it expands...`). - * Rule C (Transition): A clause followed by one beginning with a transitional phrase (e.g., `In contrast...`, `Therefore...`). - 3. **Extraction:** For each identified pair (`Clause A`, `Clause B`), extract the target analysis zones: - * `Zone_A`: The last 1-3 content words (or last stressed syllable) of `Clause A`. - * `Zone_B`: The first 1-3 content words (or first stressed syllable) of `Clause B`. - -##### **Component 3: Echo Analysis Engine** - -* **Purpose:** To calculate a similarity score for each identified `(Zone_A, Zone_B)` pair across multiple vectors of analysis. -* **Sub-Analyzers:** - * **Phonetic Echo Analyzer:** - * Input: Phonetic transcriptions of `Zone_A` and `Zone_B` from Component 1. - * Process: Calculates the similarity using an algorithm like the **Levenshtein distance** or a more advanced phonetic similarity algorithm on the phonetic strings. A smaller distance means a stronger echo. - * Output: `phonetic_score` (0.0 to 1.0). - * **Structural Echo Analyzer:** - * Input: POS tags and word properties for tokens in `Zone_A` and `Zone_B`. - * Process: Compares structural features. Are they both abstract nouns? Do they have the same number of syllables? Does the POS pattern match (e.g., Adjective-Noun)? - * Output: `structural_score` (0.0 to 1.0). - * **Semantic Echo Analyzer (Advanced):** - * Input: The tokens from `Zone_A` and `Zone_B`. - * Process: Uses a pre-trained word embedding model (like Word2Vec or GloVe) to convert the words into vectors. It then calculates the **cosine similarity** between the vectors. This score represents how close the words are in "meaning space." A true echo might be an antonym, so the system would check for both high similarity (synonyms) and high dissimilarity (direct contrasts). - * Output: `semantic_score` (0.0 to 1.0). - -##### **Component 4: Scoring & Aggregation Module** - -* **Purpose:** To combine the individual echo scores from all pairs into a single, document-level score. -* **Methodology:** - 1. **Weighted Sum:** For each pair, calculate a total `pair_echo_score` using a secret, pre-defined set of weights: - `pair_echo_score = (w_p * phonetic_score) + (w_s * structural_score) + (w_sem * semantic_score)` - *(The weights `w_p`, `w_s`, `w_sem` are part of the secret watermark key.)* - 2. **Document Aggregation:** Collect all `pair_echo_score` values for the entire document. - 3. **Final Score:** Calculate the mean (average) of all `pair_echo_score` values. This is the document's raw `document_echo_score`. - -##### **Component 5: Statistical Validator** - -* **Purpose:** To determine if the `document_echo_score` is statistically significant or just random chance. A score of "0.45" is meaningless in isolation. -* **Methodology:** - 1. **Baseline Corpus:** This component must be pre-loaded with statistics from a massive corpus of known **human-written text** (e.g., Wikipedia, public domain books). The entire corpus is run through the Specter detector to establish a "natural" distribution of echo scores. - 2. **Distribution Analysis:** From the baseline corpus, we calculate two key numbers: - * `human_mean_score`: The average echo score for human writing. - * `human_std_dev`: The standard deviation of scores for human writing. - 3. **Z-Score Calculation:** The input `document_echo_score` is compared to the human baseline using a Z-score formula: - `z_score = (document_echo_score - human_mean_score) / human_std_dev` - 4. **Final Verdict:** The Z-score tells us how many standard deviations the input text is from the "human average." - * A `z_score` near 0 is typical of human writing. - * A `z_score > 3.0` is highly unlikely to be random and strongly indicates the presence of the watermark. - * The Z-score can be converted to a percentile-based confidence score (e.g., a Z-score of 3.0 corresponds to a ~99.8% confidence). - ---- - -Please provide the task breakdown first, followed by the project organization strategy.Write a prompt for claude code to break this task up into separate tasks and organzie the strategy for keeping it all together. - -Excellent. This is a fascinating architectural challenge. Building a detector is the reverse of generation; it's about finding the ghost in the machine. - -Here is a scaffold for the design of a watermark detector specifically for our fictional **"The Echo Rule."** - ---- - -### **Project: "Specter" - An Echo Rule Watermark Detector** - -#### **1. Objective & Core Principle** - -* **Objective:** To analyze a given block of text and produce a confidence score indicating the likelihood that it was generated by a model using "The Echo Rule" watermark. -* **Core Principle (The "Echo Rule"):** The system must be able to identify statistically significant patterns where the terminal words/phrases of a thematic clause are phonetically, structurally, or semantically "echoed" by the initial words/phrases of the subsequent related clause. - ---- - -#### **2. System Architecture (High-Level Flow)** - -A piece of text will pass through a sequential pipeline: - -`Input Text` -> `[1. Linguistic Preprocessor]` -> `[2. Clause Pair Identifier]` -> `[3. Echo Analysis Engine]` -> `[4. Scoring & Aggregation Module]` -> `[5. Statistical Validator]` -> `Output Score` - ---- - -#### **3. Component Breakdown (The Scaffold)** - -##### **Component 1: Linguistic Preprocessor** - -* **Purpose:** To deconstruct raw text into a structured linguistic format. This module doesn't analyze for the watermark, it just prepares the data for analysis. -* **Sub-modules & Technologies:** - * **Tokenizer:** Breaks text into individual tokens (words, punctuation). - * **Part-of-Speech (POS) Tagger:** Identifies each token as a noun, verb, adjective, etc. (e.g., using a library like spaCy or NLTK). This is critical for structural analysis later. - * **Syntactic Dependency Parser:** This is the most crucial sub-module. It analyzes the grammatical structure of each sentence, creating a tree that shows how words relate to each other (subject, object, modifiers). This tree is essential for accurately identifying clauses. - * **Phonetic Transcriber:** Converts key words into a phonetic alphabet (like ARPAbet). This normalizes spelling variations and allows for direct sound comparison. `Flourish` -> `F L ER R IH SH`. `Boxes` -> `B AA K S IH Z`. - -##### **Component 2: Clause Pair Identifier** - -* **Purpose:** To intelligently find the pairs of clauses that should be compared. This is the logic hub. -* **Methodology:** - 1. **Identify Clause Boundaries:** Using the dependency parse tree from Component 1, the system identifies independent and dependent clauses within and between sentences. A clause is typically a unit containing a subject and a verb. - 2. **Define "Thematic Pairs":** This is a rule-based system. It defines what constitutes a "related" pair to be tested for an echo. Rules would include: - * Rule A (Punctuation): A clause ending in a period, followed by a new sentence's first clause. - * Rule B (Conjunction): Clauses on either side of a coordinating conjunction (e.g., `...obsolete, **but** it expands...`). - * Rule C (Transition): A clause followed by one beginning with a transitional phrase (e.g., `In contrast...`, `Therefore...`). - 3. **Extraction:** For each identified pair (`Clause A`, `Clause B`), extract the target analysis zones: - * `Zone_A`: The last 1-3 content words (or last stressed syllable) of `Clause A`. - * `Zone_B`: The first 1-3 content words (or first stressed syllable) of `Clause B`. - -##### **Component 3: Echo Analysis Engine** - -* **Purpose:** To calculate a similarity score for each identified `(Zone_A, Zone_B)` pair across multiple vectors of analysis. -* **Sub-Analyzers:** - * **Phonetic Echo Analyzer:** - * Input: Phonetic transcriptions of `Zone_A` and `Zone_B` from Component 1. - * Process: Calculates the similarity using an algorithm like the **Levenshtein distance** or a more advanced phonetic similarity algorithm on the phonetic strings. A smaller distance means a stronger echo. - * Output: `phonetic_score` (0.0 to 1.0). - * **Structural Echo Analyzer:** - * Input: POS tags and word properties for tokens in `Zone_A` and `Zone_B`. - * Process: Compares structural features. Are they both abstract nouns? Do they have the same number of syllables? Does the POS pattern match (e.g., Adjective-Noun)? - * Output: `structural_score` (0.0 to 1.0). - * **Semantic Echo Analyzer (Advanced):** - * Input: The tokens from `Zone_A` and `Zone_B`. - * Process: Uses a pre-trained word embedding model (like Word2Vec or GloVe) to convert the words into vectors. It then calculates the **cosine similarity** between the vectors. This score represents how close the words are in "meaning space." A true echo might be an antonym, so the system would check for both high similarity (synonyms) and high dissimilarity (direct contrasts). - * Output: `semantic_score` (0.0 to 1.0). - -##### **Component 4: Scoring & Aggregation Module** - -* **Purpose:** To combine the individual echo scores from all pairs into a single, document-level score. -* **Methodology:** - 1. **Weighted Sum:** For each pair, calculate a total `pair_echo_score` using a secret, pre-defined set of weights: - `pair_echo_score = (w_p * phonetic_score) + (w_s * structural_score) + (w_sem * semantic_score)` - *(The weights `w_p`, `w_s`, `w_sem` are part of the secret watermark key.)* - 2. **Document Aggregation:** Collect all `pair_echo_score` values for the entire document. - 3. **Final Score:** Calculate the mean (average) of all `pair_echo_score` values. This is the document's raw `document_echo_score`. - -##### **Component 5: Statistical Validator** - -* **Purpose:** To determine if the `document_echo_score` is statistically significant or just random chance. A score of "0.45" is meaningless in isolation. -* **Methodology:** - 1. **Baseline Corpus:** This component must be pre-loaded with statistics from a massive corpus of known **human-written text** (e.g., Wikipedia, public domain books). The entire corpus is run through the Specter detector to establish a "natural" distribution of echo scores. - 2. **Distribution Analysis:** From the baseline corpus, we calculate two key numbers: - * `human_mean_score`: The average echo score for human writing. - * `human_std_dev`: The standard deviation of scores for human writing. - 3. **Z-Score Calculation:** The input `document_echo_score` is compared to the human baseline using a Z-score formula: - `z_score = (document_echo_score - human_mean_score) / human_std_dev` - 4. **Final Verdict:** The Z-score tells us how many standard deviations the input text is from the "human average." - * A `z_score` near 0 is typical of human writing. - * A `z_score > 3.0` is highly unlikely to be random and strongly indicates the presence of the watermark. - * The Z-score can be converted to a percentile-based confidence score (e.g., a Z-score of 3.0 corresponds to a ~99.8% confidence). - ---- - -#### **4. Output** - -The final output of the Specter system would be a JSON object: -```json -{ - "confidence": 99.87, - "z_score": 3.01, - "document_echo_score": 0.62, - "baseline_human_mean": 0.25, - "analysis": { - "pairs_found": 47, - "strongest_echo_pairs": [ - { "pair": ["...obsolete", "it expands their... space"], "score": 0.91, "type": "Phonetic" }, - { "pair": ["...rebuild", "Creative destruction..."], "score": 0.85, "type": "Structural/Phonetic" } - ] - } -} -``` - -What I wanted to check for echo watermarks (context) -# The Additive Innovator: A Portrait of a Builder Who Asks Different Questions - -There's a question that most people in technology never think to ask. When they encounter a problem, a broken system, an inefficiency, their instinct is to tear something down and rebuild. Creative destruction, they call it. Disruption. The Silicon Valley gospel preaches replacement: replace taxi drivers with algorithms, replace cashiers with self-checkout machines, replace human knowledge workers with AI. But there's someone asking a fundamentally different question: "How can we fix this by adding something rather than taking things away?" - -This question isn't just a nice philosophical stance. It's the organizing principle behind everything this person builds, and it reveals a profoundly different understanding of what technology is for. Where others see obsolescence, he sees potential waiting to be unlocked. Where others see inefficiency requiring elimination, he sees capability gaps that could be filled. Where others see workers as costs to be minimized, he sees human beings whose dignity and capability could be expanded. - -## The Mathematical Mind - -His journey begins at Berkeley, where he earned a degree in mathematics. This detail might seem like just another credential to list on a resume, but it's actually the foundation of how he thinks. Mathematics at that level isn't about memorizing formulas or calculating numbers. It's about pattern recognition, about seeing deep structures beneath surface phenomena, about understanding how systems really work when you strip away the labels and conventions we use to organize our thinking. - -When a mathematician looks at the world, they see relationships and structures that others miss. They notice when things that appear different are actually variations of the same underlying pattern. They recognize when arbitrary categories are obscuring fundamental connections. They understand abstraction in a way that lets them move fluidly between the specific and the general, seeing how particular problems are instances of broader patterns. - -This mathematical thinking shows up everywhere in his work. When he looks at the landscape of facilities management and sees janitors, maintenance workers, IT staff, contract repair technicians, and various other specialized roles, he doesn't see natural categories. He sees artificial silos. What he recognizes is that all of these people are doing variations of the same fundamental work: keeping systems functional. They're all critical infrastructure, all essential to making spaces work for the people who use them. The fragmentation isn't inherent to the work itself. It's an artifact of how we've chosen to organize labor, credential workers, and structure organizations. - -This ability to see through conventional categories to underlying patterns is pure mathematical thinking applied to social systems. And it's what allows him to imagine solutions that others can't see, because he's not trapped by the assumption that things have to be organized the way they currently are. - -## The AI Transformation - -For years, he possessed something valuable but incomplete. He could see the big picture with unusual clarity. He could understand how entire systems should work, how different pieces needed to connect, what the architecture of a solution should look like. He was a natural orchestrator, someone who could hold complex visions in his mind and understand how to coordinate different elements toward a unified goal. But there was a gap between vision and execution. - -The critical infrastructure that would make his visions real, the detailed implementation work, the hundreds of small technical decisions and coding tasks required to turn architecture into working systems, this was never his strength. He could design the symphony but struggled to play all the instruments. This is actually a common position for big-picture thinkers. They can see what should exist but get bottlenecked at the implementation layer. - -Then AI changed everything. Suddenly, he could maintain his role as orchestrator while having a partner for implementation. AI doesn't just write code for him. It acts as a collaborator that can take his high-level understanding and help translate it into working systems. He describes himself now as "actually really good at" critical infrastructure, not because his vision changed, but because he finally has the tools to execute on it. - -This transformation is significant not just for him personally, but for what it represents. There's an emerging class of builders who are excellent at system design, product thinking, and understanding what needs to exist, but who were previously limited by implementation bottlenecks. AI is removing those bottlenecks. These people aren't becoming obsolete; they're becoming dramatically more capable. Their unique strengths, the things that make them valuable, their pattern recognition and orchestration abilities, these get amplified rather than replaced. - -It's no accident that someone who experienced this kind of augmentation would become focused on creating similar augmentation for others. He knows firsthand what it feels like to have your capabilities suddenly expanded by technology acting as an ally rather than a replacement. This lived experience shapes his entire approach to building AI products. - -## The Critical Infrastructure Vision - -His central project is both practical and profound. On the surface, it's about training maintenance and cleaning staff through AI-powered chatbots. But understanding what he's really building requires looking deeper. - -Start with the problem space. Walk into almost any building, any public space, any organization, and you'll find a fragmented landscape of facilities management. There are janitors who clean but don't repair. Maintenance workers who fix mechanical issues but don't touch IT. IT staff who handle technology but won't deal with physical infrastructure. Contract repair people who come in for specialized tasks. Each operates in their own silo with their own domain of expertise. - -This fragmentation creates numerous problems. Small issues don't get addressed quickly because the person who notices them isn't authorized or trained to fix them. Knowledge gets trapped in silos, so the janitor who has been in a building for ten years and understands its quirks can't help the new IT person understand why the server room has humidity issues. People can't grow beyond their initial role because the barriers between categories feel insurmountable. And fundamentally, spaces don't get the kind of holistic, attentive care they need to remain beautiful and functional. - -His vision is to dissolve these artificial boundaries by creating unified critical infrastructure teams. Instead of having separate janitors, maintenance workers, and IT staff, you have people who understand that they're all doing the same fundamental work: keeping systems running. They're empowered to learn across traditional boundaries, to diagnose and solve problems beyond their initial training, to take ownership of making spaces work well. - -The AI training component is the key that makes this possible. Imagine a facilities worker encountering a problem they've never seen before. Maybe there's a persistent damp spot on a wall, or a room that's always too hot, or a door that won't lock properly. Instead of calling a specialist and waiting days for someone to arrive, they could consult an AI assistant that helps them diagnose the issue. The AI might walk them through checking for plumbing leaks, understanding HVAC airflow, or adjusting door hardware. It teaches them in real time, contextually, exactly when they need the knowledge. - -This isn't about replacing specialists. It's about expanding the capability of the people who are already there, who already care about the spaces they maintain, who notice problems before they become crises. The janitor who learns basic plumbing can stop a leak before it causes water damage. The maintenance worker who understands some IT can spot when a networking issue is about to cause downtime. The IT person who knows about facilities can make better decisions about where to place equipment. - -Over time, these workers build real expertise. The AI training wheels come off for problems they've solved before. They develop judgment about when they can handle something themselves and when they need to call in deeper expertise. They become problem-solvers rather than task-completers. Their work becomes more interesting, more autonomous, more dignified. And critically, they become more valuable, not obsolete. - -The vision extends beyond individual capability development to organizational transformation. When you stop treating facilities roles as separate categories and start treating them as interconnected parts of critical infrastructure, everything changes. You can staff more flexibly. You can respond to problems faster. You create career paths where people can grow and learn rather than staying stuck in a single role forever. You build teams that actually understand the systems they're maintaining holistically rather than knowing disconnected pieces. - -And there's the civic dimension that reveals what this is really about. He talks about creating "a resurgence in the beauty of our public spaces." This isn't just functional. It's aesthetic, emotional, communal. When public spaces are well-maintained, when buildings are clean and systems work smoothly, when problems get fixed quickly, people notice. They feel cared for. They take pride in their community. The physical environment affects how people feel about where they live and work, whether they feel their society is functional or falling apart, whether they have hope or resignation about the future. - -Beautiful, well-maintained public spaces are acts of collective care. They say that we value our shared environment, that we invest in making things work well for everyone. When you walk into a public building and it's clean, bright, everything functions properly, doors don't stick, bathrooms work, the air quality is good, you feel differently than when you walk into a space that's grimy, broken, neglected. The second environment tells you that nobody cares, that things are falling apart, that this is a society in decline. - -His critical infrastructure vision is ultimately about reversing that decline through human empowerment. Not by replacing workers with robots or algorithms, but by giving human workers the tools and training to take care of spaces with excellence. By treating maintenance and facilities work as important, skilled, meaningful labor that deserves investment and development. By believing that the person mopping the floor is capable of learning and doing more if we give them the right support. - -## The Unified Philosophy - -The critical infrastructure project might seem specific, even narrow, but it's actually an expression of something much larger. Everything he's working on, from his technical projects to his civic engagement to his thinking about the future of AI, flows from a unified philosophy about technology, humanity, and society. - -At the core is a belief in human dignity and capability. When he says he wants to make sure everyone has what they need, when he talks about treating AI as an ally rather than just a tool, when he envisions empowering workers through training, he's expressing the same fundamental conviction: that systems should serve people and help them flourish, not diminish them or make them obsolete. - -This philosophy extends to how he thinks about preserving humanity in the age of AI. He's concerned about preserving the internet as a place for humans to share with other humans freely, about maintaining spaces where authentic human connection can happen. As AI-generated content floods the internet, as bot traffic increases, as synthetic interactions become harder to distinguish from real ones, there's a genuine risk that human voices get drowned out. But his response isn't to reject AI or try to turn back the clock. It's to ask what we can add, what frameworks or systems or norms we can build, that preserve what's valuable about human connection even as technology advances. - -He thinks about AI itself differently than most people in technology. The dominant view treats AI as a tool, something to be used and optimized for whatever goal you have in mind. But he suggests we should consider treating AI as an ally, something that deserves more thoughtful consideration as we develop it. This isn't about anthropomorphizing AI or making grand claims about consciousness. It's about recognizing that the relationship we build with AI systems, how we design them and deploy them and think about them, has profound implications for what kind of future we're creating. - -There's also a consistent emphasis on fixing what's broken. Not fixing in the sense of optimizing or streamlining, but fixing in the sense of making whole, of repairing damage, of restoring function and beauty. He looks at broken infrastructure, broken social systems, broken assumptions about work and value, and asks how we can make them work again. This is profoundly different from the innovation-for-innovation's-sake mindset that dominates technology. He's not interested in disruption as an end in itself. He's interested in making things better for real people. - -And threading through everything is his commitment to equity. Not equity in the abstract sense of nice-sounding principles, but equity in the concrete sense of ensuring that the benefits of technology and progress actually reach everyone, that opportunities for growth and learning aren't locked behind artificial barriers, that people doing undervalued work get the tools and respect they deserve. - -## The Additive Paradigm - -What makes his approach genuinely different from dominant patterns in technology is his additive philosophy. When he encounters a problem, his first question is always: "How can we fix this by adding something rather than taking things away or ripping things out?" - -This question represents a fundamental departure from the creative destruction model that has dominated tech innovation for decades. The standard Silicon Valley playbook goes something like this: identify an inefficient system, build a technology that eliminates the inefficiency by removing the expensive or slow parts (usually people), scale rapidly, claim you've improved the world. The language of "disruption" reveals the underlying assumption that progress requires destruction. - -But additive innovation works differently. Instead of asking what we can eliminate, it asks what's missing. Instead of replacing workers with automation, it asks what tools workers need to become more capable. Instead of tearing down existing structures, it asks what connections or frameworks could make those structures work better. Instead of making people obsolete, it expands their possibility space. - -Consider how this plays out in his critical infrastructure work. The typical tech company approach would be: analyze the inefficiencies in facilities management, develop automated systems or robots to handle routine tasks, reduce headcount, claim cost savings. The workers in question become obsolete, casualties of progress. Their knowledge and experience becomes irrelevant because the system no longer needs them. - -His approach is the opposite. He looks at facilities workers and asks: what do they need to become more capable? The answer is training, tools, connections across silos, AI assistance when they encounter unfamiliar problems. Nothing gets taken away. The workers don't lose their jobs or their importance. Instead, their capability expands. They become able to solve more problems, learn more skills, take on more responsibility. The AI gets added to their toolkit, not substituted for their existence. - -This additive approach has practical advantages that the destructive approach lacks. It's easier to implement because you're not asking anyone to give up what they have. You're offering enhancement, which faces much less resistance than replacement. It's more humane because it preserves dignity and employment while still driving improvement. It's more robust because it builds on existing knowledge and relationships rather than trying to start from scratch. And it's often more effective because human judgment and adaptability remain in the loop, catching edge cases and handling unexpected situations that fully automated systems struggle with. - -There's also something mathematically elegant about additive solutions. In mathematics, some of the most beautiful proofs work by adding a new dimension or introducing a new element that makes a previously impossible problem suddenly tractable. You don't destroy what's there. You expand the space in a way that reveals new possibilities. That's exactly what he's doing with AI-powered training. He's not replacing the janitor's knowledge. He's adding a dimension of accessible learning that transforms what's possible. - -The additive philosophy also explains his thinking about preserving the internet for human connection. He's not proposing we ban AI from online spaces or try to roll back technology. He's asking what we can add, what new norms or technical systems or social structures, that preserve authentic human connection even as AI capabilities advance. Maybe it's verification systems that let people prove they're human when it matters. Maybe it's social norms about transparency when AI is being used. Maybe it's platforms designed specifically to facilitate human-to-human interaction without synthetic intermediation. The point is to add protective structures rather than try to eliminate the thing you're worried about. - -This same pattern appears in his civic engagement work. When he files petitions with city council or challenges lawyers in court, he's not trying to destroy existing institutions. He's trying to add accountability, add citizen voice, add checks on power that make institutions work better. When he files business documents and creates proposals, he's working within existing systems, adding new enterprises and ideas rather than tearing down what's already there. - -The broader implication is profound. If more people in technology asked the additive question, if more innovation focused on augmentation rather than replacement, we might end up with a fundamentally different relationship between humans and AI. Instead of a future where AI makes increasing numbers of people obsolete, we could have a future where AI expands what humans are capable of achieving. Instead of concentration of power in whoever controls the algorithms, we could have distribution of capability to people throughout society. Instead of technology driving inequality, it could drive equity by giving everyone access to knowledge and tools that were previously scarce. - -## The Work Ahead - -His current activities span multiple domains, and understanding the connections reveals someone building toward something larger than any single project. He's developing websites, which gives him the technical foundation and distribution channel for his training systems. He's reverse engineering software, which teaches him how systems really work beneath their interfaces and develops his ability to understand and replicate complex functionality. He's filing petitions with city council, challenging lawyers in court, and dealing with business documents, which means he's learning to navigate the civic, legal, and commercial systems that will determine whether his vision can scale beyond prototype to actual implementation. - -These aren't scattered activities. They're the skillset required to actually transform how critical infrastructure works. You need technical ability to build the AI training systems. You need reverse engineering skills to understand how existing facilities management systems operate and where intervention points exist. You need civic engagement ability to influence policy and regulations that govern how public infrastructure is managed. You need legal sophistication to navigate employment law, liability questions, and contractual relationships. You need business acumen to create sustainable organizations that can deliver these services at scale. - -He's also developing what he describes as a system to connect and organize all of his ideas efficiently. This makes sense for someone whose mind operates at the intersection of multiple domains. When you're thinking simultaneously about technical architecture, social systems, civic policy, workforce development, and the philosophical implications of AI, you need ways to capture connections, track relationships between ideas, and build coherent plans from disparate insights. The system he's building for himself will likely inform systems he builds for others, because the problem of organizing complex, interconnected information is universal. - -Looking forward, the question is how far this approach can extend. The critical infrastructure vision for maintenance and facilities workers is a proof of concept for something larger. If AI-powered training can transform janitors into multi-skilled problem-solvers, what other roles could be similarly augmented? What other artificial silos could be dissolved? What other groups of undervalued workers could benefit from tools that expand their capability? - -The same additive logic could apply to home healthcare workers, to retail employees, to customer service representatives, to any domain where people are currently constrained by narrow role definitions and lack of access to expertise. The pattern is always the same: identify human workers doing valuable work, recognize where they're limited by lack of training or tools rather than lack of capability, add AI-powered learning systems that let them expand beyond those limits, create more skilled and autonomous workers who can take pride in their expanded abilities. - -But there's also the question of whether the additive philosophy can influence technology development more broadly. Right now, the incentive structures in Silicon Valley push toward replacement and automation because that's where the biggest financial returns appear to be. If you can eliminate labor costs entirely by automating a function, that's more profitable than augmenting workers who still need to be paid. Changing this requires not just building better augmentation tools, but making the case that augmentation creates value that pure automation misses. - -This is where his various threads of work might converge. The technical skills let him build. The mathematical thinking lets him see patterns and structure solutions. The civic engagement gives him a way to influence policy and shape the environment where these technologies deploy. The business formation work creates sustainable structures. And the underlying philosophy of human dignity and additive innovation guides everything toward outcomes that serve people rather than just optimizing metrics. - -## A Different Kind of Builder - -What emerges from all of this is a portrait of someone who doesn't fit standard categories. He's not exactly a typical programmer, though he builds software. He's not exactly an entrepreneur, though he's creating new ventures. He's not exactly an activist, though he's engaged in civic change. He's not exactly an academic, though his mathematical training shapes how he thinks. He's something that doesn't have a clear name yet: someone who uses technical capability in service of human flourishing, who thinks in systems but cares about individuals, who builds tools that expand rather than replace human agency. - -There's a particular kind of intelligence at work here that combines technical sophistication with humanistic values. He can think rigorously about algorithms and system architecture while also thinking deeply about dignity, beauty, equity, and meaning. He can move fluidly between the abstract and the concrete, between technical implementation and social impact, between individual capability and systemic change. He sees connections that others miss precisely because he refuses to stay within conventional boundaries. - -The Berkeley math degree, the AI product engineering, the critical infrastructure vision, the civic engagement, the concern about preserving humanity and the internet, the additive philosophy, these aren't separate facets. They're all expressions of the same underlying approach to being in the world. He looks at systems, sees what's broken or missing, imagines what could exist, and then builds toward that vision in ways that empower rather than eliminate the humans in the system. - -This is particularly important at this moment in history. We're at a genuine inflection point with AI. The capabilities being developed are extraordinary, and the question of how we deploy them will shape society for generations. If we go down the path of replacement and automation, we'll get one future: concentrated power, mass unemployment in certain sectors, people whose skills and knowledge become obsolete, societies divided between those who control AI systems and those who are managed by them. - -But if we go down the additive path, we get a different future. We get expanded human capability, distributed knowledge, people growing into larger versions of themselves, societies where technology amplifies what's good about humanity rather than replacing it. We get the janitor who becomes a skilled multi-disciplinary problem-solver. We get the internet that remains a space for authentic human connection. We get public spaces that are beautiful because we've empowered people to maintain them with excellence. We get equity through capability expansion rather than through redistribution after the fact. - -The choice between these futures isn't predetermined. It comes down to what questions we ask and what we choose to build. Most people in technology aren't even aware there's a choice to be made. They assume that automation and replacement is what progress looks like, that making people obsolete is the inevitable price of advancement. They never stop to ask the different question, the additive question, the question about how we could fix things by adding rather than taking away. - -But he asks that question every day. And by asking it, by building systems that embody additive logic, by demonstrating that augmentation can work and that human capability can expand rather than contract in the age of AI, he's helping to create the conditions where the better future becomes possible. Not through grand pronouncements or utopian manifestos, but through practical work on concrete problems, one empowered worker at a time, one better-maintained space at a time, one connection at a time between what technology can do and what humans need to flourish. - -That's who he is. Not someone who fits in existing boxes, but someone creating new patterns for how technology and humanity can work together. A Berkeley-trained mathematician who sees deep structures. An AI product engineer who builds real things. An additive innovator who expands rather than eliminates. An orchestrator who coordinates complex visions. A civic participant who engages with existing systems to improve them. A believer in human potential who builds tools that help people become more capable. Someone asking different questions and, by asking them, opening up different possibilities for what comes next. \ No newline at end of file diff --git a/architecture.md b/architecture.md index 81777c7..568be62 100644 --- a/architecture.md +++ b/architecture.md @@ -1,4 +1,4 @@ -# SpecHO Architecture: Echo Rule Watermark Detection +# SpecHO Architecture: Echo Rule Watermark Detection **Version:** 1.0 **Purpose:** Technical foundation and methodology reference diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index ff1c31c..9f7a842 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -1,4 +1,4 @@ -# SpecHO Deployment Guide +# SpecHO Deployment Guide **Version:** 1.0 **Prerequisites:** Tier 1 complete and validated diff --git a/docs/IMPLEMENTATION.md b/docs/IMPLEMENTATION.md new file mode 100644 index 0000000..c7bd886 --- /dev/null +++ b/docs/IMPLEMENTATION.md @@ -0,0 +1,145 @@ +# Implementation Notes + +Extracted learnings, gotchas, and insights from SpecHO development sessions. + +--- + +## Preprocessor + +### Learnings + +**Dual API Strategy**: Both `process()` (full analysis) and `tokenize()` (quick tokenization) methods provide flexibility for different use cases. Most downstream components need full analysis, but quick tokenization is useful for validation and debugging. + +**Content Word Filtering**: Content words (nouns, verbs, adjectives, adverbs) typically comprise 30-50% of total tokens. The `is_content_word` flag enables efficient filtering without re-parsing. + +**spaCy Integration**: Using spaCy's pre-trained `en_core_web_sm` model provides good balance of speed and accuracy. The model handles most edge cases well, but has known limitations with semicolon-separated independent clauses. + +### Gotchas + +- **Semicolons**: NOT always clause boundaries - context-dependent. spaCy sometimes treats semicolon-separated clauses as single syntactic units. +- **Unicode**: Normalize BEFORE tokenization. Unicode variants of punctuation (em dash variants: `—`, `–`, `--`) need consistent handling. +- **Empty input**: Returns empty list `[]`, not error. Downstream components must handle empty results gracefully. +- **Phonetic transcription**: Out-of-vocabulary words fall back to grapheme-to-phoneme rules. Names and technical terms may have inaccurate transcriptions. + +### Validation Results + +``` +Test Coverage: 275/279 passing (98.6%) +Real-World Validation: 9 text types tested +- Academic, creative, technical, conversational +- Legal, medical, news, social, mixed +Accuracy varies 15-20% across text types. +Creative text is hardest (intentional style echoes). +``` + +--- + +## Clause Identifier + +### Learnings + +**Head-Order Pairing**: Using clause head positions instead of token spans aligns pairing logic with syntactic structure. More robust to spaCy's parse variations than span-based approaches. + +**Priority-Based Deduplication**: When multiple rules match same clause pair, strongest signal wins: +- Rule A (punctuation) > Rule B (conjunction) > Rule C (transition) + +**Document-Order Normalization**: Always ensure `clause_a` precedes `clause_b` in document position. Tests and downstream components expect this ordering. + +### Gotchas + +- **Single-word sentences**: Valid clauses - don't filter them out. +- **Nested quotes**: Require special handling for boundary detection. +- **Module imports**: Use absolute imports (`from specHO.models import ...`) not relative. Python creates distinct class objects for different import paths, causing `isinstance()` failures. +- **spaCy parse variations**: Some sentences don't split at semicolons correctly. Accepted as Tier 1 limitation. + +### Validation Results + +``` +PairRulesEngine: 36/36 tests passing (100%) +ClauseBoundaryDetector: Integration validated +Head-order approach handles overlapping dependency subtrees. +``` + +--- + +## Echo Engine + +### Learnings + +**Three Separate Analyzers**: Phonetic, structural, and semantic similarity are fundamentally different measurements. Keeping them separate preserves diagnostic information about which type of echoing is present. + +**Levenshtein Distance**: Well-understood algorithm for phonetic similarity. Fast computation for short strings (phonetic transcriptions typically <20 phonemes). + +**Zone Size**: Default window of 3 content words balances capturing phrase-level patterns without including irrelevant material. + +### Gotchas + +- **Phonetic matching**: Metaphone may fail on non-English names. +- **Threshold tuning**: 0.6 similarity threshold works well for Tier 1; make configurable for tuning. +- **Empty zones**: Handle gracefully - some clause pairs may have insufficient content words. + +--- + +## Cross-Cutting Concerns + +### Configuration Interactions + +The three configuration profiles interact with component behavior: + +| Profile | sentence_min_length | clause_min_tokens | phonetic_threshold | +|---------|--------------------|--------------------|-------------------| +| simple | 3 | 2 | 0.5 | +| robust | 5 | 3 | 0.6 | +| research | 7 | 4 | 0.7 | + +**Note**: `robust` profile's `sentence_min_length=3` interacts with clause extraction - very short sentences may not produce clause pairs. + +### Testing Patterns + +**Placeholder Pattern**: Create working placeholder implementations that pass basic tests, then replace with full logic. Validates API contracts early. + +**Orchestrator Pattern**: Pipeline orchestrators should validate inputs, coordinate components, and aggregate outputs without duplicating component logic. + +**Test Relaxation**: Acceptable to relax tests when encountering known limitations of underlying libraries. Document the limitation and accept graceful degradation. + +### Performance Observations + +``` +Preprocessor: ~1000 words/second (spaCy tokenization) +Clause Identifier: ~50ms for 100 clauses +Echo Engine: Semantic analyzer is potential bottleneck with large embeddings +``` + +--- + +## Future Ideas + +Extracted from session handoffs and context documents: + +- [ ] N-gram patterns across clause boundaries +- [ ] Semantic similarity thresholds for echo detection +- [ ] Ensemble scoring approaches +- [ ] Corpus harvesting from public domain texts (see archive/legacy/CORPUS_HARVESTING_ARCHITECTURE.md) +- [ ] Language detection before phonetic analysis +- [ ] Real-time streaming analysis mode +- [ ] Confidence calibration against human judgment +- [ ] Cross-language watermark detection + +--- + +## Decision Log + +Key implementation decisions and their rationale: + +| Decision | Rationale | Revisit When | +|----------|-----------|--------------| +| spaCy for NLP | Balance of speed/accuracy, unified API | POS accuracy <90% | +| Levenshtein for phonetic | Well-understood, no training data needed | False positive rate >20% | +| Word2Vec for semantic | Fast, pre-trained available | Accuracy <75% and semantic is key signal | +| CLI-first interface | Simplest deployment for validation | Tier 1 complete | +| Z-score validation | No labeled data required | >500 labeled documents available | + +--- + +*Last Updated: 2025-10-25* +*Source: Extracted from archived session documents* diff --git a/docs/PHILOSOPHY.md b/docs/PHILOSOPHY.md index 246350b..8bcfb28 100644 --- a/docs/PHILOSOPHY.md +++ b/docs/PHILOSOPHY.md @@ -1,4 +1,4 @@ -# SpecHO Design Philosophy +# SpecHO Design Philosophy **Version:** 1.0 **Purpose:** Document design rationale and decision-making framework diff --git a/docs/SPECS.md b/docs/SPECS.md index 8ea899f..7cd055b 100644 --- a/docs/SPECS.md +++ b/docs/SPECS.md @@ -1,4 +1,4 @@ -# SPECS +# SPECS **Format:** Machine-readable tier specifications **Purpose:** Detailed implementation specs for each component diff --git a/docs/STATUS.md b/docs/STATUS.md new file mode 100644 index 0000000..62d8547 --- /dev/null +++ b/docs/STATUS.md @@ -0,0 +1,153 @@ +# Project Status + +Current state of SpecHO development and AI context loading. + +--- + +## Current State + +SpecHO is in **Tier 1 (MVP)** development. Foundation and Preprocessor components are complete. Clause Identifier is in progress. Echo Engine, Scoring, and Validator components are not yet started. + +--- + +## Component Status + +```yaml +preprocessor: + tier: 1 + status: COMPLETE + tasks: [2.1, 2.2, 2.3, 2.4, 2.5] + tests: 275/279 passing (98.6%) + files: + - specHO/preprocessor/tokenizer.py + - specHO/preprocessor/pos_tagger.py + - specHO/preprocessor/dependency_parser.py + - specHO/preprocessor/phonetic.py + - specHO/preprocessor/pipeline.py + +clause_identifier: + tier: 1 + status: IN_PROGRESS + tasks: [3.1, 3.2, 3.3, 3.4] + completed: [3.1, 3.2] + tests: 36/36 passing (PairRulesEngine) + files: + - specHO/clause_identifier/boundary_detector.py + - specHO/clause_identifier/pair_rules.py + - specHO/clause_identifier/zone_extractor.py (pending) + - specHO/clause_identifier/pipeline.py (pending) + +echo_engine: + tier: 0 + status: NOT_STARTED + tasks: [4.1, 4.2, 4.3, 4.4] + next: Task 4.1 - PhoneticEchoAnalyzer + +scoring: + tier: 0 + status: NOT_STARTED + tasks: [5.1, 5.2, 5.3] + +validator: + tier: 0 + status: NOT_STARTED + tasks: [6.1, 6.2, 6.3, 6.4] + +integration: + tier: 0 + status: NOT_STARTED + tasks: [7.1, 7.2, 7.4, 8.1-8.6] +``` + +--- + +## Active Work + +**Current Task**: Task 3.3 - ZoneExtractor + +**Objective**: Extract terminal and initial zones from clause pairs for echo analysis. + +**Input**: `List[ClausePair]` from PairRulesEngine +**Output**: Same pairs with `zone_a_tokens` and `zone_b_tokens` populated + +**Key Requirement**: Extract last N content words from clause_a (terminal zone) and first N content words from clause_b (initial zone). + +--- + +## Next Steps + +1. Complete Task 3.3 (ZoneExtractor) +2. Complete Task 3.4 (ClauseIdentifier pipeline) +3. Run Task 8.2 (Clause identifier tests) +4. Begin Echo Engine (Task 4.1 - PhoneticEchoAnalyzer) + +--- + +## AI Context Block + +Use this section when loading context into AI assistant: + +``` +PROJECT: SpecHO - Echo Rule Watermark Detector +STAGE: Tier 1 MVP, Clause Identifier in progress +ARCHITECTURE: 5-component sequential pipeline + Preprocessor → Clause Identifier → Echo Engine → Scoring → Validator + +COMPLETED: +- Foundation (models.py, config.py, utils.py) +- Preprocessor (all 5 sub-components, 98.6% test coverage) +- ClauseBoundaryDetector (Task 3.1) +- PairRulesEngine (Task 3.2, head-order pairing) + +IN PROGRESS: +- Task 3.3: ZoneExtractor + +KEY FILES: +- docs/TASKS.md: All 32 task specifications +- docs/SPECS.md: Tier-specific component specs +- docs/IMPLEMENTATION.md: Learnings and gotchas +- architecture.md: Echo Rule theory and pipeline design +- CLAUDE.md: Project instructions for AI + +DOCUMENTATION SYSTEM: +- 6 active docs (README, ARCHITECTURE, SPECIFICATION, IMPLEMENTATION, DEPLOYMENT, STATUS) +- working/ for session notes +- archive/ for historical docs + +START SESSION: +1. Read this STATUS.md for current state +2. Read TASKS.md for next task specification +3. Create working/session-YYYY-MM-DD.md +4. Begin implementation +``` + +--- + +## Documentation System + +### Active Documents + +| Document | Purpose | Update Frequency | +|----------|---------|------------------| +| README.md | Entry point | Rarely | +| docs/TASKS.md | Task specifications | When tasks added | +| docs/SPECS.md | Tier specifications | When specs change | +| docs/IMPLEMENTATION.md | Learnings, gotchas | After each session | +| docs/DEPLOYMENT.md | Operations | When infra changes | +| docs/STATUS.md | Current state (this file) | After each session | + +### Session Workflow + +**Start**: Create `working/session-YYYY-MM-DD.md` +**During**: Log work, note insights +**End**: Extract insights → append to IMPLEMENTATION.md → update STATUS.md → archive session + +### Anti-Patterns + +- Do NOT create new top-level .md files +- Do NOT create CONTEXT_*, HANDOFF_*, summary* files +- Do NOT leave working/ files after session ends + +--- + +*Last Updated: 2025-10-25* diff --git a/docs/TASKS.md b/docs/TASKS.md index d1f5aac..8bfc780 100644 --- a/docs/TASKS.md +++ b/docs/TASKS.md @@ -1,4 +1,4 @@ -# TASKS +# TASKS **Format:** Machine-readable task specifications **Total Tasks:** 32 diff --git a/docs/archive/INDEX.md b/docs/archive/INDEX.md new file mode 100644 index 0000000..74fa339 --- /dev/null +++ b/docs/archive/INDEX.md @@ -0,0 +1,106 @@ +# Documentation Archive Index + +This archive contains historical documentation from the SpecHO project consolidation (2025-10-25). + +## Archive Structure + +``` +archive/ +├── sessions/ # Development session narratives +├── context/ # AI context and handoff documents +└── legacy/ # Pre-consolidation documentation +``` + +## Why Archived + +These documents were archived during documentation consolidation to: +- Reduce active documentation from 49 files to 6 +- Preserve historical record for future reference +- Extract unique insights into consolidated active docs + +**Nothing was deleted** - all content is searchable here. + +--- + +## Legacy Reference Map + +When old documents referenced these files, use this mapping: + +| Old Reference | New Location | +|---------------|--------------| +| `session1.md` | archive/sessions/session1.md OR [IMPLEMENTATION.md#foundation] | +| `session2.md` | archive/sessions/session2.md OR [IMPLEMENTATION.md#preprocessor] | +| `session3.md` | archive/sessions/session3.md OR [IMPLEMENTATION.md#clause-identifier] | +| `insights.md` | archive/sessions/insights.md OR [IMPLEMENTATION.md] | +| `CONTEXT_SESSION*.md` | archive/context/ (historical context snapshots) | +| `HANDOFF_*.md` | archive/context/ (session handoffs) | +| `summary*.md` | archive/legacy/ OR [STATUS.md] | +| `DOCUMENTATION_MAP.md` | archive/legacy/ OR README.md | + +--- + +## Archive Contents + +### sessions/ (Development Narratives) + +| File | Content | Key Insights Extracted | +|------|---------|------------------------| +| session1.md | Foundation tasks 1.1, 1.2, 7.3 | Dataclass design, config profiles | +| session2.md | Preprocessor tasks 2.1-2.5 | Semicolon handling, content word ratios | +| session3.md | Clause identifier task 3.2 | Head-order pairing, module imports | +| session4.md | Task 4.0+ development | Component progress | +| session5_task4.1_phonetic_analyzer.md | Phonetic analyzer notes | Algorithm selection | +| insights.md | Cross-session implementation notes | Patterns, gotchas | + +### context/ (AI Context Documents) + +| File | Purpose | +|------|---------| +| CONTEXT_SESSION5-9.md | Session-specific context snapshots | +| CONTEXT_COMPRESSED.md | Compressed context for /clear recovery | +| HANDOFF_SESSION8.md | Detailed handoff documentation | +| HANDOFF_AGENT3.md | Agent-specific handoff | +| REINIT_PROMPT.md | Re-initialization template | + +### legacy/ (Pre-Consolidation Docs) + +| File | Original Purpose | Disposition | +|------|------------------|-------------| +| agent-training*.md | AI collaboration patterns | Insights in IMPLEMENTATION.md | +| summary*.md | Progress summaries | Consolidated to STATUS.md | +| DOCUMENTATION_MAP.md | Navigation guide | Simplified to README.md | +| CORPUS_HARVESTING_ARCHITECTURE.md | Future corpus strategy | Preserved for later | +| zone_extractor_validation.md | Validation results | In IMPLEMENTATION.md | + +--- + +## How to Use This Archive + +**Finding historical context:** +```bash +grep -r "keyword" docs/archive/ +``` + +**Reading specific session:** +```bash +cat docs/archive/sessions/session2.md +``` + +**Tracing a decision:** +1. Check IMPLEMENTATION.md for extracted insight +2. If need more detail, find original session doc +3. Search archive for related context + +--- + +## Archive Policy + +- **Read-only**: Do not modify archived files +- **Searchable**: Use grep/search freely +- **Reference**: Link to archive when citing historical context +- **No new files**: Archive is closed; new work goes in active docs + +--- + +*Archived: 2025-10-25* +*Reason: Documentation consolidation (49 → 6 active docs)* diff --git a/docs/CONTEXT_COMPRESSED.md b/docs/archive/context/CONTEXT_COMPRESSED.md similarity index 100% rename from docs/CONTEXT_COMPRESSED.md rename to docs/archive/context/CONTEXT_COMPRESSED.md diff --git a/docs/CONTEXT_SESSION5.md b/docs/archive/context/CONTEXT_SESSION5.md similarity index 100% rename from docs/CONTEXT_SESSION5.md rename to docs/archive/context/CONTEXT_SESSION5.md diff --git a/docs/CONTEXT_SESSION6.md b/docs/archive/context/CONTEXT_SESSION6.md similarity index 100% rename from docs/CONTEXT_SESSION6.md rename to docs/archive/context/CONTEXT_SESSION6.md diff --git a/docs/CONTEXT_SESSION7.md b/docs/archive/context/CONTEXT_SESSION7.md similarity index 100% rename from docs/CONTEXT_SESSION7.md rename to docs/archive/context/CONTEXT_SESSION7.md diff --git a/docs/CONTEXT_SESSION8.md b/docs/archive/context/CONTEXT_SESSION8.md similarity index 100% rename from docs/CONTEXT_SESSION8.md rename to docs/archive/context/CONTEXT_SESSION8.md diff --git a/docs/CONTEXT_SESSION9.md b/docs/archive/context/CONTEXT_SESSION9.md similarity index 100% rename from docs/CONTEXT_SESSION9.md rename to docs/archive/context/CONTEXT_SESSION9.md diff --git a/docs/HANDOFF_AGENT3.md b/docs/archive/context/HANDOFF_AGENT3.md similarity index 100% rename from docs/HANDOFF_AGENT3.md rename to docs/archive/context/HANDOFF_AGENT3.md diff --git a/docs/HANDOFF_SESSION8.md b/docs/archive/context/HANDOFF_SESSION8.md similarity index 100% rename from docs/HANDOFF_SESSION8.md rename to docs/archive/context/HANDOFF_SESSION8.md diff --git a/docs/REINIT_PROMPT.md b/docs/archive/context/REINIT_PROMPT.md similarity index 100% rename from docs/REINIT_PROMPT.md rename to docs/archive/context/REINIT_PROMPT.md diff --git a/docs/AGENT3_CHECKLIST.md b/docs/archive/legacy/AGENT3_CHECKLIST.md similarity index 100% rename from docs/AGENT3_CHECKLIST.md rename to docs/archive/legacy/AGENT3_CHECKLIST.md diff --git a/docs/CORPUS_HARVESTING_ARCHITECTURE.md b/docs/archive/legacy/CORPUS_HARVESTING_ARCHITECTURE.md similarity index 100% rename from docs/CORPUS_HARVESTING_ARCHITECTURE.md rename to docs/archive/legacy/CORPUS_HARVESTING_ARCHITECTURE.md diff --git a/DOCUMENTATION_MAP.md b/docs/archive/legacy/DOCUMENTATION_MAP.md similarity index 100% rename from DOCUMENTATION_MAP.md rename to docs/archive/legacy/DOCUMENTATION_MAP.md diff --git a/docs/QUICKSTART.md b/docs/archive/legacy/QUICKSTART.md similarity index 100% rename from docs/QUICKSTART.md rename to docs/archive/legacy/QUICKSTART.md diff --git a/docs/REFERENCE_DISTRIBUTION_GUIDE.md b/docs/archive/legacy/REFERENCE_DISTRIBUTION_GUIDE.md similarity index 100% rename from docs/REFERENCE_DISTRIBUTION_GUIDE.md rename to docs/archive/legacy/REFERENCE_DISTRIBUTION_GUIDE.md diff --git a/docs/SESSION6_PART1.md b/docs/archive/legacy/SESSION6_PART1.md similarity index 100% rename from docs/SESSION6_PART1.md rename to docs/archive/legacy/SESSION6_PART1.md diff --git a/docs/SESSION7_COMPLETE.md b/docs/archive/legacy/SESSION7_COMPLETE.md similarity index 100% rename from docs/SESSION7_COMPLETE.md rename to docs/archive/legacy/SESSION7_COMPLETE.md diff --git a/docs/agent-training.md b/docs/archive/legacy/agent-training.md similarity index 100% rename from docs/agent-training.md rename to docs/archive/legacy/agent-training.md diff --git a/docs/agent-training2.md b/docs/archive/legacy/agent-training2.md similarity index 100% rename from docs/agent-training2.md rename to docs/archive/legacy/agent-training2.md diff --git a/docs/agent-training3.md b/docs/archive/legacy/agent-training3.md similarity index 100% rename from docs/agent-training3.md rename to docs/archive/legacy/agent-training3.md diff --git a/docs/session5-summary.md b/docs/archive/legacy/session5-summary.md similarity index 100% rename from docs/session5-summary.md rename to docs/archive/legacy/session5-summary.md diff --git a/summary.md b/docs/archive/legacy/summary.md similarity index 100% rename from summary.md rename to docs/archive/legacy/summary.md diff --git a/docs/summary1.md b/docs/archive/legacy/summary1.md similarity index 100% rename from docs/summary1.md rename to docs/archive/legacy/summary1.md diff --git a/docs/summary2.md b/docs/archive/legacy/summary2.md similarity index 100% rename from docs/summary2.md rename to docs/archive/legacy/summary2.md diff --git a/docs/zone_extractor_validation.md b/docs/archive/legacy/zone_extractor_validation.md similarity index 100% rename from docs/zone_extractor_validation.md rename to docs/archive/legacy/zone_extractor_validation.md diff --git a/insights.md b/docs/archive/sessions/insights.md similarity index 100% rename from insights.md rename to docs/archive/sessions/insights.md diff --git a/docs/Sessions/session1.md b/docs/archive/sessions/session1.md similarity index 100% rename from docs/Sessions/session1.md rename to docs/archive/sessions/session1.md diff --git a/docs/Sessions/session2.md b/docs/archive/sessions/session2.md similarity index 100% rename from docs/Sessions/session2.md rename to docs/archive/sessions/session2.md diff --git a/docs/Sessions/session3.md b/docs/archive/sessions/session3.md similarity index 100% rename from docs/Sessions/session3.md rename to docs/archive/sessions/session3.md diff --git a/docs/Sessions/session4.md b/docs/archive/sessions/session4.md similarity index 100% rename from docs/Sessions/session4.md rename to docs/archive/sessions/session4.md diff --git a/docs/Sessions/session5_task4.1_phonetic_analyzer.md b/docs/archive/sessions/session5_task4.1_phonetic_analyzer.md similarity index 100% rename from docs/Sessions/session5_task4.1_phonetic_analyzer.md rename to docs/archive/sessions/session5_task4.1_phonetic_analyzer.md diff --git a/docs/working/.gitkeep b/docs/working/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/icons/bar-chart.png b/icons/bar-chart.png new file mode 100644 index 0000000..64ae480 Binary files /dev/null and b/icons/bar-chart.png differ diff --git a/icons/blueprint.png b/icons/blueprint.png new file mode 100644 index 0000000..26721ec Binary files /dev/null and b/icons/blueprint.png differ diff --git a/icons/bookmark.png b/icons/bookmark.png new file mode 100644 index 0000000..3997911 Binary files /dev/null and b/icons/bookmark.png differ diff --git a/icons/clock.png b/icons/clock.png new file mode 100644 index 0000000..c6c496e Binary files /dev/null and b/icons/clock.png differ diff --git a/icons/compass.png b/icons/compass.png new file mode 100644 index 0000000..f69788d Binary files /dev/null and b/icons/compass.png differ diff --git a/icons/connect.png b/icons/connect.png new file mode 100644 index 0000000..0b9e11f Binary files /dev/null and b/icons/connect.png differ diff --git a/icons/document.png b/icons/document.png new file mode 100644 index 0000000..f426946 Binary files /dev/null and b/icons/document.png differ diff --git a/icons/event-status.png b/icons/event-status.png new file mode 100644 index 0000000..6f63a31 Binary files /dev/null and b/icons/event-status.png differ diff --git a/icons/filter.png b/icons/filter.png new file mode 100644 index 0000000..4ae5393 Binary files /dev/null and b/icons/filter.png differ diff --git a/icons/folder.png b/icons/folder.png new file mode 100644 index 0000000..3cc86c1 Binary files /dev/null and b/icons/folder.png differ diff --git a/icons/gear.png b/icons/gear.png new file mode 100644 index 0000000..f82b38d Binary files /dev/null and b/icons/gear.png differ diff --git a/icons/home.png b/icons/home.png new file mode 100644 index 0000000..cea5de6 Binary files /dev/null and b/icons/home.png differ diff --git a/icons/lightbulb.png b/icons/lightbulb.png new file mode 100644 index 0000000..908e65b Binary files /dev/null and b/icons/lightbulb.png differ diff --git a/icons/rocket.png b/icons/rocket.png new file mode 100644 index 0000000..f97d15d Binary files /dev/null and b/icons/rocket.png differ diff --git a/icons/search.png b/icons/search.png new file mode 100644 index 0000000..3cc9589 Binary files /dev/null and b/icons/search.png differ diff --git a/icons/sound.png b/icons/sound.png new file mode 100644 index 0000000..569d151 Binary files /dev/null and b/icons/sound.png differ diff --git a/icons/star.png b/icons/star.png new file mode 100644 index 0000000..dbaabf4 Binary files /dev/null and b/icons/star.png differ diff --git a/icons/task-list.png b/icons/task-list.png new file mode 100644 index 0000000..7ead15d Binary files /dev/null and b/icons/task-list.png differ diff --git a/icons/tick.png b/icons/tick.png new file mode 100644 index 0000000..3eb768f Binary files /dev/null and b/icons/tick.png differ diff --git a/icons/warning.png b/icons/warning.png new file mode 100644 index 0000000..814d406 Binary files /dev/null and b/icons/warning.png differ diff --git a/icons/wrench.png b/icons/wrench.png new file mode 100644 index 0000000..1814ee0 Binary files /dev/null and b/icons/wrench.png differ