From 160ba20a5f6d2d3192c1595828bfe8ea3f38a3e0 Mon Sep 17 00:00:00 2001 From: omsherikar Date: Thu, 29 Jan 2026 13:25:24 +0530 Subject: [PATCH 1/5] docs: update README.md for v1.0.12 release --- README.md | 90 ++++++++++++++++++++++++++++++++----------------------- 1 file changed, 53 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index 9386797..e7016fd 100644 --- a/README.md +++ b/README.md @@ -15,27 +15,34 @@ Refactron analyzes Python code for security vulnerabilities, performance issues, ### Code Analysis -- Security scanning: SQL injection, code injection, hardcoded secrets, SSRF vulnerabilities -- Code quality: magic numbers, long functions, excessive parameters, deep nesting -- Complexity metrics: cyclomatic complexity, maintainability index, nested loops -- Type hints: missing or incomplete annotations -- Dead code detection: unused functions and unreachable code -- Dependency analysis: circular imports, wildcard imports -- Performance issues: N+1 queries, inefficient iterations +- **Security scanning**: SQL injection, code injection, hardcoded secrets, SSRF vulnerabilities +- **Code quality**: magic numbers, long functions, excessive parameters, deep nesting +- **Complexity metrics**: cyclomatic complexity, maintainability index, nested loops +- **Type hints**: missing or incomplete annotations +- **Dead code detection**: unused functions and unreachable code +- **Dependency analysis**: circular imports, wildcard imports +- **Performance issues**: N+1 queries, inefficient iterations -### Refactoring +### Pattern Learning System -- Extract constants, simplify conditionals, reduce parameters -- Add docstrings, extract methods -- Preview changes before applying -- Risk scoring for each refactoring -- **Pattern Learning** - Learns from your feedback to improve suggestions over time +Refactron now learns from your project-specific coding standards. +- **Pattern Learning Engine**: Identifies and learns project-specific refactoring patterns. +- **Intelligent Ranking**: Ranks suggestions based on risk and potential impact. +- **Feedback Loop**: Improves accuracy over time based on your feedback. -### Auto-Fix +### Performance & Scalability -- 14 automated fixers for common issues -- Configurable safety levels -- Automatic backups and rollback support +Optimized for large-scale codebases. +- **AST Cache**: Blazing fast repeated analysis. +- **Incremental Analysis**: Only process files modified since the last run. +- **Parallel Processing**: Multi-threaded execution for maximum speed. + +### Refactoring & Auto-Fix + +- **Extract constants**, simplify conditionals, reduce parameters, add docstrings, extract methods. +- **14 automated fixers** for common issues with configurable safety levels. +- **Preview changes** before applying with risk scoring. +- **Backup & Rollback**: Git-integrated safety system to undo changes. ## Installation @@ -45,23 +52,15 @@ pip install refactron ## Usage -### Python API - -```python -from refactron import Refactron - -refactron = Refactron() -analysis = refactron.analyze("path/to/code.py") -print(analysis.report()) - -result = refactron.refactor("path/to/code.py", preview=True) -result.show_diff() -``` - ### Command Line +Refactron features a modern, interactive CLI. + ```bash -# Analyze code +# Log in to Refactron +refactron login + +# Analyze code (requires authentication) refactron analyze myproject/ # Generate report @@ -71,12 +70,29 @@ refactron report myproject/ --format json -o report.json refactron refactor myfile.py --preview # Auto-fix issues -refactron autofix myfile.py --preview refactron autofix myfile.py --apply + +# Rollback changes +refactron rollback --list +refactron rollback --session +``` + +### Python API + +```python +from refactron import Refactron + +refactron = Refactron() +analysis = refactron.analyze("path/to/code.py") +print(analysis.report()) + +result = refactron.refactor("path/to/code.py", preview=True) +result.show_diff() ``` ## Documentation +- [Pattern Learning Guide](docs/PATTERN_LEARNING.md) - [Quick Reference](docs/QUICK_REFERENCE.md) - [Tutorial](docs/TUTORIAL.md) - [Architecture](ARCHITECTURE.md) @@ -84,11 +100,11 @@ refactron autofix myfile.py --apply ## Development Status -Stable release (v1.0.1). Tested on Python 3.8-3.12. +Stable release (v1.0.12). Tested on Python 3.8-3.12. -- 135 tests, 84% code coverage -- 96.8% analyzer module coverage -- Validated on 5,800+ lines of production code +- 669 tests, 78% code coverage +- Validated on 10,000+ lines of production code +- CI/CD ready with GitHub Actions and GitLab CI templates ## Contributing @@ -97,7 +113,7 @@ Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines ```bash git clone https://github.com/Refactron-ai/Refactron_lib.git cd Refactron_lib -bash setup_dev.sh # or setup_dev.bat on Windows +bash setup_dev.sh ``` ## Security From c65a858ebacb850806421221be194d71cdd96367 Mon Sep 17 00:00:00 2001 From: omsherikar Date: Thu, 29 Jan 2026 15:01:44 +0530 Subject: [PATCH 2/5] fix: add missing requests dependency and remove unused astroid --- pyproject.toml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index a3d2559..9f5eef1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -33,7 +33,7 @@ dependencies = [ "pyyaml>=6.0", "rich>=13.0.0", "radon>=6.0.0", - "astroid>=3.0.0", + "requests>=2.25.0", ] [project.optional-dependencies] @@ -44,6 +44,8 @@ dev = [ "mypy>=1.0.0", "flake8>=6.0.0", "isort>=5.12.0", + "types-requests", + "types-PyYAML", ] [project.urls] From 0a7d1783dbed9e1e1b532faed4561261f068ce01 Mon Sep 17 00:00:00 2001 From: omsherikar Date: Thu, 29 Jan 2026 15:31:22 +0530 Subject: [PATCH 3/5] fix: use __version__ in click version_option to avoid mismatch --- refactron/cli.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/refactron/cli.py b/refactron/cli.py index 85adcf2..74a0e47 100644 --- a/refactron/cli.py +++ b/refactron/cli.py @@ -22,7 +22,7 @@ from rich.text import Text from rich.theme import Theme -from refactron import Refactron +from refactron import Refactron, __version__ from refactron.autofix.engine import AutoFixEngine from refactron.autofix.models import FixRiskLevel from refactron.core.analysis_result import AnalysisResult @@ -725,7 +725,7 @@ def print_header() -> None: @click.group(cls=CustomHelpGroup, invoke_without_command=True) -@click.version_option(version="1.0.12") +@click.version_option(version=__version__) @click.pass_context def main(ctx: click.Context) -> None: """ From e98319315163ceaa3fa9f2e672efd5fd55d3e06a Mon Sep 17 00:00:00 2001 From: omsherikar Date: Thu, 29 Jan 2026 15:33:45 +0530 Subject: [PATCH 4/5] style: fix trailing whitespace and end-of-file issues in documentation --- DIRECTORY_ANALYSIS.md | 21 ++++++++++----------- FEATURES.md | 3 --- 2 files changed, 10 insertions(+), 14 deletions(-) diff --git a/DIRECTORY_ANALYSIS.md b/DIRECTORY_ANALYSIS.md index 419e4a9..a66a88e 100644 --- a/DIRECTORY_ANALYSIS.md +++ b/DIRECTORY_ANALYSIS.md @@ -27,7 +27,7 @@ Refactron_Lib/ │ │ ├── code_smell_analyzer.py # 7 rules (S001-S007) │ │ ├── complexity_analyzer.py # 4 rules (C001, C002, C003, C004) │ │ ├── dead_code_analyzer.py # 6 rules (DEAD001-DEAD006) -│ │ ├── dependency_analyzer.py # 7 rules (DEP001-DEP007) +│ │ ├── dependency_analyzer.py # 7 rules (DEP001-DEP007) │ │ ├── performance_analyzer.py # 6 rules (P001-P006) │ │ ├── security_analyzer.py # 13 rules (SEC001-SEC013) │ │ └── type_hint_analyzer.py # 5 rules (TYPE001-TYPE005) @@ -575,15 +575,14 @@ Refactron_Lib/ Refactron is a **well-architected, production-ready** Python code analysis and refactoring library with: -✅ **Modular design** - Easy to extend and maintain -✅ **Comprehensive analysis** - 48 rules across 7 categories -✅ **Safe refactoring** - Preview mode, risk scoring, backups -✅ **Automated fixes** - 14 fixers for common issues -✅ **Pattern Learning** - Feedback collection and pattern recognition system -✅ **Production-ready** - Performance optimizations, monitoring, logging -✅ **Well-tested** - 84% coverage, 31 test files -✅ **Well-documented** - Comprehensive docs and examples -✅ **Developer-friendly** - Automated setup, clear architecture +✅ **Modular design** - Easy to extend and maintain +✅ **Comprehensive analysis** - 48 rules across 7 categories +✅ **Safe refactoring** - Preview mode, risk scoring, backups +✅ **Automated fixes** - 14 fixers for common issues +✅ **Pattern Learning** - Feedback collection and pattern recognition system +✅ **Production-ready** - Performance optimizations, monitoring, logging +✅ **Well-tested** - 84% coverage, 31 test files +✅ **Well-documented** - Comprehensive docs and examples +✅ **Developer-friendly** - Automated setup, clear architecture **The codebase demonstrates professional software engineering practices and is ready for open-source contributions and production use.** - diff --git a/FEATURES.md b/FEATURES.md index cd1263a..8b65965 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -782,6 +782,3 @@ Refactron is perfect for: **Refactron - Making Python code better, one refactoring at a time!** 🚀 **Now with Pattern Learning** - The more you use Refactron, the smarter it gets! 🧠 - - - From 2b8c7b89d6335a6afc63f7623e6021d53eb7e7e8 Mon Sep 17 00:00:00 2001 From: omsherikar Date: Thu, 29 Jan 2026 17:24:22 +0530 Subject: [PATCH 5/5] chore: remove analysis files from tracking and add to .gitignore --- .gitignore | 4 + DIRECTORY_ANALYSIS.md | 588 ------------------------------- FEATURES.md | 784 ------------------------------------------ 3 files changed, 4 insertions(+), 1372 deletions(-) delete mode 100644 DIRECTORY_ANALYSIS.md delete mode 100644 FEATURES.md diff --git a/.gitignore b/.gitignore index 0ec8f98..471a3b1 100644 --- a/.gitignore +++ b/.gitignore @@ -138,3 +138,7 @@ benchmarks/benchmark_history.json refactron_ast_cache/ refactron_incremental_state.json .refactron/ + +# Exclude analysis files +DIRECTORY_ANALYSIS.md +FEATURES.md diff --git a/DIRECTORY_ANALYSIS.md b/DIRECTORY_ANALYSIS.md deleted file mode 100644 index a66a88e..0000000 --- a/DIRECTORY_ANALYSIS.md +++ /dev/null @@ -1,588 +0,0 @@ -# 📁 Refactron Directory Analysis - -**Complete Codebase Structure and Organization Analysis** - ---- - -## 📊 Project Statistics - -- **Total Python Files**: 51 in `refactron/`, 31 in `tests/`, 11 in `examples/` -- **Total Lines of Code**: ~9,844 in `refactron/`, ~7,982 in `tests/` -- **Version**: 1.0.1 (Production/Stable) -- **Python Support**: 3.8, 3.9, 3.10, 3.11, 3.12 -- **License**: MIT - ---- - -## 🗂️ Directory Structure - -``` -Refactron_Lib/ -├── refactron/ # Main library package (51 Python files) -│ ├── __init__.py # Package initialization and exports -│ ├── cli.py # Command-line interface (10 commands) -│ │ -│ ├── analyzers/ # Code analysis modules (8 files) -│ │ ├── base_analyzer.py -│ │ ├── code_smell_analyzer.py # 7 rules (S001-S007) -│ │ ├── complexity_analyzer.py # 4 rules (C001, C002, C003, C004) -│ │ ├── dead_code_analyzer.py # 6 rules (DEAD001-DEAD006) -│ │ ├── dependency_analyzer.py # 7 rules (DEP001-DEP007) -│ │ ├── performance_analyzer.py # 6 rules (P001-P006) -│ │ ├── security_analyzer.py # 13 rules (SEC001-SEC013) -│ │ └── type_hint_analyzer.py # 5 rules (TYPE001-TYPE005) -│ │ -│ ├── refactorers/ # Code refactoring modules (7 files) -│ │ ├── base_refactorer.py -│ │ ├── extract_method_refactorer.py -│ │ ├── magic_number_refactorer.py -│ │ ├── simplify_conditionals_refactorer.py -│ │ ├── reduce_parameters_refactorer.py -│ │ └── add_docstring_refactorer.py -│ │ -│ ├── autofix/ # Auto-fix engine (5 files) -│ │ ├── engine.py # Main auto-fix orchestration -│ │ ├── fixers.py # 14 concrete fixer implementations -│ │ ├── file_ops.py # File operation utilities -│ │ └── models.py # Fix-related data models -│ │ -│ ├── core/ # Core functionality (18 files) -│ │ ├── refactron.py # Main Refactron class (587 lines) -│ │ ├── config.py # Configuration management (YAML support, profiles) -│ │ ├── config_loader.py # Profile/environment config loading -│ │ ├── config_validator.py # Schema validation for configs -│ │ ├── config_templates.py # Framework-specific config templates -│ │ ├── models.py # Core data models (CodeIssue, FileMetrics, etc.) -│ │ ├── analysis_result.py # Analysis result handling -│ │ ├── refactor_result.py # Refactoring result handling -│ │ ├── exceptions.py # Custom exception classes -│ │ ├── cache.py # AST caching system -│ │ ├── incremental.py # Incremental analysis tracking -│ │ ├── parallel.py # Parallel processing support -│ │ ├── memory_profiler.py # Memory profiling -│ │ ├── metrics.py # Metrics collection -│ │ ├── prometheus_metrics.py # Prometheus integration -│ │ ├── telemetry.py # Telemetry collection -│ │ ├── logging_config.py # Structured logging setup -│ │ ├── backup.py # Backup and rollback system -│ │ └── false_positive_tracker.py # False positive detection -│ │ -│ ├── ai/ # AI-related modules (placeholder for future) -│ ├── multifile/ # Multi-file analysis (placeholder) -│ ├── patterns/ # Pattern Learning System (8 files) -│ │ ├── models.py # Data models (RefactoringFeedback, Pattern, etc.) -│ │ ├── storage.py # Persistent pattern storage (JSON, thread-safe) -│ │ ├── fingerprint.py # AST-based code fingerprinting -│ │ ├── matcher.py # Pattern matching and scoring -│ │ ├── learner.py # Pattern learning engine -│ │ ├── learning_service.py # Background learning service -│ │ ├── ranker.py # Suggestion ranking engine (RefactoringRanker) -│ │ └── tuner.py # Project-specific rule tuning (RuleTuner) -│ └── rules/ # Custom rules (placeholder) -│ -├── tests/ # Test suite (31 Python files) -│ ├── test_analyzers.py # Analyzer unit tests -│ ├── test_analyzer_enhancements.py # Enhanced analyzer tests -│ ├── test_analyzer_edge_cases.py # Edge case tests -│ ├── test_analyzer_coverage_supplement.py -│ ├── test_integration_analyzer_coverage.py # Integration tests -│ ├── test_refactorers.py # Refactorer tests -│ ├── test_refactron.py # Main class tests -│ ├── test_cli.py # CLI command tests -│ ├── test_patterns_models.py # Pattern learning models tests -│ ├── test_patterns_storage.py # Pattern storage tests -│ ├── test_patterns_fingerprint.py # Pattern fingerprinting tests -│ ├── test_patterns_matcher.py # Pattern matching tests -│ ├── test_patterns_ranker.py # Suggestion ranking tests -│ ├── test_patterns_tuner.py # Project-specific rule tuning tests -│ └── test_patterns_feedback.py # Feedback collection tests -│ ├── test_autofix/ # Auto-fix tests -│ │ ├── test_engine.py -│ │ ├── test_fixers.py -│ │ └── test_file_ops.py -│ ├── test_backup.py # Backup system tests -│ ├── test_error_handling.py # Error handling tests -│ ├── test_exceptions.py # Exception tests -│ ├── test_logging.py # Logging tests -│ ├── test_metrics.py # Metrics tests -│ ├── test_performance_optimization.py # Performance tests -│ ├── test_prometheus.py # Prometheus tests -│ ├── test_telemetry.py # Telemetry tests -│ ├── test_config_management.py # Configuration management tests -│ ├── test_config_loader_edge_cases.py # Config loader edge cases -│ ├── test_false_positive_reduction.py -│ ├── test_phase2_analyzers.py -│ └── test_real_world_patterns.py # Real-world code pattern tests -│ -├── examples/ # Example code and demos (11 Python files) -│ ├── demo.py # Main demonstration script -│ ├── phase2_demo.py # Phase 2 features demo -│ ├── refactoring_demo.py # Refactoring examples -│ ├── bad_code_example.py # Code with issues (for testing) -│ ├── good_code_example.py # Clean code example -│ ├── simple_clean_example.py -│ ├── cli_tool_example.py # CLI usage examples -│ ├── data_science_example.py # Data science patterns -│ ├── flask_api_example.py # Flask application example -│ ├── error_handling_example.py -│ ├── enhanced_error_handling_example.py -│ └── refactron_monitoring.yaml # Prometheus monitoring config -│ -├── real_world_tests/ # Real-world code testing -│ ├── run_tests.py -│ ├── test_runner.py -│ └── results/ # Test results directory -│ -├── benchmarks/ # Performance benchmarks -│ └── performance_benchmark.py -│ -├── docs/ # Documentation -│ ├── TUTORIAL.md -│ ├── QUICK_REFERENCE.md -│ ├── ERROR_HANDLING.md -│ ├── FALSE_POSITIVE_REDUCTION.md -│ ├── MONITORING.md -│ ├── PERFORMANCE_OPTIMIZATION.md -│ ├── index.html # Documentation site -│ ├── _config.yml # Site configuration -│ └── images/ -│ └── Refactron-logo-TM.png -│ -├── .github/ # GitHub configuration -│ ├── workflows/ # CI/CD workflows -│ └── ISSUE_TEMPLATE/ # Issue templates -│ -├── dist/ # Distribution packages -│ ├── refactron-1.0.0-py3-none-any.whl -│ ├── refactron-1.0.0.tar.gz -│ ├── refactron-0.1.0b1-py3-none-any.whl -│ └── refactron-0.1.0b1.tar.gz -│ -├── htmlcov/ # Test coverage HTML reports -│ -├── setup_dev.sh # Development setup script (macOS/Linux) -├── setup_dev.bat # Development setup script (Windows) -│ -├── pyproject.toml # Project configuration -├── requirements.txt # Runtime dependencies -├── requirements-dev.txt # Development dependencies -├── MANIFEST.in # Package manifest -├── LICENSE # MIT License -│ -├── README.md # Main project README -├── ARCHITECTURE.md # Architecture documentation -├── CONTRIBUTING.md # Contribution guidelines -├── CODE_OF_CONDUCT.md # Code of conduct -├── SECURITY.md # Security policy -├── CHANGELOG.md # Version changelog -├── FEATURES.md # Complete feature list -│ -└── (Additional documentation) -``` - ---- - -## 🏗️ Architecture Overview - -### **Modular Design Principles** - -1. **Separation of Concerns**: Each module has a single, well-defined responsibility -2. **Plugin Architecture**: Base classes enable easy extension -3. **Configuration-Driven**: Behavior customizable via YAML config files -4. **Extensible**: Easy to add new analyzers, refactorers, and fixers - -### **Core Layers** - -``` -┌─────────────────────────────────────────┐ -│ CLI Layer (cli.py) │ -│ Command-line interface (10 commands) │ -└──────────────┬──────────────────────────┘ - │ -┌──────────────▼──────────────────────────┐ -│ Core Layer (core/refactron.py) │ -│ Main orchestrator and coordinator │ -└──────────────┬──────────────────────────┘ - │ - ┌──────────┴──────────┐ - │ │ -┌───▼────────┐ ┌──────▼──────────┐ -│ Analyzers │ │ Refactorers │ -│ (7 types) │ │ (5 types) │ -└────────────┘ └─────────────────┘ - │ │ - └──────────┬──────────┘ - │ - ┌──────────┴──────────┐ - │ │ -┌───▼────────┐ ┌──────▼──────────┐ -│ Auto-Fix │ │ Pattern Learning│ -│ Engine │ │ System │ -│ (14 fixers)│ │ (Feedback Track)│ -└────────────┘ └─────────────────┘ -``` - ---- - -## 📦 Package Dependencies - -### **Runtime Dependencies** (`requirements.txt`) -- `libcst>=1.1.0` - Concrete Syntax Tree (preserves formatting) -- `click>=8.0.0` - CLI framework -- `pyyaml>=6.0` - YAML configuration support -- `rich>=13.0.0` - Beautiful terminal output -- `radon>=6.0.0` - Complexity metrics -- `astroid>=3.0.0` - Advanced AST analysis - -### **Development Dependencies** (`requirements-dev.txt`) -- `pytest>=7.0.0` - Testing framework -- `pytest-cov>=4.0.0` - Coverage reporting -- `black>=23.0.0` - Code formatting -- `mypy>=1.0.0` - Type checking -- `flake8>=6.0.0` - Linting -- `isort>=5.12.0` - Import sorting - ---- - -## 🔍 Code Organization Details - -### **Analyzers Module** (`refactron/analyzers/`) - -**Base Class**: `BaseAnalyzer` (abstract) -- All analyzers inherit from this -- Defines interface: `analyze(file_path, source_code) -> List[CodeIssue]` - -**Implementations** (7 analyzers, 48 total rules): -1. **SecurityAnalyzer** - 13 security rules (SEC001-SEC013) -2. **PerformanceAnalyzer** - 6 performance rules (P001-P006) -3. **ComplexityAnalyzer** - 4 complexity rules (C001-C004, M001) -4. **CodeSmellAnalyzer** - 7 code smell rules (S001-S007) -5. **DeadCodeAnalyzer** - 6 dead code rules (DEAD001-DEAD006) -6. **DependencyAnalyzer** - 7 dependency rules (DEP001-DEP007) -7. **TypeHintAnalyzer** - 5 type hint rules (TYPE001-TYPE005) - -### **Refactorers Module** (`refactron/refactorers/`) - -**Base Class**: `BaseRefactorer` (abstract) -- All refactorers inherit from this -- Defines interface: `refactor(file_path, source_code) -> List[RefactoringOperation]` - -**Implementations** (5 refactorers): -1. `ExtractMethodRefactorer` - Extract methods from complex functions -2. `MagicNumberRefactorer` - Extract magic numbers to constants -3. `SimplifyConditionalsRefactorer` - Simplify nested conditionals -4. `ReduceParametersRefactorer` - Reduce function parameters -5. `AddDocstringRefactorer` - Add docstrings to functions/classes - -### **Auto-Fix Module** (`refactron/autofix/`) - -**Engine**: `AutoFixEngine` -- Orchestrates fix application -- Risk-based safety levels -- Preview and apply modes - -**Fixers** (14 automated fixers): -1. `RemoveUnusedImportsFixer` -2. `ExtractMagicNumbersFixer` -3. `AddDocstringsFixer` -4. `RemoveDeadCodeFixer` -5. `FixTypeHintsFixer` -6. `SortImportsFixer` -7. `RemoveTrailingWhitespaceFixer` -8. `NormalizeQuotesFixer` -9. `SimplifyBooleanFixer` -10. `ConvertToFStringFixer` -11. `RemoveUnusedVariablesFixer` -12. `FixIndentationFixer` -13. `AddMissingCommasFixer` -14. `RemovePrintStatementsFixer` - -### **Core Module** (`refactron/core/`) - -**Main Components** (18 files): - -1. **`refactron.py`** - Main `Refactron` class (687+ lines) - - Orchestrates analysis and refactoring - - Manages analyzers and refactorers - - Handles file discovery and processing - - Integrates Pattern Learning System (feedback collection, pattern fingerprinting) - -2. **`config.py`** - `RefactronConfig` dataclass - - YAML configuration support - - Profile and environment support - - Default configurations - - Comprehensive settings (analyzers, thresholds, etc.) - -3. **`config_loader.py`** - Configuration loader with profiles - - Profile-based configuration loading (dev, staging, prod) - - Environment overrides - - Config inheritance and composition - - Base + profile merging - -4. **`config_validator.py`** - Configuration schema validation - - Strict schema validation - - Type checking for all fields - - Value range validation - - Actionable error messages - - Version compatibility checking - -5. **`config_templates.py`** - Framework-specific templates - - Base template (all frameworks) - - Django-specific template - - FastAPI-specific template - - Flask-specific template - -6. **`models.py`** - Core data models - - `CodeIssue` - Represents detected issues - - `FileMetrics` - File-level metrics - - `RefactoringOperation` - Proposed refactorings (with unique operation IDs) - - Enumerations: `IssueLevel`, `IssueCategory` - -7. **`analysis_result.py`** - Analysis result handling - - Aggregates issues across files - - Report generation (text, JSON, HTML) - -8. **`refactor_result.py`** - Refactoring result handling - - Manages refactoring operations - - Preview and apply modes - -9. **Performance Optimizations**: - - `cache.py` - AST caching - - `incremental.py` - Incremental analysis - - `parallel.py` - Parallel processing - - `memory_profiler.py` - Memory profiling - -10. **Monitoring & Observability**: - - `metrics.py` - Metrics collection - - `prometheus_metrics.py` - Prometheus integration - - `telemetry.py` - Telemetry collection - - `logging_config.py` - Structured logging - -11. **Safety & Recovery**: - - `backup.py` - Backup and rollback system - - `exceptions.py` - Custom exceptions - - `false_positive_tracker.py` - False positive tracking - ---- - -## 🧪 Testing Structure - -### **Test Organization** (31 test files) - -**Unit Tests**: -- `test_analyzers.py` - Analyzer unit tests -- `test_refactorers.py` - Refactorer unit tests -- `test_refactron.py` - Main class tests -- `test_cli.py` - CLI command tests -- `test_autofix/` - Auto-fix unit tests - -**Integration Tests**: -- `test_integration_analyzer_coverage.py` - Full integration tests -- `test_real_world_patterns.py` - Real-world code patterns - -**Specialized Tests**: -- `test_analyzer_enhancements.py` - Enhanced analyzer features -- `test_analyzer_edge_cases.py` - Edge case handling -- `test_error_handling.py` - Error handling -- `test_performance_optimization.py` - Performance optimizations - -**Pattern Learning Tests**: -- `test_patterns_models.py` - Pattern data models -- `test_patterns_storage.py` - Pattern storage and persistence -- `test_patterns_fingerprint.py` - Code fingerprinting -- `test_patterns_matcher.py` - Pattern matching and scoring -- `test_patterns_feedback.py` - Feedback collection system - -**Infrastructure Tests**: -- `test_backup.py` - Backup system -- `test_logging.py` - Logging configuration -- `test_metrics.py` - Metrics collection -- `test_prometheus.py` - Prometheus integration -- `test_telemetry.py` - Telemetry - -### **Coverage** -- Overall: ~84% code coverage -- Analyzers: 96.8% coverage -- HTML coverage reports in `htmlcov/` - ---- - -## 📚 Documentation Structure - -### **User Documentation** (`docs/`) -- `TUTORIAL.md` - Getting started tutorial -- `QUICK_REFERENCE.md` - Quick reference guide -- `ERROR_HANDLING.md` - Error handling guide -- `FALSE_POSITIVE_REDUCTION.md` - Reducing false positives -- `MONITORING.md` - Monitoring and metrics guide -- `PERFORMANCE_OPTIMIZATION.md` - Performance optimization guide - -### **Developer Documentation** (Root) -- `ARCHITECTURE.md` - Architecture overview -- `CONTRIBUTING.md` - Contribution guidelines -- `CODE_OF_CONDUCT.md` - Code of conduct -- `SECURITY.md` - Security policy -- `CHANGELOG.md` - Version history -- `FEATURES.md` - Complete feature list -- `README.md` - Project overview - - ---- - -## 🔧 Build & Distribution - -### **Package Configuration** (`pyproject.toml`) -- **Build System**: setuptools -- **Python Versions**: 3.8, 3.9, 3.10, 3.11, 3.12 -- **Entry Point**: `refactron = refactron.cli:main` - -### **Distribution Files** (`dist/`) -- Wheel files (`.whl`) -- Source distributions (`.tar.gz`) -- Version 1.0.0 (current stable) -- Version 0.1.0b1 (beta release) - -### **Development Setup** -- `setup_dev.sh` - Automated setup for macOS/Linux -- `setup_dev.bat` - Automated setup for Windows -- Creates venv, installs dependencies, sets up pre-commit hooks - ---- - -## 🚀 Key Features by Module - -### **Analysis Capabilities** -- **48 detection rules** across 7 analyzer types -- AST-based analysis (fast, accurate) -- Context-aware confidence scoring -- Configurable thresholds and rules - -### **Refactoring Capabilities** -- **5 refactoring operations** -- Preview mode with diffs -- Risk scoring (0.0-1.0) -- Safe by default (preview before apply) - -### **Auto-Fix Capabilities** -- **14 automated fixers** -- Risk-based safety levels -- AST-based transformations (deterministic) -- Backup and rollback support - -### **Configuration Management** -- **Environment-specific profiles** (dev, staging, prod) -- **Schema validation** with actionable error messages -- **Config inheritance** and composition (base + overrides) -- **Framework templates** (Django, FastAPI, Flask, base) -- **Configuration versioning** and backward compatibility -- **CLI profile support** (`--profile`, `--environment` options) -- **Template selection** (`refactron init --template`) - -### **Pattern Learning System** -- **Feedback Collection** - Tracks developer acceptance/rejection of refactorings -- **Pattern Fingerprinting** - AST-based code pattern identification -- **Pattern Storage** - Thread-safe, persistent JSON storage -- **Pattern Matching** - Intelligent matching with scoring algorithms -- **Operation Tracking** - Unique IDs for all refactoring operations -- **Project-Specific Learning** - Project root detection and context awareness - -### **Performance Features** -- AST caching for repeated analysis -- Incremental analysis (only changed files) -- Parallel processing (multi-threaded/multi-process) -- Memory profiling and optimization - -### **Observability Features** -- Structured logging (JSON/text) -- Metrics collection -- Prometheus integration -- Telemetry (opt-in) - ---- - -## 📊 Code Metrics Summary - -| Metric | Value | -|--------|-------| -| **Total Python Files** | 93 (51 library + 31 tests + 11 examples) | -| **Library Code** | ~11,000+ lines | -| **Test Code** | ~9,500+ lines | -| **Test Coverage** | ~84% overall, 96.8% analyzers | -| **Analyzers** | 7 types, 48 rules | -| **Refactorers** | 5 types | -| **Auto-Fixers** | 14 fixers | -| **CLI Commands** | 10 commands | -| **Pattern Learning** | Feedback collection, fingerprinting, matching | -| **Dependencies** | 6 runtime, 6 development | - ---- - -## 🎯 Design Patterns Used - -1. **Strategy Pattern** - Analyzers and refactorers are pluggable strategies -2. **Template Method** - Base classes define the interface -3. **Factory Pattern** - Configuration-driven analyzer/refactorer creation -4. **Observer Pattern** - Metrics and telemetry collection -5. **Builder Pattern** - Complex result objects built incrementally -6. **Singleton Pattern** - Metrics and telemetry collectors - ---- - -## 🔐 Security & Safety - -- **Preview Mode Default** - Changes shown before applying -- **Backup System** - Automatic backups before modifications -- **Rollback Support** - Easy restoration of previous versions -- **Risk Scoring** - Each operation has a safety score -- **Git Integration** - Uses Git for rollback when available - ---- - -## 📈 Future Extensibility - -### **Placeholder Modules** (ready for future features) -- `refactron/ai/` - AI-powered features -- `refactron/multifile/` - Multi-file analysis -- `refactron/rules/` - Custom rule definitions - -### **Pattern Learning System** (`refactron/patterns/`) -- **`models.py`** - Data models for feedback, patterns, metrics, and project profiles -- **`storage.py`** - Thread-safe JSON storage for pattern learning data -- **`fingerprint.py`** - AST-based code pattern fingerprinting -- **`matcher.py`** - Pattern matching with scoring algorithms -- Integrated into `Refactron` class for feedback collection and pattern learning - -### **Extension Points** -- Custom analyzers (inherit from `BaseAnalyzer`) -- Custom refactorers (inherit from `BaseRefactorer`) -- Custom fixers (inherit from `BaseFixer`) -- Custom reporters (extend report generation) - ---- - -## ✅ Code Quality Standards - -- **Type Hints** - Full type annotations (mypy checked) -- **Code Formatting** - Black (100 char line length) -- **Import Sorting** - isort (black-compatible) -- **Linting** - flake8 compliance -- **Testing** - pytest with coverage reporting -- **Documentation** - Comprehensive docstrings - ---- - -## 🎉 Summary - -Refactron is a **well-architected, production-ready** Python code analysis and refactoring library with: - -✅ **Modular design** - Easy to extend and maintain -✅ **Comprehensive analysis** - 48 rules across 7 categories -✅ **Safe refactoring** - Preview mode, risk scoring, backups -✅ **Automated fixes** - 14 fixers for common issues -✅ **Pattern Learning** - Feedback collection and pattern recognition system -✅ **Production-ready** - Performance optimizations, monitoring, logging -✅ **Well-tested** - 84% coverage, 31 test files -✅ **Well-documented** - Comprehensive docs and examples -✅ **Developer-friendly** - Automated setup, clear architecture - -**The codebase demonstrates professional software engineering practices and is ready for open-source contributions and production use.** diff --git a/FEATURES.md b/FEATURES.md deleted file mode 100644 index 8b65965..0000000 --- a/FEATURES.md +++ /dev/null @@ -1,784 +0,0 @@ -# 🚀 Refactron - Complete Feature List - -**The Intelligent Code Refactoring Transformer** - -This document provides a comprehensive overview of all features and capabilities in the Refactron library. - ---- - -## 📊 Core Capabilities - -### 🔍 **Code Analysis** (7 Analyzers) - -Refactron analyzes Python code using AST (Abstract Syntax Tree) to detect issues across multiple categories: - ---- - -## 1. 🔒 Security Analyzer (13 Rules) - -Detects security vulnerabilities and unsafe code patterns with context-aware confidence scoring. - -### Rule IDs & What They Detect: - -- **SEC001** - Dangerous functions (`eval()`, `exec()`, `compile()`, `__import__`, `input()`) -- **SEC002** - Dangerous modules (`pickle`, `marshal`, `shelve`) - unsafe deserialization -- **SEC003** - Hardcoded secrets (passwords, API keys, tokens, credentials) -- **SEC004** - SQL injection vulnerabilities (f-strings, % formatting in SQL queries) -- **SEC005** - Command injection risks (`os.system`, `subprocess` with `shell=True`) -- **SEC006** - Weak cryptographic algorithms (MD5, SHA1) -- **SEC007** - Unsafe YAML loading (`yaml.load()` instead of `yaml.safe_load()`) -- **SEC008** - Assert statements used for security checks (can be disabled with -O flag) -- **SEC009** - SQL parameterization issues (string concatenation, `.format()` in SQL queries) -- **SEC010** - SSRF vulnerabilities (dynamic URLs in HTTP requests) -- **SEC011** - Insecure random module usage (warns about `random` module) -- **SEC012** - Weak SSL/TLS configuration (CERT_NONE in SSL contexts) -- **SEC013** - SSL verification disabled (`verify=False` in requests) - -### Features: -- Context-aware confidence scoring (0.0-1.0) -- Lower confidence for test files and examples -- File-based ignore patterns -- Rule-specific whitelist support -- False positive filtering - ---- - -## 2. ⚡ Performance Analyzer (6 Rules) - -Detects performance bottlenecks and inefficient code patterns. - -### Rule IDs & What They Detect: - -- **P001** - N+1 query antipattern (database queries inside loops) -- **P002** - Inefficient list comprehension patterns (`list(filter(...))`, `list(map(...))`) -- **P003** - Deeply nested list comprehensions (depth > 2 levels) -- **P004** - Unnecessary iterations (same variable iterated multiple times in a function) -- **P005** - Inefficient string concatenation (`+=` in loops) -- **P006** - Redundant list() calls (wrapping list comprehensions) - ---- - -## 3. 🧠 Complexity Analyzer (4 Rules) - -Measures and detects code complexity issues. - -### Rule IDs & What They Detect: - -- **C001** - High cyclomatic complexity (functions exceeding threshold) -- **C002** - Long functions (exceeding max_function_length threshold) -- **C003** - Deeply nested loops (nesting depth > 3) -- **C004** - Complex method call chains (chains longer than 4 calls) - -### Additional Metrics: -- **M001** - Low maintainability index (MI score < 20) - ---- - -## 4. 👃 Code Smell Analyzer (7 Rules) - -Detects code smells and anti-patterns that reduce code quality. - -### Rule IDs & What They Detect: - -- **S001** - Too many parameters (functions with excessive parameters) -- **S002** - Deep nesting (excessive nesting depth) -- **S003** - Duplicate code (functions with numbered suffixes suggesting duplication) -- **S004** - Magic numbers (unexplained numeric constants) -- **S005** - Missing docstrings (functions/classes without documentation) -- **S006** - Unused imports (imported modules that are never used) -- **S007** - Repeated code blocks (3+ consecutive statements repeated within functions) - ---- - -## 5. 💀 Dead Code Analyzer (6 Rules) - -Identifies unused and unreachable code. - -### Rule IDs & What They Detect: - -- **DEAD001** - Unused functions (defined but never called) -- **DEAD002** - Unused variables (assigned but never used) -- **DEAD003** - Unreachable code (code after return/break statements) -- **DEAD004** - Empty functions (functions with only `pass`) -- **DEAD005** - Always-true/false conditions (`if True:`, `if False:`) -- **DEAD006** - Redundant comparisons (`if x == True:`, `if x == False:`) - ---- - -## 6. 🔗 Dependency Analyzer (7 Rules) - -Analyzes import statements and module dependencies. - -### Rule IDs & What They Detect: - -- **DEP001** - Unused imports (imported modules never used) -- **DEP002** - Wildcard imports (`from module import *`) -- **DEP003** - Circular imports (imports inside functions, suggesting circular dependencies) -- **DEP004** - Import order violations (not following PEP 8: stdlib, third-party, local) -- **DEP005** - Relative imports (using `from . import` instead of absolute imports) -- **DEP006** - Duplicate imports (same module imported multiple times) -- **DEP007** - Deprecated modules (`imp`, `optparse`, `xml.etree.cElementTree`) - ---- - -## 7. 📝 Type Hint Analyzer (5 Rules) - -Checks for missing or incomplete type annotations. - -### Rule IDs & What They Detect: - -- **TYPE001** - Missing return type annotations -- **TYPE002** - Missing parameter type annotations -- **TYPE003** - Missing class attribute type annotations -- **TYPE004** - Usage of `Any` type (defeats type checking) -- **TYPE005** - Incomplete generic types (`List`, `Dict`, `Set`, `Tuple` without element types) - ---- - -## 🔧 Refactoring Operations (5 Types) - -Suggests and applies safe code transformations with preview and risk scoring. - -### Refactoring Types: - -1. **Extract Method** (`extract_method`) - - Suggests extracting complex code blocks into separate methods - - Identifies functions with multiple logical blocks - -2. **Extract Constant** (`extract_constant`) - - Replaces magic numbers with named constants - - Groups related magic numbers in functions - -3. **Simplify Conditionals** (`simplify_conditionals`) - - Transforms nested `if` statements into guard clauses - - Improves code readability - -4. **Reduce Parameters** (`reduce_parameters`) - - Suggests converting parameter lists into configuration objects - - Reduces function parameter count - -5. **Add Docstrings** (`add_docstring`) - - Automatically generates docstrings for functions and classes - - Uses context-aware documentation generation - -### Refactoring Features: -- Preview mode (see changes before applying) -- Risk scoring (0.0 = perfectly safe, 1.0 = high risk) -- Before/after code previews -- Selective application (apply specific refactoring types) -- **Unique operation IDs** - Each refactoring gets a unique identifier for tracking -- **Pattern fingerprinting** - Code patterns are automatically fingerprinted for learning -- **Feedback collection** - Track which refactorings are accepted, rejected, or ignored - -### Suggestion Ranking: - -Refactron ranks refactoring suggestions based on learned patterns: - -- **RefactoringRanker** (`refactron/patterns/ranker.py`) ranks operations by predicted value -- **Scoring Factors**: - - Pattern acceptance rate - - Project-specific weights from project profiles - - Pattern recency and frequency - - Metrics improvements (complexity, maintainability) - - Risk penalty (higher risk = lower score) -- **Integration**: - - `Refactron.refactor()` ranks operations before display - - Ranking scores are stored in `operation.metadata["ranking_score"]` - - CLI preview shows `Ranking Score: X.XXX` per operation - - Summary shows `📊 N operations ranked by learned patterns` - ---- - -## 🤖 Auto-Fix Engine (14 Fixers) - -Automatically fixes common issues using AST-based transformations (no AI APIs required!). - -### Available Fixers: - -1. **RemoveUnusedImportsFixer** - Removes unused import statements -2. **ExtractMagicNumbersFixer** - Extracts magic numbers into constants -3. **AddDocstringsFixer** - Adds docstrings to functions/classes -4. **RemoveDeadCodeFixer** - Removes unreachable code -5. **FixTypeHintsFixer** - Adds missing type hints -6. **SortImportsFixer** - Sorts and organizes imports (PEP 8 compliant) -7. **RemoveTrailingWhitespaceFixer** - Removes trailing whitespace -8. **NormalizeQuotesFixer** - Normalizes quote style (single/double) -9. **SimplifyBooleanFixer** - Simplifies boolean expressions -10. **ConvertToFStringFixer** - Converts string formatting to f-strings -11. **RemoveUnusedVariablesFixer** - Removes unused variables -12. **FixIndentationFixer** - Fixes indentation issues -13. **AddMissingCommasFixer** - Adds missing commas in sequences -14. **RemovePrintStatementsFixer** - Removes debug print statements - -### Auto-Fix Features: -- Risk-based safety levels (safe, low, moderate, high) -- Preview mode before applying -- Backup and rollback support -- AST-based transformations (reliable, deterministic) - ---- - -## 📊 Reporting & Output - -### Report Formats: -- **Text** - Human-readable console output -- **JSON** - Machine-readable format for CI/CD integration -- **HTML** - Visual report with formatting - -### Report Contents: -- Issue categorization by type and severity -- Detailed issue descriptions with suggestions -- Code snippets for each issue -- File-level and project-level summaries -- Technical debt quantification - ---- - -## 🎛️ CLI Commands - -### Core Commands: - -1. **`refactron analyze `** - - Analyze code for issues - - Options: `--detailed`, `--config`, `--log-level`, `--metrics`, `--show-metrics` - -2. **`refactron refactor `** - - Suggest refactoring operations - - Options: `--preview`, `--apply`, `-t` (filter by type), `--feedback` (collect feedback) - -3. **`refactron report `** - - Generate detailed reports - - Options: `--format` (text/json/html), `--output` - -4. **`refactron autofix `** - - Automatically fix issues - - Options: `--preview`, `--apply`, `--safety-level` - -5. **`refactron init`** - - Initialize configuration file - -6. **`refactron rollback`** - - Rollback changes using backups or Git - - Options: `--session`, `--use-git`, `--list`, `--clear` - -7. **`refactron telemetry `** - - Manage telemetry (enable/disable/show) - -8. **`refactron metrics `** - - Show metrics (json/text) - -9. **`refactron serve-metrics `** - - Start Prometheus metrics server - -10. **`refactron feedback `** - - Provide feedback on refactoring operations - - Options: `--action` (accepted/rejected/ignored), `--reason`, `--config` - -11. **`refactron patterns `** - - Project-specific pattern analysis and tuning - - Subcommands: - - `analyze` - Show project pattern statistics - - `recommend` - Show tuning recommendations - - `tune` - Apply recommendations - - `profile` - Show current project profile - ---- - -## ⚙️ Configuration & Customization - -### Configurable Settings: - -- **Enabled Analyzers** - Enable/disable specific analyzers -- **Enabled Refactorers** - Enable/disable specific refactoring types -- **Complexity Thresholds** - Customize max complexity, function length, parameters -- **File Patterns** - Include/exclude specific files or directories -- **Security Settings** - Ignore patterns, rule whitelists, confidence thresholds -- **Performance Optimizations** - AST caching, incremental analysis, parallel processing -- **Logging** - Log levels, formats (JSON/text), file logging -- **Metrics** - Enable metrics collection, Prometheus integration -- **Telemetry** - Opt-in telemetry collection -- **Pattern Learning** - Enable/disable pattern learning, ranking, and custom storage paths - -### 🎯 Advanced Configuration Management - -Refactron now supports enterprise-grade configuration management with profiles, validation, templates, and versioning. - -#### **Environment-Specific Profiles** - -Switch between development, staging, and production configurations: - -```bash -# Use development profile -refactron analyze . --profile dev - -# Use production environment (overrides profile) -refactron refactor . --environment prod -``` - -**Available Profiles:** -- **dev** - Development settings (DEBUG logging, detailed metrics) -- **staging** - Staging settings (INFO logging, standard metrics) -- **prod** - Production settings (WARNING logging, Prometheus enabled) - -**Profile Features:** -- Base configuration + profile overrides -- Environment overrides profile -- Nested configuration merging -- CLI options: `--profile` (`-p`) and `--environment` (`-e`) - -#### **Schema Validation** - -Strict configuration validation with actionable error messages: - -- **Type Checking** - Validates all configuration field types -- **Value Validation** - Ensures values are within acceptable ranges -- **Required Fields** - Validates required configuration fields -- **Clear Error Messages** - Actionable errors with recovery suggestions -- **Version Compatibility** - Validates configuration version compatibility - -#### **Config Inheritance & Composition** - -Powerful configuration composition with base + overrides: - -```yaml -version: "1.0" -base: - enabled_analyzers: [complexity, security, performance] - max_function_complexity: 10 - -profiles: - dev: - log_level: DEBUG - show_details: true - prod: - log_level: WARNING - enable_prometheus: true -``` - -**Features:** -- Base configuration with shared settings -- Profile-specific overrides -- Nested dictionary merging -- Deep merging for complex configurations - -#### **Framework-Specific Templates** - -Ready-made configuration templates for popular Python frameworks: - -```bash -# Generate Django-specific config -refactron init --template django - -# Generate FastAPI-specific config -refactron init --template fastapi - -# Generate Flask-specific config -refactron init --template flask -``` - -**Available Templates:** -- **base** - Default template (all frameworks) -- **django** - Django-specific exclusions (migrations, settings.py, etc.) -- **fastapi** - FastAPI-specific settings (higher complexity thresholds for routes) -- **flask** - Flask-specific settings (blueprint support) - -**Template Features:** -- Pre-configured exclude patterns -- Framework-specific complexity thresholds -- Custom rule configurations -- Profile definitions included - -#### **Configuration Versioning** - -Version-aware configuration system with backward compatibility: - -- **Version Tracking** - Config files include version field -- **Migration Support** - Automatic version compatibility checking -- **Backward Compatibility** - Legacy configs continue to work -- **Future-Proof** - Version support for future enhancements - -#### **Enhanced CLI Options** - -All CLI commands now support profile and environment options: - -```bash -# Analyze with dev profile -refactron analyze . --profile dev - -# Refactor with production environment -refactron refactor . --environment prod --apply - -# Generate report with staging config -refactron report . --profile staging --format html - -# Auto-fix with dev profile -refactron autofix . --profile dev --safety-level safe -``` - -**New Options:** -- `--profile` / `-p` - Select configuration profile (dev, staging, prod) -- `--environment` / `-e` - Override with environment (overrides profile) -- `--template` / `-t` - Select template when running `refactron init` - -#### **Configuration Files** - -Enhanced YAML configuration structure: - -```yaml -version: "1.0" -base: - # All base settings here -profiles: - dev: - # Dev overrides - staging: - # Staging overrides - prod: - # Prod overrides -``` - -**Benefits:** -- Single config file for all environments -- Easy switching between environments -- Reduced configuration duplication -- Better organization and maintainability - ---- - -## 🧠 Pattern Learning System - -Refactron learns from developer feedback to improve refactoring suggestions over time. - -### **Feedback Collection** - -Tracks developer actions on refactoring suggestions: - -- **Operation Tracking** - Each refactoring operation has a unique UUID for tracking -- **Feedback Recording** - Record acceptance, rejection, or ignoring of suggestions -- **Interactive Collection** - Prompt for feedback during preview with `--feedback` flag -- **Auto-Recording** - Automatically records feedback when using `--apply` flag -- **Manual Entry** - Use `refactron feedback ` to provide feedback later - -### **Pattern Fingerprinting** - -AST-based code pattern identification: - -- **Normalized Fingerprinting** - Removes whitespace, comments for consistent matching -- **AST Pattern Extraction** - Captures structural patterns, not just text -- **Hash-Based Identification** - SHA256 hashing for fast pattern lookups -- **Automatic Integration** - Pattern hashes stored in operation metadata - -### **Pattern Storage** - -Thread-safe, persistent storage system: - -- **Project-Specific Storage** - Stores patterns in `.refactron/patterns/` directory -- **Fallback to User Home** - Uses `~/.refactron/patterns/` if no project root found -- **JSON Persistence** - Human-readable JSON format for easy inspection -- **Thread-Safe Operations** - Uses RLock for concurrent access -- **In-Memory Caching** - Reduces file I/O for better performance - -### **Pattern Matching** - -Intelligent pattern matching with scoring: - -- **Hash-Based Lookups** - O(1) pattern matching using hash indexing -- **Similarity Scoring** - Combines acceptance rate, project weights, recency, and frequency -- **Project Context** - Project-specific pattern weights for personalized suggestions -- **Cache Management** - TTL-based caching for pattern data - -### **Pattern Learning Engine** - -Automatic learning from feedback to improve pattern database: - -- **Automatic Learning** - Learns from feedback automatically when recorded -- **Single Feedback Learning** - `learn_from_feedback()` processes individual feedback records -- **Batch Learning** - `batch_learn()` efficiently processes multiple feedback records at once -- **Pattern Metrics Updates** - Tracks before/after code metrics (complexity, maintainability, LOC) -- **Pattern Score Recalculation** - Automatically updates pattern benefit scores based on metrics -- **Pattern Statistics** - Updates acceptance rates, occurrence counts, and benefit scores -- **Background Processing** - `LearningService` processes pending feedback in background -- **Pattern Cleanup** - Automatically removes old patterns (90+ days inactive by default) -- **Operation Reconstruction** - Reconstructs operations from feedback metadata for learning - -### Suggestion Ranking: - -Refactron ranks refactoring suggestions based on learned patterns: - -- **RefactoringRanker** (`refactron/patterns/ranker.py`) ranks operations by predicted value -- **Scoring Factors**: - - Pattern acceptance rate - - Project-specific weights from project profiles - - Pattern recency and frequency - - Metrics improvements (complexity, maintainability) - - Risk penalty (higher risk = lower score) -- **Integration**: - - `Refactron.refactor()` ranks operations before display - - Ranking scores are stored in `operation.metadata["ranking_score"]` - - CLI preview shows `Ranking Score: X.XXX` per operation - - Summary shows `📊 N operations ranked by learned patterns` - -### **Project-Specific Rule Tuning** - -Customize pattern behavior per project based on historical feedback: - -- **RuleTuner** (`refactron/patterns/tuner.py`) analyzes project-specific pattern history -- **Enable/Disable Patterns** per project based on acceptance rates and feedback volume -- **Pattern Weights** per project to influence ranking scores -- **Safe Defaults** - Requires sufficient feedback before making tuning decisions -- **CLI Integration** - `refactron patterns analyze/recommend/tune/profile` - -### **Learning Service Features** - -Background service for pattern maintenance: - -- **Pending Feedback Processing** - Processes all unprocessed feedback records -- **Score Updates** - Recalculates all pattern scores based on latest feedback -- **Pattern Cleanup** - Removes patterns that haven't been seen recently -- **Configurable Retention** - Customizable retention period for old patterns -- **Efficient Batch Operations** - Optimized for processing large amounts of feedback - -### **Usage Examples** - -```bash -# Collect feedback interactively during preview -refactron refactor file.py --preview --feedback - -# Auto-record feedback when applying changes -refactron refactor file.py --apply - -# Provide feedback manually later -refactron feedback abc-123-def --action accepted --reason "Improved readability" -``` - -### **Configuration** - -Pattern learning can be configured via `.refactron.yaml` or programmatically: - -**YAML Configuration:** -```yaml -# Pattern learning settings -enable_pattern_learning: true # Master switch (default: true) -pattern_storage_dir: null # Custom storage path (null = auto-detect) -pattern_learning_enabled: true # Enable learning from feedback (default: true) -pattern_ranking_enabled: true # Enable ranking based on patterns (default: true) -``` - -**Python API:** -```python -from refactron import Refactron -from refactron.core.config import RefactronConfig - -# Disable pattern learning entirely -config = RefactronConfig(enable_pattern_learning=False) -refactron = Refactron(config) - -# Enable pattern learning but disable ranking -config = RefactronConfig( - enable_pattern_learning=True, - pattern_ranking_enabled=False -) -refactron = Refactron(config) - -# Use custom storage directory -config = RefactronConfig( - enable_pattern_learning=True, - pattern_storage_dir=Path("/custom/path/patterns") -) -refactron = Refactron(config) -``` - -**Configuration Options:** - -- **`enable_pattern_learning`** (bool, default: `true`) - - Master switch for all pattern learning features - - When `false`: No pattern components initialized, no storage, no learning, no ranking - - When `true`: Pattern system initializes (subject to other flags) - -- **`pattern_storage_dir`** (Path | null, default: `null`) - - Custom directory for pattern data storage - - `null`: Auto-detects project root (`.refactron/patterns/`) or falls back to `~/.refactron/patterns/` - - Set to a Path to use a specific directory - -- **`pattern_learning_enabled`** (bool, default: `true`) - - Controls whether Refactron learns from feedback - - When `false`: Feedback is recorded but not used for learning - - When `true`: Feedback automatically triggers pattern learning - -- **`pattern_ranking_enabled`** (bool, default: `true`) - - Controls whether refactoring suggestions are ranked by learned patterns - - When `false`: Operations returned in original order, no ranking scores - - When `true`: Operations ranked by pattern acceptance rates and project-specific weights - -**Disabling Pattern Learning:** - -To completely disable pattern learning: -```yaml -enable_pattern_learning: false -``` - -To disable only learning (keep ranking): -```yaml -enable_pattern_learning: true -pattern_learning_enabled: false -pattern_ranking_enabled: true -``` - -To disable only ranking (keep learning): -```yaml -enable_pattern_learning: true -pattern_learning_enabled: true -pattern_ranking_enabled: false -``` - -### **Integration** - -Seamlessly integrated into Refactron: - -- **Automatic Fingerprinting** - Code patterns fingerprinted automatically during refactoring -- **Automatic Learning** - Pattern learning triggered automatically when feedback is recorded -- **Non-Blocking** - Learning failures don't interrupt feedback recording -- **Graceful Degradation** - Works even if pattern storage unavailable -- **Non-Invasive** - Doesn't slow down or interfere with normal refactoring operations -- **Backward Compatible** - Existing workflows continue to work without changes -- **Config-Driven** - All features can be enabled/disabled via configuration - -### **Learning Workflow** - -How pattern learning works: - -1. **Feedback Collection** - Developer provides feedback (accepted/rejected/ignored) -2. **Pattern Identification** - System identifies or creates pattern from code hash -3. **Statistics Update** - Pattern acceptance rate and counts updated -4. **Metrics Calculation** - If accepted, before/after metrics calculated and stored -5. **Score Recalculation** - Pattern benefit score updated based on metrics -6. **Pattern Storage** - Updated pattern saved to persistent storage -7. **Background Maintenance** - LearningService periodically updates scores and cleans old patterns - ---- - -## 🚀 Performance Features - -### Optimizations: - -1. **AST Caching** - Caches parsed ASTs for faster repeated analysis -2. **Incremental Analysis** - Only analyzes changed files -3. **Parallel Processing** - Multi-threaded/multi-process file analysis -4. **Memory Profiling** - Tracks memory usage during analysis -5. **Metrics Collection** - Detailed performance metrics - ---- - -## 🔐 Safety Features - -### Backup & Recovery: -- **Automatic Backups** - Creates backups before applying changes -- **Git Integration** - Uses Git for rollback when available -- **Session Management** - Tracks backup sessions -- **Rollback Support** - Easy restoration of previous versions - -### Preview Mode: -- **Safe Default** - Preview mode enabled by default -- **Before/After Diffs** - See exact changes before applying -- **Risk Scoring** - Know the safety level of each operation - ---- - -## 📈 Monitoring & Metrics - -### Metrics Collection: -- Analysis time per file -- Issues found by category -- Analyzer performance -- Memory usage tracking -- File analysis success/failure rates - -### Prometheus Integration: -- Metrics server for production monitoring -- Customizable host and port -- Real-time metrics export - -### Telemetry (Opt-in): -- Usage statistics -- Feature adoption -- Error tracking -- Performance metrics - ---- - -## 🔌 Integration Features - -### Python API: -```python -from refactron import Refactron - -refactron = Refactron() -analysis = refactron.analyze("mycode.py") -result = refactron.refactor("mycode.py", preview=True) -``` - -### CLI Integration: -- Can be integrated into CI/CD pipelines -- Exit codes for automation -- JSON output for programmatic processing - ---- - -## 📚 Advanced Features - -### False Positive Tracking: -- Learn from false positives -- Filter known false positives -- Improve accuracy over time - -### Context-Aware Analysis: -- Different confidence scores for test files -- Lower confidence for example/demo files -- File-type specific rules - -### Structured Logging: -- JSON format for log aggregation -- Text format for human reading -- Configurable log levels -- File rotation support - ---- - -## 📊 Summary Statistics - -### Total Detection Rules: **48 Rules** -- Security: 13 rules -- Performance: 6 rules -- Complexity: 4 rules -- Code Smells: 7 rules -- Dead Code: 6 rules -- Dependencies: 7 rules -- Type Hints: 5 rules - -### Refactoring Operations: **5 Types** -### Auto-Fix Capabilities: **14 Fixers** -### CLI Commands: **11 Commands** -### Report Formats: **3 Formats** (Text, JSON, HTML) - ---- - -## 🎯 Use Cases - -Refactron is perfect for: - -- **Code Reviews** - Automated issue detection before review -- **Legacy Code Modernization** - Identify and fix technical debt -- **Security Audits** - Find security vulnerabilities automatically -- **Performance Optimization** - Detect performance bottlenecks -- **Code Quality Improvement** - Maintain consistent code quality -- **Technical Debt Tracking** - Quantify and monitor technical debt -- **CI/CD Integration** - Automated quality checks in pipelines -- **Developer Onboarding** - Help new developers understand code quality standards -- **Pattern Learning** - Improve suggestions based on historical feedback -- **Smart Recommendations** - Get better refactoring suggestions over time - ---- - -**Refactron - Making Python code better, one refactoring at a time!** 🚀 - -**Now with Pattern Learning** - The more you use Refactron, the smarter it gets! 🧠