Release v3.4.0

🎉 What's New in v3.4.0

This release brings three major features and improvements to cel2sql!

🔢 Multi-Dimensional Array Support (fixes #49)

Full support for PostgreSQL multi-dimensional arrays with automatic dimension detection!

Features:

• ✨ Automatic dimension detection from PostgreSQL type strings
• 📊 Support for 1D, 2D, 3D, and 4D+ arrays
• 🔧 Works with both bracket notation (integer[][]) and underscore notation (_int4[])
• 🔄 Full backward compatibility (defaults to 1D)

Example:

// 2D array in schema
schema := pg.NewSchema([]pg.FieldSchema{
    {Name: "matrix", Type: "integer", Repeated: true, Dimensions: 2},
})

// CEL expression
ast, _ := env.Compile("size(data.matrix) > 0")

// Generated SQL
// COALESCE(ARRAY_LENGTH(data.matrix, 2), 0) > 0

Testing:

• 22 unit tests for dimension detection
• Integration tests for 1D-4D arrays
• Backward compatibility tests

---

🔤 CEL String Extension Functions (fixes #87)

Implemented three powerful CEL string extension functions with PostgreSQL conversion!

Functions:

• split(delimiter [, limit]) → STRING_TO_ARRAY()
  • Full limit support (0, 1, -1, N)
  • Handles edge cases correctly
• join([delimiter]) → ARRAY_TO_STRING()
  • Optional delimiter (defaults to empty string)
  • Null handling
• format(args) → FORMAT()
  • Supported specifiers: %s, %d, %f
  • Format string validation (max 1000 chars)
  • Argument count validation

Features:

• ✅ All functions work in comprehensions (exists, all, filter, map)
• 🔒 Security validations: null byte checks, format string limits, specifier whitelisting
• 📚 100+ test cases
• 📈 Performance benchmarks
• 💡 Working example in examples/string_extensions/

Example:

// CEL
"a,b,c".split(",")

// SQL
STRING_TO_ARRAY('a,b,c', ',')

---

🛡️ Comprehension Pattern Matching Validation (fixes #44)

Strengthened comprehension pattern matching to prevent potential misidentification of custom accumulator expressions.

Improvements:

• Enhanced result expression validation for all comprehension types
• Stricter validation of accumulator increment patterns
• Added 22 comprehensive edge case tests

Test Coverage:

• Pattern detection order verification
• Complex nested expressions
• Empty lists handling
• Chained operations
• Variable name edge cases

---

📦 Installation

go get github.com/spandigital/cel2sql/v3@v3.4.0

📖 Documentation

• README
• CHANGELOG
• Examples

🙏 Contributors

Thanks to all contributors who made this release possible!

🤖 Generated with Claude Code

Release v3.3.0

What's New in v3.3.0

This release fixes critical bugs and adds comprehensive support for CEL string extension functions.

🐛 Bug Fixes

LIKE Pattern Escaping (fixes #40, #43, #84)

• Fixed LIKE pattern escaping to use ESCAPE E'\\' syntax for proper backslash handling
• Updated startsWith() and endsWith() to properly escape special LIKE characters (%, _, \)
• Prevents SQL syntax errors from patterns containing backslashes

JSON Comprehensions (fixes #48, #84)

• Fixed comprehensions over JSON arrays to properly use jsonb_array_elements()
• Corrects SQL generation for expressions like data.items.all(i, i.quantity > 0)
• Ensures JSON array comprehensions work correctly with PostgreSQL

String Functions Panic (fixes #85, #86)

• Critical Fix: Eliminated panic when using CEL string extension functions as methods
• Added defensive checks in callCasting, visitCallIndex, visitCallMapIndex, visitCallListIndex, visitCallUnary
• All string functions now properly handle both method calls (target) and function calls (args)
• No more panic: index out of range [0] errors!

✨ New Features

CEL String Extension Functions (#86)

Implemented 10 CEL string extension functions with PostgreSQL SQL conversion:

CEL Function	PostgreSQL SQL	Example
lowerAscii()	LOWER()	"HELLO".lowerAscii() → LOWER('HELLO')
upperAscii()	UPPER()	"hello".upperAscii() → UPPER('hello')
trim()	TRIM()	" text ".trim() → TRIM(' text ')
charAt(index)	SUBSTRING(str, index+1, 1)	"test".charAt(0) → SUBSTRING('test', 1, 1)
indexOf(search, [offset])	POSITION()	"hello".indexOf('e') → returns 1
lastIndexOf(search)	REVERSE() logic	"hello".lastIndexOf('l') → returns 3
substring(start, [end])	SUBSTRING()	"hello".substring(0, 3) → SUBSTRING('hello', 1, 3)
replace(old, new, [limit])	REPLACE()	"hello".replace('l', 'L') → REPLACE('hello', 'l', 'L')
reverse()	REVERSE()	"hello".reverse() → REVERSE('hello')

Features:

• Works in comprehensions: data.tags.map(t, t.upperAscii())
• Works as standalone expressions: item.name.trim() == "test"
• Clear error messages for unsupported functions (split, join, format, quote)
• Comprehensive test coverage in string_functions_test.go

📊 Testing

• ✅ All 6 failing test cases from issue #85 now pass
• ✅ Comprehensive test suite for all string functions
• ✅ Panic prevention tests
• ✅ All tests pass: make test
• ✅ All lint checks pass: make lint

🔗 Related Issues

• Fixes #40 - LIKE pattern escaping
• Fixes #43 - Backslash handling in patterns
• Fixes #48 - JSON array comprehensions
• Fixes #85 - String functions panic
• Closes #84 - Phase 1 Correctness
• Closes #86 - String extension functions PR

📦 Installation

go get github.com/spandigital/cel2sql/v3@v3.3.0

🙏 Credits

This release includes contributions and bug reports from the community. Thank you!

---

Full Changelog: v3.2.0...v3.3.0

Release v3.2.0

What's Changed

Fixed

Standardized Error Message Format (fixes #38)

This release significantly improves error handling throughout cel2sql:

• Added 16 exported sentinel errors to enable programmatic error checking with errors.Is():

  • ErrUnsupportedExpression, ErrInvalidFieldName, ErrInvalidSchema
  • ErrInvalidRegexPattern, ErrMaxDepthExceeded, ErrMaxOutputLengthExceeded
  • ErrInvalidComprehension, ErrMaxComprehensionDepthExceeded
  • ErrInvalidArguments, ErrInvalidTimestampOperation, ErrInvalidDuration
  • ErrInvalidJSONPath, ErrInvalidOperator, ErrUnsupportedType
  • ErrContextCanceled, ErrInvalidByteArrayLength

• Improved error wrapping across ~60 error sites using fmt.Errorf() with %w

• Enhanced error messages with operation context for better debugging

• Maintained security - no credential exposure in error messages

Benefits: Better debugging, programmatic error handling, improved code maintainability

Byte Array Length Validation (fixes #36)

Security improvement to prevent resource exhaustion:

• Added 10,000 byte maximum for byte arrays in non-parameterized mode
• Protection against memory exhaustion from hex-encoded SQL (4x expansion)
• Parameterized mode bypasses limit (bytes passed directly to database)
• Clear error messages guide users when limits exceeded

Security Context: Addresses CWE-400 (Uncontrolled Resource Consumption)

Added

Comprehensive Performance Benchmarks in CI/CD (fixes #52)

Automated performance monitoring and tracking:

• Historical benchmark tracking with data storage on GitHub Pages
• Visual performance charts at https://spandigital.github.io/cel2sql/dev/bench/
• PR comments when performance changes exceed 150%
• Comprehensive coverage: operators, comprehensions, JSON, regex, timestamps, complex expressions

Full Changelog

See CHANGELOG.md for complete release notes including v3.1.0.

Full Changelog: v3.1.0...v3.2.0

Release v3.1.0

Features

• Query Analysis and Index Recommendations (#81, fixes #50)
  • New AnalyzeQuery() function to analyze CEL expressions and recommend database indexes
  • Automatic detection of B-tree, GIN, and GIN with pg_trgm index opportunities
  • Support for JSON path operations, array operations, and regex pattern matching
  • Complete working example in examples/index_analysis/
  • See CLAUDE.md for detailed usage

Documentation

• Regex Conversion Limitations (#80, fixes #46)
  • Added detailed documentation of automatic RE2 to POSIX regex conversions
  • Listed unsupported RE2 features that will return errors
  • Added examples of supported and unsupported patterns

What's Changed

Full Changelog: v3.0.1...v3.1.0

Release v3.0.1

What's Changed

This patch release includes several bug fixes to improve reliability and correctness of SQL generation.

Bug Fixes

• Validate database connection on pool creation by @richardwooding in #79 (fixes #45)
• Use correct modulo arithmetic for getDayOfWeek() to handle Sunday by @richardwooding in #78 (fixes #42)
• Use correct AT TIME ZONE syntax in EXTRACT by @richardwooding in #77 (fixes #41)
• Complete SQL validation coverage by @richardwooding in #76 (fixes #26)
• Wrap ARRAY_LENGTH and jsonb_array_length in COALESCE for NULL safety by @richardwooding in #75 (fixes #25)

Full Changelog: v3.0.0...v3.0.1

Release v3.0.0

Breaking Changes 🚨

This is a major release with breaking API changes to improve performance.

Module Path Update

Required: Update all imports from v2 to v3:

// Old
import "github.com/spandigital/cel2sql/v2"
import "github.com/spandigital/cel2sql/v2/pg"

// New
import "github.com/spandigital/cel2sql/v3"
import "github.com/spandigital/cel2sql/v3/pg"

Schema API Changes

Required: Update schema creation to use constructor:

// Old API (v2.x)
schema := pg.Schema{
    {Name: "field", Type: "text"},
}

// New API (v3.0.0)
schema := pg.NewSchema([]pg.FieldSchema{
    {Name: "field", Type: "text"},
})

// Iteration
for _, field := range schema.Fields() { ... }

// O(1) Lookup
if field, found := schema.FindField("name"); found { ... }

Performance Improvements 🚀

O(1) Constant-Time Field Lookups

Schema field lookups are now O(1) instead of O(n), providing massive performance improvements for large schemas:

Schema Size	Lookup Time	Improvement
10 fields	241.3 ns	baseline
100 fields	241.5 ns	~10x faster
1000 fields	241.5 ns	~100x faster

The lookup time remains constant at ~241ns regardless of schema size!

What's Changed

Core Changes

• pg.Schema changed from type alias to struct with internal hash map indexing
• Added pg.NewSchema() constructor for creating schemas
• Added schema.Fields() method for ordered field iteration
• Added schema.FindField(name) method for O(1) field lookups
• All 44 files updated (code, tests, examples, documentation)

Security

• Added nested comprehension depth limits (max depth: 3)
• Prevents DoS attacks through resource exhaustion (CWE-400)
• Protection against expensive nested UNNEST/subquery operations

Migration Guide

See the CHANGELOG.md for complete migration details.

Full Changelog

Full Changelog: v2.12.1...v3.0.0

Fixes #28

Release v2.12.1

Bug Fixes

• Regex Conversion: Complete RE2 to POSIX regex conversion, ensuring proper handling of all supported RE2 features (#24)
• Type Safety: Fail on unknown PostgreSQL types instead of defaulting to string, preventing silent type mismatches (#30)
• Bytea Literals: Use PostgreSQL hex format (\x...) for bytea literals instead of base64 (#32)
• Array Safety: Add array index overflow protection to prevent out-of-bounds access (#19)

What's Changed

This patch release focuses on improved correctness and safety:

1. Enhanced regex support: The regex conversion now properly handles all supported RE2 features with comprehensive validation
2. Stricter type checking: Unknown types now fail early rather than silently converting to strings
3. PostgreSQL compliance: Bytea literals now use the correct hex format (\x prefix)
4. Array bounds checking: Array access is now validated to prevent potential security issues

Full Changelog: v2.12.0...v2.12.1

Release v2.12.0

New Features

Parameterized Query Support (#67, #29)

• Add ConvertParameterized() function for parameterized SQL generation
• Returns Result struct with SQL and parameter values ($1, $2, etc.)
• Enables query plan caching and provides additional SQL injection protection
• Constants (strings, numbers, bytes) are parameterized
• Boolean and NULL values kept inline for query plan optimization

Example:

result, err := cel2sql.ConvertParameterized(ast,
    cel2sql.WithSchemas(schemas))
// result.SQL: "user.age = $1 AND user.name = $2"
// result.Parameters: []interface{}{18, "John"}

rows, err := db.Query("SELECT * FROM users WHERE "+result.SQL, result.Parameters...)

Recursion Depth Limits (#64, #27)

• Add WithMaxDepth() option to configure maximum expression nesting depth
• Default limit: 100 levels (prevents stack overflow - CWE-674)
• Configurable per-conversion for different use cases
• Clear error messages when limit is exceeded

Example:

sql, err := cel2sql.Convert(ast,
    cel2sql.WithMaxDepth(150))

Nested Comprehension Depth Limits (#35)

• Fixed limit: 3 levels of nested comprehensions
• Prevents resource exhaustion from deeply nested UNNEST/subquery operations (CWE-400)
• Protects against DoS via complex comprehension expressions

Security Enhancements

SQL Output Length Limits (#69, #33)

• Add WithMaxOutputLength() option to limit generated SQL size
• Default limit: 50,000 characters (prevents memory exhaustion - CWE-400)
• Configurable for different deployment environments
• Prevents DoS attacks via extremely large SQL queries

Example:

sql, err := cel2sql.Convert(ast,
    cel2sql.WithMaxOutputLength(100000))

Error Message Sanitization (#66, #34)

• Sanitize error messages to prevent information disclosure
• Removed sensitive data from error outputs
• Prevents accidental exposure of internal details

Connection String Validation (#65, #37)

• Validate PostgreSQL connection strings to prevent credential exposure
• Prevents insecure connection configurations
• Protects against credential leakage in logs

Bug Fixes

Code Quality Improvements (#68)

• Remove ineffectual assignment in wrapConversionError
• Improved code quality and linting compliance

Breaking Changes

None - fully backward compatible.

Upgrade Notes

No changes required for existing code. New features are opt-in via functional options:

// Existing code (still works)
sql, err := cel2sql.Convert(ast)

// New features (optional)
result, err := cel2sql.ConvertParameterized(ast,
    cel2sql.WithSchemas(schemas))

sql, err := cel2sql.Convert(ast,
    cel2sql.WithMaxDepth(150),
    cel2sql.WithMaxOutputLength(100000),
    cel2sql.WithContext(ctx),
    cel2sql.WithSchemas(schemas))

Installation

go get github.com/spandigital/cel2sql/v2@v2.12.0

What's Changed Since v2.11.0

• feat: Add recursion depth limits to prevent stack overflow (#27) (#64)
• fix: Add connection string validation and prevent credential exposure (#37) (#65)
• fix: Sanitize error messages to prevent information disclosure (#34) (#66)
• feat: Add parameterized query support (#29) (#67)
• fix: Remove ineffectual assignment in wrapConversionError (#68)
• feat: Add nested comprehension depth limits (#35)
• fix: Add SQL output length limits to prevent DoS (#33) (#69)

Full Changelog: v2.11.0...v2.12.0

---

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

Release v2.11.0

New Features

Context Support (#31)

• Add WithContext() option for cancellation and timeout protection
• Context checked at key recursion points (visit, visitCall, visitComprehension)
• Enables integration with distributed tracing and observability
• Defense-in-depth protection against complex expressions

Structured Logging (#47)

• Add WithLogger() option for observability using log/slog
• Zero overhead when disabled (uses slog.DiscardHandler by default)
• Comprehensive logging points: JSON detection, comprehensions, schema lookups, regex conversion
• Supports both JSON (production) and text (development) handlers
• Includes performance metrics and error contexts

Functional Options API (#62)

• Refactored to use functional options pattern
• WithSchemas(), WithContext(), WithLogger() options
• Backward compatible - existing code works without changes
• More extensible for future enhancements

Security Enhancements

ReDoS Protection (#63)

• Comprehensive validation prevents catastrophic backtracking (CWE-1333)
• Pattern length limit (500 chars)
• Nested quantifier detection with state machine
• Capture group limit (20 groups)
• Quantified alternation detection
• Nesting depth limit (10 levels)
• Descriptive error messages for blocked patterns

Field Name Validation (#57)

• Prevents SQL injection via malicious field names
• Maximum length (63 chars - PostgreSQL limit)
• Format validation (alphanumeric + underscore only)
• Blocks 60+ SQL reserved keywords
• Empty string rejection

JSON Field Escaping (#58)

• Single quotes in JSON field names automatically escaped
• Prevents SQL injection via JSON paths
• Applied at multiple pipeline stages (defense-in-depth)
• Protects all JSON operations: ->, ->>, ?, jsonb_extract_path_text()

Schema-Based JSON Detection (#62, #60)

• Removed hardcoded JSON field lists
• Schema-based detection via WithSchemas()
• Added IsJSON, IsJSONB, ElementType fields to FieldSchema
• Correct table.column handling with JSON operators

Test Coverage

Coverage Improvements (#51)

• Increased from 55.6% to 80.1% (+24.5pp)
• Zero-coverage functions tested (exists_one, filter comprehensions)
• Edge case tests (constants, string functions, operators)
• JSON/JSONB detection tests
• Final comprehensive tests for remaining gaps

Documentation

Complete Documentation Update

• Updated all docs to reflect v2.11.0 features
• New comprehensive Security Guide (docs/security.md)
• Context support examples and best practices
• ReDoS protection detailed documentation
• Security checklist for production deployment
• Functional options API throughout examples

Breaking Changes

None - fully backward compatible.

Upgrade Notes

No changes required for existing code. New features are opt-in via functional options:

// Before (still works)
sql, err := cel2sql.Convert(ast)

// New features (optional)
sql, err := cel2sql.Convert(ast,
    cel2sql.WithContext(ctx),
    cel2sql.WithSchemas(schemas),
    cel2sql.WithLogger(logger))

Installation

go get github.com/spandigital/cel2sql/v2@v2.11.0

What's Changed Since v2.10.0

• docs: Add Claude Code documentation and custom agents (#56)
• fix: Add field name validation to prevent SQL injection (#57)
• fix: Escape single quotes in JSON field names to prevent SQL injection (#22) (#58)
• fix: Use schema-based JSON field detection and correct table.column handling (#59, #20) (#60)
• fix: Complete schema-based JSON detection and remove hardcoded field lists (#61, #59) (#62)
• feat: Add context support for cancellation and timeout (#31)
• fix: Add comprehensive ReDoS protection to prevent catastrophic backtracking (#23) (#63)
• test: Add coverage tests for zero-coverage functions (#51)
• test: Add Priority 2 coverage tests for JSON, struct, and map functions (#51)
• test: Add Priority 3 edge case tests for moderate-coverage functions (#51)
• test: Add final comprehensive tests to reach 80% coverage goal (#51)
• feat: Add structured logging with log/slog for observability (#47)
• docs: Update documentation for v2.11.0 release
• docs: Fix version number in CLAUDE.md to v2.11.0
• fix: Add package comment to examples/logging/main.go for linter

Full Changelog: v2.10.0...v2.11.0

---

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

Release v2.10.0

🎉 Release v2.10.0

This release adds comprehensive fuzzing infrastructure and fixes critical security vulnerabilities discovered through fuzz testing.

✨ What's New

Comprehensive Fuzzing Infrastructure

• Three specialized fuzz tests for thorough security testing:
  • FuzzConvert - Main conversion pipeline testing
  • FuzzEscapeLikePattern - LIKE pattern escaping validation
  • FuzzConvertRE2ToPOSIX - Regex pattern conversion testing
• Fuzzing dictionary with 100+ tokens (CEL operators, SQL injection vectors, regex patterns, Unicode)
• CI/CD integration:
  • 60-second fuzzing on every PR
  • 10-minute extended runs weekly (Sundays at 2 AM UTC)
  • Manual workflow dispatch with configurable duration
  • Automatic crash artifact preservation
• Immediate results: Discovered 3 security vulnerabilities within minutes of initial CI runs!

🔒 Critical Security Fixes

The fuzzing infrastructure immediately discovered and helped fix three null byte handling vulnerabilities:

1. String literals (#14) - Null bytes in string comparisons causing SQL corruption
2. LIKE patterns (#16) - Null bytes in startsWith() and endsWith() patterns
3. Regex patterns (#18) - Null bytes in matches() function patterns

All three vulnerabilities could have allowed null bytes (\x00) to reach PostgreSQL queries, potentially causing:

• SQL query corruption
• Unexpected query behavior
• Data integrity issues

All users are strongly encouraged to upgrade to this version.

🐛 Bug Fixes

• Replaced panic with proper error handling in timestamp operation validation
• Fixed crash case preserved in testdata/fuzz/FuzzConvert/299bc76ba66bca6b

📚 Documentation

• Completely rewrote README.md for better first-time user experience
• Added beginner-friendly examples and clear getting started guide
• Improved API documentation with real-world use cases

🔧 Technical Details

Null Byte Protection

All string processing entry points are now protected:

• ✅ visitConst() - String literals in comparisons
• ✅ callStartsWith() / callEndsWith() - LIKE pattern generation
• ✅ callMatches() - Regex pattern generation
• ✅ callContains() - POSITION search (via visitConst)

Test Coverage

• 41.3% code coverage maintained
• 100+ unit tests passing
• PostgreSQL 17 integration tests passing
• Continuous fuzzing in CI/CD

📊 Statistics

• 5 PRs merged: #13, #14, #16, #18, and documentation updates
• 3 security issues discovered and fixed by fuzzing
• 579 lines added in fuzzing infrastructure
• 100+ tokens in fuzzing dictionary
• Detection time: Security issues found within 2 seconds to 2 minutes of fuzzing

🙏 Credits

This release demonstrates the exceptional value of comprehensive fuzz testing for security-critical code. The fuzzing infrastructure systematically found all null byte entry points that could have been exploited in production.

📦 Installation

go get github.com/spandigital/cel2sql/v2@v2.10.0

🔗 Links

• Full Changelog
• Documentation
• Fuzzing Infrastructure PR

---

Previous release: v2.9.0