Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

140 changes: 26 additions & 114 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,136 +1,48 @@
# Nvisy Runtime for Rust
# Nvisy Runtime

[![build](https://img.shields.io/github/actions/workflow/status/nvisycom/runtime/build.yaml?branch=main&color=000000&style=flat-square)](https://github.com/nvisycom/runtime/actions/workflows/build.yaml)
[![crates.io](https://img.shields.io/crates/v/nvisy-core?color=000000&style=flat-square)](https://crates.io/crates/nvisy-core)
[![docs.rs](https://img.shields.io/docsrs/nvisy-core?color=000000&style=flat-square)](https://docs.rs/nvisy-core)
[![rust version](https://img.shields.io/badge/Rust-1.89+-000000?style=flat-square&logo=rust&logoColor=white)](https://www.rust-lang.org/)
Document processing runtime for the Nvisy platform.

High-performance runtime library for data redaction and sensitive information
processing.
## Crates

## Features
| Crate | Description |
|-------|-------------|
| `nvisy-rt-core` | Core types, error handling, content management |
| `nvisy-rt-document` | Document abstraction layer and traits |
| `nvisy-rt-engine` | Processing engine with format registry |
| `nvisy-rt-archive` | Archive handling (ZIP, TAR, compression) |
| `nvisy-rt-text` | Text formats (Markdown, JSON, CSV, XML, YAML) |
| `nvisy-rt-pdf` | PDF document support (stub) |
| `nvisy-rt-docx` | Microsoft Word support (stub) |
| `nvisy-rt-image` | Image formats (JPEG, PNG) (stub) |

- Modern Rust 2024 edition with strict type safety
- High-performance async runtime powered by Tokio
- Flexible pattern matching and data detection
- Built-in archive and compression support
- Comprehensive error handling with structured diagnostics
- Modular architecture with optimized crate separation

## Installation

Add the core library to your `Cargo.toml`:
## Usage

```toml
[dependencies]
nvisy-core = "0.1"
nvisy-rt-engine = "0.1"
```

Or install additional crates as needed:

```toml
[dependencies]
nvisy-core = "0.1"
nvisy-engine = "0.1"
nvisy-archive = "0.1"
```

## Quick Start

### Using the Core Library

```rust
use nvisy_core::prelude::*;
use nvisy_rt_engine::Engine;
use nvisy_rt_document::{ContentData, TextExtractor};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Initialize the runtime
let runtime = Runtime::new().await?;

// Process sensitive data
let result = runtime.process("example data").await?;

Ok(())
}
let engine = Engine::new();
let data = ContentData::from("Hello, world!");
let doc = engine.load_by_extension("txt", data).await?;
let text = doc.extract_text().await?;
```

## Architecture

The runtime is organized into specialized crates:

- **nvisy-core** - Core types, traits, and runtime primitives
- **nvisy-engine** - Processing engine and orchestration
- **nvisy-archive** - Archive handling and compression

## Requirements

- Rust 1.89 or higher
- Cargo with workspace support

## Development

### Building

```bash
# Build all crates
cargo build

# Build with release optimizations
cargo build --release

# Build specific crate
cargo build -p nvisy-core
```

### Testing

```bash
# Run all tests
cargo test

# Run tests for specific crate
cargo test -p nvisy-core

# Run with coverage
cargo test --all-features
```

### Linting and Formatting

```bash
# Check formatting
cargo fmt --check

# Format code
cargo fmt

# Run clippy
cargo clippy --all-targets --all-features
```

## Performance

The runtime is designed for high-throughput scenarios:

- Async I/O with Tokio for concurrent request handling
- Memory-mapped file processing for large datasets
- Parallel pattern matching with Rayon
- Zero-copy operations where possible

## Changelog

See [CHANGELOG.md](CHANGELOG.md) for release notes and version history.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for development guidelines.

## License

MIT License - see [LICENSE.txt](LICENSE.txt) for details.
MIT License - see [LICENSE.txt](LICENSE.txt)

## Support

- Documentation: [docs.nvisy.com](https://docs.nvisy.com)
- Issues: [GitHub Issues](https://github.com/nvisycom/runtime/issues)
- Email: [support@nvisy.com](mailto:support@nvisy.com)
- **Documentation**: [docs.nvisy.com](https://docs.nvisy.com)
- **Issues**: [GitHub Issues](https://github.com/nvisycom/server/issues)
- **Email**: [support@nvisy.com](mailto:support@nvisy.com)
- **API Status**: [nvisy.openstatus.dev](https://nvisy.openstatus.dev)
5 changes: 4 additions & 1 deletion crates/nvisy-archive/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,12 @@ tokio = { workspace = true, features = ["fs", "io-util", "rt"] }
tempfile = { workspace = true }

# Macros
derive_more = { workspace = true, features = ["debug", "display", "deref"] }
strum = { workspace = true, features = ["derive"] }

# Logging
tracing = { workspace = true }

# Archive formats (optional)
tar = { version = "0.4", optional = true }
zip = { version = "7.1", optional = true }
Expand All @@ -56,4 +60,3 @@ xz2 = { version = "0.1", optional = true }
[dev-dependencies]
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
tokio-test = { workspace = true }
tempfile = { workspace = true }
44 changes: 21 additions & 23 deletions crates/nvisy-archive/README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,30 @@
# nvisy-archive
# nvisy-rt-archive

Archive handling and compression library for the Nvisy runtime.
Archive handling and compression for the Nvisy runtime.

[![rust](https://img.shields.io/badge/Rust-1.89+-000000?style=flat-square&logo=rust&logoColor=white)](https://www.rust-lang.org/)
## Supported Formats

- **ZIP** - Standard ZIP archives
- **TAR** - Tape archives with optional compression
- **GZIP** - GNU zip compression
- **BZIP2** - Block-sorting compression
- **XZ** - LZMA2 compression
- **7z** - 7-Zip archives (optional)

## Features

- `zip` - ZIP archive support (enabled by default)
- `tar` - TAR archive support (enabled by default)
All features except `sevenz` are enabled by default:

- `zip` - ZIP archive support
- `tar` - TAR archive support
- `gzip` - GZIP compression
- `bzip2` - BZIP2 compression
- `xz` - XZ/LZMA compression
- `sevenz` - 7z archive support
- `gzip` - GZIP compression support (enabled by default)
- `bzip2` - BZIP2 compression support (enabled by default)
- `xz` - XZ/LZMA compression support (enabled by default)

## Capabilities

- **Multiple Formats** - ZIP, TAR, TAR.GZ, TAR.BZ2, TAR.XZ, GZIP, BZIP2, and XZ
- **Async Operations** - Full async/await support with Tokio
- **Flexible Loading** - Load from file paths, memory, or byte streams
- **Type Safety** - Strong typing with `ArchiveType` enum
- **Memory Efficient** - Stream-based processing for large archives
- **Cross-Platform** - Works on Windows, macOS, and Linux

## Dependencies

- `tokio` - Async runtime for I/O operations
- `tar` - TAR archive format support
- `zip` - ZIP archive format support
- `flate2` - GZIP compression
- `bzip2` - BZIP2 compression
- `xz2` - XZ compression
- Async operations with Tokio
- Load from paths, memory, or byte streams
- Stream-based processing for large archives
- Cross-platform (Windows, macOS, Linux)
134 changes: 134 additions & 0 deletions crates/nvisy-archive/src/error.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
//! Archive-specific error types and extensions
//!
//! This module provides error extension traits for creating archive-specific
//! errors and converting errors from underlying archive libraries.

pub use nvisy_rt_core::error::{Error, ErrorResource, ErrorType, Result};

/// Extension trait for creating archive-specific errors
pub trait ArchiveErrorExt {
/// Create an unsupported format error
fn unsupported_format(format: impl Into<String>) -> Error;

/// Create an invalid archive error
fn invalid_archive(message: impl Into<String>) -> Error;

/// Create an entry not found error
fn entry_not_found(name: impl Into<String>) -> Error;

/// Create a permission denied error
fn archive_permission_denied(message: impl Into<String>) -> Error;

/// Create a corrupted archive error
fn corrupted(message: impl Into<String>) -> Error;

/// Create a resource limit error
fn archive_resource_limit(message: impl Into<String>) -> Error;
}

impl ArchiveErrorExt for Error {
fn unsupported_format(format: impl Into<String>) -> Error {
Error::new(format!("Unsupported archive format: {}", format.into()))
.with_type(ErrorType::Runtime)
.with_resource(ErrorResource::Archive)
}

fn invalid_archive(message: impl Into<String>) -> Error {
Error::new(format!("Invalid archive: {}", message.into()))
.with_type(ErrorType::Runtime)
.with_resource(ErrorResource::Archive)
}

fn entry_not_found(name: impl Into<String>) -> Error {
Error::new(format!("Entry not found: {}", name.into()))
.with_type(ErrorType::Runtime)
.with_resource(ErrorResource::Archive)
}

fn archive_permission_denied(message: impl Into<String>) -> Error {
Error::new(format!("Permission denied: {}", message.into()))
.with_type(ErrorType::Runtime)
.with_resource(ErrorResource::Archive)
}

fn corrupted(message: impl Into<String>) -> Error {
Error::new(format!("Corrupted archive: {}", message.into()))
.with_type(ErrorType::Runtime)
.with_resource(ErrorResource::Archive)
}

fn archive_resource_limit(message: impl Into<String>) -> Error {
Error::new(format!("Resource limit exceeded: {}", message.into()))
.with_type(ErrorType::Runtime)
.with_resource(ErrorResource::Archive)
}
}

/// Extension to convert zip::Result to our Result type
#[cfg(feature = "zip")]
pub(crate) trait ZipResultExt<T> {
fn map_zip_err(self) -> Result<T>;
}

#[cfg(feature = "zip")]
impl<T> ZipResultExt<T> for std::result::Result<T, zip::result::ZipError> {
fn map_zip_err(self) -> Result<T> {
self.map_err(|e| {
Error::from_source("ZIP operation failed", e)
.with_type(ErrorType::Runtime)
.with_resource(ErrorResource::Archive)
})
}
}

/// Extension to convert tar I/O errors to our Result type
#[cfg(feature = "tar")]
pub(crate) trait TarResultExt<T> {
fn map_tar_err(self) -> Result<T>;
}

#[cfg(feature = "tar")]
impl<T> TarResultExt<T> for std::result::Result<T, std::io::Error> {
fn map_tar_err(self) -> Result<T> {
self.map_err(|e| {
Error::from_source("TAR operation failed", e)
.with_type(ErrorType::Runtime)
.with_resource(ErrorResource::Archive)
})
}
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn test_error_creation() {
let error = <Error as ArchiveErrorExt>::unsupported_format("custom");
assert_eq!(error.resource, ErrorResource::Archive);

let error = <Error as ArchiveErrorExt>::invalid_archive("test message");
assert_eq!(error.resource, ErrorResource::Archive);

let error = <Error as ArchiveErrorExt>::entry_not_found("missing.txt");
assert_eq!(error.resource, ErrorResource::Archive);

let error = <Error as ArchiveErrorExt>::archive_permission_denied("access denied");
assert_eq!(error.resource, ErrorResource::Archive);

let error = <Error as ArchiveErrorExt>::corrupted("bad data");
assert_eq!(error.resource, ErrorResource::Archive);

let error = <Error as ArchiveErrorExt>::archive_resource_limit("too big");
assert_eq!(error.resource, ErrorResource::Archive);
}

#[test]
fn test_error_display() {
let error = <Error as ArchiveErrorExt>::unsupported_format("test");
assert!(error.to_string().contains("Unsupported archive format"));

let error = <Error as ArchiveErrorExt>::invalid_archive("bad archive");
assert!(error.to_string().contains("Invalid archive"));
}
}
Loading
Loading