one archive per file?

[ThomasWaldmann]

"Per-File Backup Mode - NEW! Backup each file as a separate archive with metadata-based retention"

That sounds a bit problematic, at least if many files are processed (= many archives are created).

borg is only designed to deal with "normal" amounts of archives (like 10, 100, 1000 or so), but will run into issues if you create excessive amounts, like >> 100.000 (precise limit depends).

So maybe include a hint that this ruborg feature is only intended for special use cases and a few files only?

[mpantel]

Thank you for raising this important concern, @ThomasWaldmann. You're absolutely right about the scalability limitations.

Context and Use Case

The "one archive per file" mode is intended for a specific scenario: backing up a manageable number of large backup files where retention policies need to be based on the file's creation date rather than the archive creation date.

For example, if you have dozens or a few hundred large database dumps or backup files, each with its own lifecycle, this approach allows:

• Clear 1:1 mapping between file and archive
• Encoding the original file creation date in archive metadata/names
• Applying retention policies based on file age, not when it was archived

Acknowledging the Limitations

This feature should definitely include prominent documentation warnings:

❌ NOT suitable for:

• Directories with many files (>>1,000)
• Regular file backups where archive count could exceed ~10,000-100,000
• Use cases where standard Borg workflows would suffice

✅ Suitable for:

• Small numbers (dozens to low hundreds) of large, independent backup files
• Files requiring individual retention policies based on original creation dates
• Scenarios where Borg's archive-based retention doesn't map to file metadata

Suggested Improvements

1. Add clear documentation explaining:

   • The intended use case (few large files, not many small files)
   • Explicit archive count limits (~1,000 maximum recommended)
   • Warning about performance degradation with excessive archives

2. Consider safeguards:

   • Configurable limit on number of files to process
   • Warning when approaching dangerous archive counts
   • Dry-run mode showing how many archives would be created

3. Document the workaround: Since Borg archives are immutable and you can't selectively delete files from them, the one-archive-per-file approach is actually cleaner for this use case than trying to maintain large archives where individual files need different retention periods.

Alternative Considered

The alternative would be custom retention scripts that parse file metadata from regular grouped archives, but this adds complexity when the simple 1:1 mapping works for the intended scale.

I'll update the README to clearly document these limitations and intended use cases. Does this adequately address your concern while keeping the feature available for its specific scenario?