ENSEMBLE - Sheet Music Converter Backend

A production-ready backend for sheet music conversion using Audiveris and music21 with a well-documented API for frontend integration.

Project Structure

sheet-music-backend/
├── app/
│   ├── main.py (FastAPI entrypoint)
│   ├── processor.py (core logic)
│   └── schemas.py (Pydantic models)
├── scripts/
│   └── setup_audiveris.sh
├── docker/
│   └── Dockerfile
├── requirements.txt
└── README.md (API documentation)

API Documentation

Endpoints

1. POST /convert

   • Description: Convert uploaded sheet music
   • Request:
     • file: Sheet music image (PNG/JPG)
   • Response:

     {
       "process_id": "uuid-string",
       "formats": {
         "midi": "/results/uuid-string/output.mid",
         "annotated_score": "/results/uuid-string/annotated.png",
         "musicxml": "/results/uuid-string/somefile.mxl"
       }
     }

2. GET /results/{process_id}/{filename}

   • Description: Download processed files
   • Parameters:
     • process_id: From conversion response
     • filename: One of: output.mid, annotated.png, or original XML filename
   • Response: File content with proper Content-Type

How It Works

Processing Workflow

sequenceDiagram
    Frontend->>Backend: POST /convert (image)
    Backend->>Audiveris: Process image
    Audiveris->>Backend: MusicXML
    Backend->>music21: Parse and annotate
    Backend->>music21: Generate MIDI
    Backend->>music21: Generate annotated PNG
    Backend->>Frontend: JSON with result URLs
    Frontend->>Backend: GET result files

Loading

Key Components

• Audiveris: Java-based OMR engine converts sheet images to MusicXML
• music21: Python library for music analysis and generation
• Conversion Steps:
  1. Image → MusicXML (via Audiveris)
  2. MusicXML → Annotated music21 Score
  3. Score → MIDI (playable audio)
  4. Score → Annotated PNG (visual sheet music)

Output Formats

• midi: Playable audio file (standard MIDI format)
• annotated_score: Sheet music image with note names below each note
• musicxml: Structured representation of the musical score

Frontend Integration Guide

Conversion Flow

// Example frontend code (pseudo-code)
async function convertSheetMusic(file) {
  // Step 1: Upload image
  const formData = new FormData();
  formData.append('file', file);
  
  const conversionRes = await fetch('/convert', {
    method: 'POST',
    body: formData
  });
  
  const { process_id, formats } = await conversionRes.json();
  
  // Step 2: Get results
  const midiUrl = formats.midi;
  const annotatedImageUrl = formats.annotated_score;
  
  return {
    midi: await fetch(midiUrl).then(r => r.blob()),
    annotatedImage: await fetch(annotatedImageUrl).then(r => r.blob())
  };
}

Handling Results

• MIDI: Play using Web Audio API or libraries like Tone.js
• Annotated Image: Display directly in tag
• MusicXML: Use for advanced music analysis if needed

Error Handling

• 400-series errors: Invalid requests
• 500-series errors: Processing failures
• Standard HTTP status codes used throughout

Deployment Instructions

Build Docker Image

docker build -f docker/Dockerfile -t sheet-music-backend .

Run Container

docker run -p 8000:8000 sheet-music-backend

Test API

curl -X POST "http://localhost:8000/convert" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@path/to/sheet-music.png"

Key Advantages for Frontend Engineers

1. Simple REST API: Only two endpoints to implement
2. Stateless Design: Process ID based result retrieval
3. Standard Formats: MIDI and PNG for easy integration
4. Well-Documented Errors: Clear error responses
5. CORS Enabled: Ready for web frontend integration
6. Production-Ready: Containerized with proper timeouts

Development

Running Locally

1. Install dependencies:

   pip install -r requirements.txt

2. Run the application:

   uvicorn app.main:app --reload

API Documentation

Once running, visit http://localhost:8000/docs for interactive API documentation.

License

This project is part of the ENSEMBLE project.