SegFormer Quantization Pipeline

An end-to-end evaluation pipeline for SegFormer models on semantic segmentation tasks, with support for various quantization methods.

Status

(DRAFT) (WIP) ----> Not fully implemented yet

For version history have a look at the CHANGELOG.

TOC

• SegFormer Quantization Pipeline
  • Status
  • TOC
  • Features
  • Setup and Usage
    • CLI
    • Docker
  • Configuration
  • Documentation
  • Project Structure
  • Architecture
    • System
    • Code
  • TODO
  • DONE
  • License

Features

• Model loading and quantization (float8, int8, int4, int2)
• Dataset processing and sharding
• Evaluation metrics computation (mean IoU, mean accuracy, overall accuracy)
• Integration with Weights & Biases for experiment tracking

↑

Setup and Usage

CLI

• make setup_env or uv sync --frozen
• make run_app or uv run [--locked] python -m src
• make test_all or uv run pytest

Hint: Set up Weights & Biases API key in environment variables

↑

Docker

docker build -t segformer-quant-eval .
docker run segformer-quant-eval

To build with different python version

docker build --build-arg \
  PYTHON_VERSION=<py_version> \
  .

↑

Configuration

Adjust settings in src/config.py for model, dataset, and evaluation parameters.

Documentation

Documentation SegFormer Quantization Pipeline

↑

Project Structure

/
├── src/
│ ├─ utils/
│ │ ├── data_processing.py
│ │ ├── evaluator.py
│ │ ├── general_utils.py
│ │ ├── model_loader.py
│ │ ├── quantization.py
│ │ └── wandb_utils.py
│ ├── app.py
│ └── config.py
└── pyproject.toml

↑

Architecture

System

Code

↑

TODO

• TDD
  • Implement tests before implementing concrete function
  • test_model_loading, test_image_preprocessing
  • test_quantization, test_predict, test_end_to_end
  • Use pydantic and python typing
• mkdocs
  • Extend workflow to copy only files in nav of mkdocs.yaml
• bump
  • Check steps summary output json, possible bug in GHA
  • Check if version tag exists, abort if so to avoid association of tag with an unrelated commit
• README.md
  • Include badge for tests
  • Insert link to report and project within WandB
• Optional
  • Hugging Face
    • Include option to call HF API instead of saving model locally
      • Might be useful for evaluation purposes
  • Docker
    • Evaluate callisto for fast cloud-native builds

DONE

• Use pt or cuda images to reduce loading time size, e.g.
  • pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime
  • nvidia/12.6.3-base-ubuntu24.04
• mkdocs
  • Add .md to LICENSE/LICENSES to avoid download instead of open
  • Remove/Change #href ↑(#toc) to avoid conflict with gh-pages
  • Remove/Change #href for light/dark png to avoid conflict with
  • Fix mkdocs not indenting checkbox ul
  • Fix mkdocs not including png with plain in-line html, assets/ not copied by mkdocsgh-pages
• bump
  • Fix bump-my-version.yaml rollback step to delete auto-created branch after failure()
    • Handle error fatal: could not read Username & Error: Process completed with exit code 128.
• CHANGELOG.md auto-generate
  • Solution: do manual bump occasionally and populate CHANGELOG.md before
  • Conventional Commits .gitmessage
  • Tools like git-changelog
• Docker
  • Where are site-packages in Dockerfile for copy to runtime located?
    • Solution: .venv
• Branch protection rules
  • Push to main with PR only
  • Use dedicated branch dev-auto-push-to-main
  • Incorporate branch to workflow bump-my-version.yaml
  • Create workflow update_changelog.yaml

↑

License

This project is licensed under the BSD 3-Clause License. See the LICENSE file for details Third-party licenses might also apply.

↑