Skip to content

Comments

E2E FPS benchmark tests#203

Open
dtronmans wants to merge 4 commits intomainfrom
feat/e2e-fps-benchmark-tests
Open

E2E FPS benchmark tests#203
dtronmans wants to merge 4 commits intomainfrom
feat/e2e-fps-benchmark-tests

Conversation

@dtronmans
Copy link
Contributor

@dtronmans dtronmans commented Feb 18, 2026

Purpose

  • Any model can be added to the benchmark_targets.json it will be automatically picked up by the test suite
  • Tested on RVC4 device, just need to run this: pytest tests/test_benchmark/ --device-ip <ip>
  • Had to expand the threshold to 10 percent lower, specifically the tests/test_benchmark/test_benchmark_regression.py::test_benchmark_fps[yolov6-nano:r2-coco-512x288] was showing 6 percent lower sometimes but that is probably because the base fps (841.98) is quite a large number
  • For now this doesn't run with CI, even though it's in the /tests directory, unittests.yaml only runs tests/test_utils

Specification

Dependencies & Potential Impact

Deployment Plan

Testing & Validation

@dtronmans dtronmans requested a review from a team as a code owner February 18, 2026 14:00
@dtronmans dtronmans requested review from conorsim, klemen1999, kozlov721 and tersekmatija and removed request for a team February 18, 2026 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant