Skip to content

Display ConfusionMatrix in StdOut (introduces different meaning of rows and cols to some implementations)#348

Merged
klemen1999 merged 5 commits intomainfrom
feat/display_conf_mat
Feb 23, 2026
Merged

Display ConfusionMatrix in StdOut (introduces different meaning of rows and cols to some implementations)#348
klemen1999 merged 5 commits intomainfrom
feat/display_conf_mat

Conversation

@klemen1999
Copy link
Collaborator

Purpose

Until now the confusion matrices were only pushed to the tracker and never displayed in the terminal (stdout). This PR changes this by correctly also showing the confusion matrix under the correct node in which it was used.
Additionally in 8f194d0 it also introduces a breaking change where it unifiys what row and column mean in the matrix for all tasks. Until now the torchmetric and our implementation differed here which can be quite confusing.

Specification

None / not applicable
Multiple matrices under same head (with instance segmentation task)
Screenshot from 2026-02-19 22-26-04

Multiple matrices under different heads
Screenshot from 2026-02-19 22-26-49

Implementation when rich_logging: False
Screenshot from 2026-02-19 22-27-44

Dependencies & Potential Impact

None / not applicable
This PR changes what rows and columns mean in order to unify all to Torchmetrics notation. With improved visualization and notes in the README I deem this a worthy change because the alternative of keeping it different just introduces confusion without any reason.

Deployment Plan

None / not applicable

Testing & Validation

None / not applicable

@klemen1999 klemen1999 requested a review from dtronmans February 19, 2026 21:36
@klemen1999 klemen1999 requested a review from a team as a code owner February 19, 2026 21:36
@klemen1999 klemen1999 requested review from conorsim, kozlov721 and tersekmatija and removed request for a team February 19, 2026 21:36
@github-actions github-actions bot added documentation Improvements or additions to documentation enhancement New feature or request labels Feb 19, 2026
Copy link
Contributor

@dtronmans dtronmans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested with instance segmentation and it works correctly. For instance segmentation we have two sub-metrics, one for detection and one for segmentation, so maybe here in the artifact keys we can just change it to {sub_name}.json instead of confusion_matrix.json? So that they are logged separately as detection_confusion_matrix.json and segmentation_confusion_matrix.json instead of both mapping to confusion_matrix.json

I verified with this script:

config= "configs/crack_instance_segmentation_constantlr.yaml"

model = LuxonisModel(
    config,
    {
        "tracker.is_mlflow": True,
        "tracker.project_name": "cm_test",
        "tracker.run_name": "cm_test_run",
    },
)

artifact_keys = model.get_mlflow_logging_keys()["artifacts"]
cm_keys = [k for k in artifact_keys if "confusion_matrix" in k]

for key in cm_keys:
    print(key)

With the proposed change I have correctly separate detection and segmentation confusion matrices:

test/metrics/300/PrecisionSegmentBBoxHead/detection_confusion_matrix.json
test/metrics/300/PrecisionSegmentBBoxHead/segmentation_confusion_matrix.json
val/metrics/0/PrecisionSegmentBBoxHead/detection_confusion_matrix.json
val/metrics/0/PrecisionSegmentBBoxHead/segmentation_confusion_matrix.json

Whereas keeping it as it is, both map to the same filename:

val/metrics/0/PrecisionSegmentBBoxHead/confusion_matrix.json
val/metrics/109/PrecisionSegmentBBoxHead/confusion_matrix.json
val/metrics/119/PrecisionSegmentBBoxHead/confusion_matrix.json

@klemen1999
Copy link
Collaborator Author

Nice find, addressed in 578aaff

Copy link
Contributor

@dtronmans dtronmans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks

@klemen1999 klemen1999 merged commit 98d5b1d into main Feb 23, 2026
11 checks passed
@klemen1999 klemen1999 deleted the feat/display_conf_mat branch February 23, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants