[ITEP-84873] Example pipelines for reid with additional attributes in metadata

[jakubsikorski]

📝 Description

Adds a document that includes examplepipelines for ReID with metadata (both raw and scenescape) showing the attributes each pipeline generates.

Contains code changes required to preserve the metadata along with ReID entry.

ITEP-84873

✨ Type of Change

Select the type of change your PR introduces:

• 🐞 Bug fix – Non-breaking change which fixes an issue
• 🚀 New feature – Non-breaking change which adds functionality
• 🔨 Refactor – Non-breaking change which refactors the code base
• 💥 Breaking change – Changes that break existing functionality
• 📚 Documentation update
• 🔒 Security update
• 🧪 Tests
• 🚂 CI

🧪 Testing Scenarios

Describe how the changes were tested and how reviewers can test them too:

• ✅ Tested manually
• 🤖 Ran automated end-to-end tests

✅ Checklist

Before submitting the PR, ensure the following:

• 🔍 PR title is clear and descriptive
• 📝 For internal contributors: If applicable, include the JIRA ticket number (e.g., ITEP-123456) in the PR title. Do not include full URLs
• 💬 I have commented my code, especially in hard-to-understand areas
• 📄 I have made corresponding changes to the documentation
• ✅ I have added tests that prove my fix is effective or my feature works

[saratpoluri]

Can you please add the DLS output here.

[jakubsikorski]

Added links to JSON files, they are too large to view in Markdown

[saratpoluri]

Only reid is coming through in the object. We need to have the reid and whatever metadata is coming through like age-gender and person attributes. Looking at this, the data seems to be getting discarded. My first suspect is the sscape_adapter. Could you please fix this?

I will work under the assumption that the age-gender, person attributes etc. will be right alonside reid entry in the output json like it is when run without reid.

[jakubsikorski]

It was only reid coming through due to the policy. In the initial commit I have fixed reid to work with model chaining at all (the reid_idx in the expected metadata was hardcoded as 1st index, which is not a given).

I have now fixed that the metadata isn't lost using reid policy, attributes like age and gender are now alongside the reid entry in the output.

[saratpoluri]

Could we have one where we do the following chain:
retail->age-gender->person attributes->reid

and make sure all of them make it through in the scenescape object json.

[saratpoluri]

This is not urgent but needed for next iteration in 2026.1:
We only pass on the confidence of the first object detection model. We currently discard the confidence of all other models. We need the confidence of all other models pass through as well into the sscape_object.

[jakubsikorski]

Could we have one where we do the following chain:
retail->age-gender->person attributes->reid

Added a pipeline that does all of them, with example output, the metadata contains:

• age
• gender
• gender: clothing
• reid entry

We need the confidence of all other models pass through as well into the sscape_object

I have updated the classificationPolicy of the adapter to preserve the confidence of all models, it will be available in the metadata as <modelName>_model_confidence

[saratpoluri]

A flattened structure of metadata will create association issues. Unless I am aware downstream of what model was used, I have no way to connect the different outputs with the confidence info.

Let's discuss how to represent confidence for each metadata in a way that enables easy association and processing.

[jakubsikorski]

The metadata is no longer flattened - each entry now includes its label, confidence (when available), and model name in a structured format.

In this format the association is clear as each metadata entry is self contained.
As for the model names - unfortunately this is all I see in the raw output from DLSPS, perhaps this is something we could request from the DLS(PS) team - to provide more descriptive model names in the metadata.

I have already implemented the changes to the code in this commit 53b0a31.

An example structure will look like this:

{
    "id": "atag-qcam1",
    "debug_mac": "41:65:97:c3:5a:e0",
    "timestamp": "2026-02-10T12:09:19.470Z",
    "debug_timestamp_end": "2026-02-10T12:09:21.931Z",
    "debug_processing_time": 2.460519552230835,
    "rate": 6.604081651768196,
    "objects": {
        "person": [
            {
                "category": "person",
                "confidence": 0.997582197189331,
                "center_of_mass": {
                    "x": 527,
                    "y": 125,
                    "width": 67.0,
                    "height": 106.75
                },
                "bounding_box_px": {
                    "x": 460,
                    "y": 20,
                    "width": 201,
                    "height": 427
                },
                "metadata": {
                    "age": {
                        "label": "35",
                        "model": "age_gender"
                    },
                    "gender": {
                        "label": "Male",
                        "confidence": 0.9521394968032837,
                        "model": "age_gender"
                    },
                    "person-attributes": {
                        "label": "M: has_longpants",
                        "confidence": 0.9166854619979858,
                        "model": "torch-jit-export"
                    },
                    "reid": {
                        "embedding": "VpGuPujDML4O5oa/UxDLvbXp1z7Cn06+Bg8/PsN7Fr6yQSS7Kzujvnvwwb0uRTs+Bc3pPYzbWT8RRxY/3+2XvhARqj5shhA/W9MUv71bcT19jYc+2/GBPkWnrL4aeoG9rN7svo5cuL0WfyY+EuLtvdEqCr8tdOY+R7XWvq4COD+cOTW/h0xdP96UcD/PmtC+BeIwvyDcR758tEc/3wqGvo71Dr34vZO+UlCePZHFWj4UO3G+/2unvoiuAz+uIhC/9D3yvGx8G7/JDvS8hx6aO4oRUrwZRXC+joaePnBdVr5gido9ofFxPpkqmD+7po0+F3+HvhWBLT64bJk+7lP3PujXFr6jaBc+IeQkP1JS3L78op0+wcg5P7Py4D6W4VK+6fXmvTAIjD70bEG/SKLavo2crz6CB6U9CJyuviDJXL52hw4/hao/vqY3AT4mYoS+YUT6PhPFIj97Q40+MJQ+PrmwBT+aioI+FBPuPlo4ur7Tr8s+oXI4vJ9BEz8NKzY96AHNvskO+DvzAcy+hv+RPh2/MT+HvhU/17XwPTYUF72MmYy+o2vzPk7NjD0eq8G/qiYOv/isZz42ML0+5HApvNnX0T3ElPm9FG+HPfl+Db/r658/fWMNv7C3hL4iSSa+Cr5NPshTHb8GWYq+Zsk7v1odor7rxKU+Z8lfv3S4xb5B0No+iNQVvrttmT7knNG+lv8bP9KA0j67fe6+SXDfPvaUpzwlP8g+GyrzPjudgby2McW91yJUPyMcND/lXgq/egFPP6h1JL8IMVE+AqOjPgvFz76q9ru+bYCnP8q8Yb2326s8eD8UP1ey27xY34i+Bdoxvvljrj0oKSa/h4HzvrbUhL0+AQy/+s3RvXJcEr/+/Tk/DEMtPxwkaD9NJy6+BzupPtwh0L5Yu1s9+1pRviCU377/CCE+FvhhPTVSGj2TqVo99bt5P8AlCL+tFSu/Y1h8vGnDKD/Cdpa9aMOhvvoMBr920J4+OF+Nvhw3IT+b6YI/Ve6vvrK6Nb/oYvG+ll9rPqUCSb+2kwW/7TEYPv/dur4DcPC+ttDbPrw+VD+cZ+O94bGavybpwb6VAii+GSkGPyFhzD3xAc89zlVwv6aGRj1kIiG/yl0mPr4/5LsGMzO+cpzRviFWQT/VBqa9q0Ycvzbwg78+Ois/NTprPhmUBz4OrXg/wz9FP8fThb4CnaA+T2EZv4JlpT6PtQQ/pY49P1UYQr8qThK+z3cpP/2Q5D02ga88HI+AP2T8lD46GO89eExoP8ED4T5wtDk+mt6Hvdwnir7q1aC+8BBuPA5rXr4lkPK+WhZZvYTQFz4uVrW+BZE5PHm8Zr53LhU+ahUCvg7N+j7aC66+9k3xvg==",
                        "model": "torch-jit-export"
                    }
                },
                "id": 1
            }
        ]
    }
}

CC: @tdorauintc

[tdorauintc]

I like the new structure, it's self descriptive and easy to parse.

Once we agree on it, we'll need to update controller/src/schema/metadata.schema.json.

[tdorauintc]

Since OMZ models are deprecated, the DLS team will propose a new set of public models soon (this is work in progress), for ReID too. Let's hope DLS metadata format does not depend on specific models.

Long term it would be best to establish a contract or schema guaranteed for DLS metadata format, otherwise our adapter might break in future if DLS implementation changes it for some reason.

[tdorauintc]

There are still some raw data inline that could be kept in a separate file for readability

[tdorauintc]

I like the new structure, it's self descriptive and easy to parse.

Once we agree on it, we'll need to update controller/src/schema/metadata.schema.json.