How to use DPT in DETR

[lingblessing]

Does the author add the DPT module to the DETR part? A little anxious, thank you very much

[volgachen]

We simply replace the PVT backbone with our DPT model, without any other modification in DETR decoder & encoder layers.

Please see the configuration here
https://github.com/CASIA-IVA-Lab/DPT/blob/main/detection/configs/detr_dpt_s_8x2_50ep_coco.py

[lingblessing]

It is to replace the ResNet50 in DETR with PVT, and the Transformer is still connected later, right?
And DPT replaces a module in PVT, right?

[volgachen]

Yeah.
For how to replace ResNet50 with PVT, please refer to the paper of PVT.

[lingblessing]

Thank you very much for your patient answer. In DETR, the backbone output dimension is 256 and the number of channels is 2048. How do you set the dimension and number of channels in DPT?

[volgachen]

The dimension of backbone output is 512, while the transformer dimension is set to be 256.
There is a layer to handle the dimension transformation.

For our configuration please refer detr_r50_8x2_50ep_coco_baseline.py

For detailed implementation please refer mmdet/models/dense_heads/transformer_head.py in mmdetection (v2.8.0).