About of visual results.

[jjlinghu]

Hi! Thanks for your excellent work! I followed your steps to re-train the GassianLSS, and I got the anticipative metrics in results.
However, I found some misalignments in wandb/xxx/files/media/images/train when I checked the visualization. Surprisingly, the log images in the val folder are correct. Could you tell me what I did in the code?

train image (image_41_47da823e5f76e64e190c.png):

There is a black car that can be seen in the back camera clearly, but we can't get the corresponding GT box in BEV.

val image (It seems no wrong happened):

Thanks for your reply!

[shu-wei-lu]

@jjlinghu since we use data augmentation on 3D while training, appling rotation and translation on extrinsics, the corresponding BEV segmentation would not be aligned to cameras relative positions.

GaussianLSS/GaussianLSS/data/transforms.py

Line 371 in cd6d305

bev_augm = self.augment_bev() if self.augment_bev else np.eye(4)

[jjlinghu]

Thanks for your professional reply!

[jjlinghu]

@shu-wei-lu Hello, I have some new confusing questions. I hope I can get help from you!

In GaussianLSS, what is the actual purpose of the learnable opacity? It seems to be used for learning the ordering between different Gaussians, such as depths[idx] = opacities[idx]

GaussianLSS/GaussianLSS/model/diff-gaussian-rasterization/cuda_rasterizer/forward.cu

Line 154 in cd6d305

depths[idx] = opacities[idx];

However, when actual ordering is required, a method with z=0 is adopted to blend the Gaussians to obtain BEV features forcibly. May I ask what the rationale is behind this design? Isn't it necessary to sort them based on the height (z) of the 3D points relative to the ground? or just the poor performance.

Additionally, is there a deeper meaning behind having a larger or smaller number of Gaussians? I sincerely look forward to your reply.

All the best to you!