Evaluation results different from the results from the paper #44
Description
I ran the evaluation script for the provided checkpoint and found the results a bit different from the paper-reported results.
Especially, the FID and R-precision are higher, but the MultiModality and Multimodal Distance are significantly lower compared to the reported results. What could the reason for this discrepancy be?
The evaluation results from the checkpoint:
========== MM Distance Summary ==========
---> [ground truth] Mean: 3.7845 CInterval: 0.0008
---> [InterGen] Mean: 3.7990 CInterval: 0.0022
========== R_precision Summary ==========
---> [ground truth](top 1) Mean: 0.4249 CInt: 0.0048;(top 2) Mean: 0.6018 CInt: 0.0061;(top 3) Mean: 0.7042 CInt: 0.0049;
---> [InterGen](top 1) Mean: 0.4316 CInt: 0.0079;(top 2) Mean: 0.5864 CInt: 0.0088;(top 3) Mean: 0.6757 CInt: 0.0075;
========== FID Summary ==========
---> [ground truth] Mean: 0.2949 CInterval: 0.0101
---> [InterGen] Mean: 6.4828 CInterval: 0.1484
========== Diversity Summary ==========
---> [ground truth] Mean: 7.7497 CInterval: 0.0296
---> [InterGen] Mean: 7.8578 CInterval: 0.0538
========== MultiModality Summary ==========
---> [InterGen] Mean: 1.2164 CInterval: 0.0343