Calculation of OOD within the paper

[dalbertweiss]

Hello together,

first of all I would like to thanks for making the paper and code to "WILDS: A Benchmark of in-the-Wild Distribution Shifts" publicly available. What caught my interest when reading the paper was the estimation of the in (IID) and out-of-distribution (OOD) which has been evaluated using empirical risk minimization (Table 1 page 20). My question is how the IIDs and OODs were calculated. Did you use the softmax with temperature scaling according to the paper "Enhancing the reliability of out of distribution image detection in neural networks"? If not, can you give reference to the way you tackled this problem?

Thank you in advance for your kindful reply

[kohpangwei]

No, we didn't use temperature scaling, and we're also not doing OOD detection. We trained the models listed in Section 5.3 of the paper (also see Appendix D for details), and then evaluated them on the ID and OOD test sets. Does that make sense?