Skip to content

Reproducing the paper results #6

@LSunQQ

Description

@LSunQQ

Thank you for your exciting work and very clean code. I am having trouble reproducing the results mentioned in the paper and would appreciate it if you could help me.

  1. Reproducing the results of cifar100
    I was trying to get the scores on the samples from Table 2 in the paper. However, the results on all datasets did not match the ones that I see in the paper. I ran your code twice with "python3 train.py --dataset cifar100 --lbl-percent 10 --novel-percent 50 --arch resnet18". The result on seen,novel, all class is 1-2% lower than that in your paper.
    logcifar100_label10_1.txt
    logcifar100_label10_2.txt

  2. With 50% label,The performance on the CIFAR-100 dataset with 50% label,the paper suggested adjusting the temperature to 0.2. However,the results on novel class is 41% while 49% in paper.
    logcifar100_label50_t0.2.txt
    with temperature set to 0.1, the result on novel class is 1% lower than that in your paper .
    logcifar100_label50_t0.1.txt

  3. I noticed that the CosineAnnealingLR with warm-up mentioned in your paper has been removed.

Thanks a lot for your time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions