Problems that are inconsistent with the results of the paper

[zhcz328]

I got the results as below, without rectifying any parameters. if there are any settings that still need to be adjusted?

The PLCC: 0.7150729626772474, SRCC: 0.7165368518360726 of using the soft preference matrix of the /archive/playground/results/ on live_test.json
The PLCC: 0.6347242170678974, SRCC: 0.4442877007623038 of using the soft preference matrix of the /archive/playground/results/ on clive_test.json
The PLCC: 0.6616077442996833, SRCC: 0.3760881101573836 of using the soft preference matrix of the /archive/playground/results/ on csiq_test.json
The PLCC: 0.5618430532738221, SRCC: 0.4790654577761054 of using the soft preference matrix of the /archive/playground/results/ on koniq10k_test.json
The PLCC: 0.7207018992298927, SRCC: 0.681105666202101 of using the soft preference matrix of the /archive/playground/results/ on bid_test.json
The PLCC: 0.6275966268787452, SRCC: 0.6258121069257544 of using the soft preference matrix of the /archive/playground/results/ on kadid10k_test.json
The PLCC: 0.7413863632825104, SRCC: 0.7036643082663387 of using the soft preference matrix of the /archive/playground/results/ on spaq_testing_set.json
The PLCC: 0.6561597978933105, SRCC: 0.5958065731148577 of using the soft preference matrix of the /archive/playground/results/ on tid_testing_set.json
The PLCC: 0.6204841579165193, SRCC: 0.5848747017216147 of using the soft preference matrix of the /archive/playground/results/ on agi.json

[h4nwei]

Hi @zhcz328, thank you for your interest in our work. I reproduced the results on another server using the code provided in this repository. Could you provide more details so I can assist you more effectively?

[zhcz328]

The other parameter configuration remains the same, is that the reason for changing the option to LoRA as below?

[h4nwei]

@zhcz328 Perhaps, instead of using LoRa to train the model, we fine-tune all the parameters.