Reasoning is slow and ChartQA doesn't run well

[warm345]

Hello, I would like to ask you, I run your model and code here, using single_turn_eval.py for ChartQA inference

The speed is shown below
72%|███████▏ | 1798/2500 [5:46:50<2:10:59, 11.20s/it]
72%|███████▏ | 1799/2500 [5:47:01<2:10:40, 11.19s/it]
72%|███████▏ | 1800/2500 [5:47:13<2:11:10, 11.24s/it]
72%|███████▏ | 1801/2500 [5:47:24<2:11:23, 11.28s/it]
72%|███████▏ | 1802/2500 [5:47:35<2:10:51, 11.25s/it]
72%|███████▏ | 1803/2500 [5:47:46<2:10:18, 11.22s/it]
72%|███████▏ | 1804/2500 [5:47:58<2:10:14, 11.23s/it]

Also, ChartQA ran out as a result
aug: 55.84
human: 41.84
Total: 48.84

Did I have a problem running at that step

[FanqingM]

The model have more than one vision encoder and this is why the speed is not fast.
For ChartQA, you can use this json
https://github.com/OpenGVLab/ChartAst/blob/main/chartqa_all.json

[FanqingM]

accessory/exps/finetune/mm/test.sh
The param please use above

[FanqingM]

For chartQA, we use COT-Answer and normal answer.
You can refer the https://github.com/OpenGVLab/ChartAst/blob/main/chartqa_all.json where we release the evaluation result.
It maybe some different with the original paper because we update the model with some stock data.