diff --git a/README.md b/README.md index aef3685..256ce29 100644 --- a/README.md +++ b/README.md @@ -9,11 +9,11 @@ Результаты тестирования моделей: - google_gemini_2_0_flash_001: 86% - openai_o3_mini_high: 85% -- google_gemini_2_5_flash_preview_05_20: 84% - deepseek_deepseek_chat_v3_0324: 82% - google_gemini_2_5_flash_preview: 82% - deepseek_deepseek_chat_free: 78% - google_gemini_2_0_flash_lite_001: 78% +- google_gemini_2_5_flash_lite_preview_06_17: 78% - google_gemma_3_27b_it: 77% - openai_gpt_4o_mini: 77% - deepseek_deepseek_chat: 76% @@ -110,6 +110,11 @@ OLLAMA_BASE_URL=http://localhost:11434 Обработка задач завершена ``` +Для проверки формата результата (не попадает ли в код посторонние теги) можно решить только одну задачу: +```shell +./bin/human_eval_solver solve tasks --model google_gemini-2.5-flash-lite-preview-06-17 --task 1 +``` + #### Запускаем решения Все: diff --git a/config/models.yml b/config/models.yml index ef05857..58ed181 100644 --- a/config/models.yml +++ b/config/models.yml @@ -122,6 +122,12 @@ openrouter: note: https://openrouter.ai/google/gemini-2.5-flash-preview-05-20 done: false + google_gemini-2.5-flash-lite-preview-06-17: + name: google/gemini-2.5-flash-lite-preview-06-17 + provider: openrouter.ai + note: https://openrouter.ai/google/gemini-2.5-flash-lite-preview-06-17 + done: false + # Ollama модели ollama: ollama_llama3_2: diff --git a/reports/human_eval_for_ruby_report_full.html b/reports/human_eval_for_ruby_report_full.html index 5941895..37fa938 100644 --- a/reports/human_eval_for_ruby_report_full.html +++ b/reports/human_eval_for_ruby_report_full.html @@ -64,18 +64,18 @@

Отчет о тестировании моделей

-

Дата: 2025-05-31 17:02:16

+

Дата: 2025-07-05 16:11:33

Результаты по моделям

- + @@ -99,8 +99,8 @@

Детальные результаты по задачам

МодельУспешность
google_gemini_2_0_flash_00187%
openai_o3_mini_high85%
google_gemini_2_5_flash_preview_05_2084%
deepseek_deepseek_chat_v3_032482%
google_gemini_2_5_flash_preview82%
deepseek_deepseek_chat_free79%
google_gemini_2_0_flash_lite_00179%
google_gemini_2_5_flash_lite_preview_06_1778%
deepseek_deepseek_chat77%
google_gemma_3_27b_it77%
openai_gpt_4o_mini77%
+ - @@ -119,7 +119,7 @@

Детальные результаты по задачам

- + @@ -319,7 +319,7 @@

Детальные результаты по задачам

- + @@ -394,7 +394,7 @@

Детальные результаты по задачам

- + @@ -445,7 +445,7 @@

Детальные результаты по задачам

- + @@ -495,7 +495,7 @@

Детальные результаты по задачам

- + @@ -624,7 +624,7 @@

Детальные результаты по задачам

- + @@ -648,7 +648,7 @@

Детальные результаты по задачам

- + @@ -845,8 +845,8 @@

Детальные результаты по задачам

- + @@ -1063,7 +1063,7 @@

Детальные результаты по задачам

- + @@ -1087,7 +1087,7 @@

Детальные результаты по задачам

- + @@ -1280,7 +1280,7 @@

Детальные результаты по задачам

- + @@ -1329,7 +1329,7 @@

Детальные результаты по задачам

- + @@ -1641,9 +1641,9 @@

Детальные результаты по задачам

- + @@ -1738,7 +1738,7 @@

Детальные результаты по задачам

- + @@ -1761,7 +1761,7 @@

Детальные результаты по задачам

- + @@ -1809,7 +1809,7 @@

Детальные результаты по задачам

- + @@ -1980,7 +1980,7 @@

Детальные результаты по задачам

- + @@ -2126,7 +2126,7 @@

Детальные результаты по задачам

- + @@ -2174,7 +2174,7 @@

Детальные результаты по задачам

- + @@ -2342,8 +2342,8 @@

Детальные результаты по задачам

- + @@ -2780,7 +2780,7 @@

Детальные результаты по задачам

- + @@ -2924,9 +2924,9 @@

Детальные результаты по задачам

- + @@ -3092,8 +3092,8 @@

Детальные результаты по задачам

- + @@ -3188,7 +3188,7 @@

Детальные результаты по задачам

- + @@ -3212,7 +3212,7 @@

Детальные результаты по задачам

- + @@ -3478,7 +3478,7 @@

Детальные результаты по задачам

- + @@ -3938,7 +3938,7 @@

Детальные результаты по задачам

- + @@ -4009,7 +4009,7 @@

Детальные результаты по задачам

- + diff --git a/reports/human_eval_for_ruby_report_total.html b/reports/human_eval_for_ruby_report_total.html index 4da14af..a2ac50f 100644 --- a/reports/human_eval_for_ruby_report_total.html +++ b/reports/human_eval_for_ruby_report_total.html @@ -64,17 +64,17 @@

Суммарный отчет о тестировании моделей

-

Дата: 2025-05-31 17:02:16

+

Дата: 2025-07-05 16:11:33

Задача deepseek_deepseek_chat_v3_0324google_gemini_2_5_flash_lite_preview_06_17 google_gemini_2_5_flash_previewgoogle_gemini_2_5_flash_preview_05_20 google_gemini_flash_1_5 google_gemma_3_27b_it inception_mercury_coder_small_beta
t0 ✓✗✓ ✗ ✓ ✓ ✓ ✓ ✗✓✗ ✓ ✗ ✓ ✓ ✗ ✓✓✗ ✓ ✗ ✗ ✓ ✓ ✓✓✗ ✓ ✓ ✓ ✓ ✓ ✗✓✗ ✓ ✗ ✓ ✗ ✓ ✓✓✗ ✓ ✓ ✗ ✓ ✓ ✓✓✗ ✓ ✓ ✓ ✓ ✓ ✗✗ ✓✗ ✓ ✗ ✗ ✗ ✓ ✗✗✓ ✗ ✓ ✓ ✓ ✓ ✓✓✗ ✓ ✓ ✓ ✓ ✓ ✗✓✗ ✓ ✗ ✓ ✗ ✓ ✓✗✓ ✗ ✓ ✓ ✗ ✓ ✗✓ ✗ ✓✓ ✗ ✗ ✗ ✓ ✓ ✗✓✗ ✓ ✓ ✗ ✗ ✓ ✗✗✓ ✗ ✓ ✓ ✓ ✓ ✓✗✓ ✗ ✓ ✓ ✓ ✓ ✓✓✗ ✓ ✓ ✓ ✓ ✓ ✓✗✓ ✓ ✓ ✓ ✓ ✓ ✓✓✗ ✓ ✓ ✓ ✓ ✓ ✓✓ ✗✓ ✗ ✓ ✗ ✓ ✓ ✓✓✗ ✓ ✓ ✓ ✓ ✗ ✓✗ ✓ ✗✗ ✓ ✓ ✓ ✓ ✓ ✓✗ ✓✗ ✓ ✗ ✓ ✓ ✓ ✓✗✓ ✗ ✗ ✓ ✓ ✓ ✓✓✗ ✓ ✓ ✓ ✓ ✓ ✗✓✗ ✗ ✗ ✗ ✗ ✓ ✗✓✗ ✓ ✗ ✓ ✓ ✓ ✗✓✗ ✓ ✓ ✓
- + diff --git a/reports/test_results.json b/reports/test_results.json index 8c6cf2b..4247974 100644 --- a/reports/test_results.json +++ b/reports/test_results.json @@ -1,5 +1,5 @@ { - "timestamp": "2025-05-31 17:02:16", + "timestamp": "2025-07-05 16:11:33", "models": [ [ "google_gemini_2_0_flash_001", @@ -9,10 +9,6 @@ "openai_o3_mini_high", 85 ], - [ - "google_gemini_2_5_flash_preview_05_20", - 84 - ], [ "deepseek_deepseek_chat_v3_0324", 82 @@ -29,6 +25,10 @@ "google_gemini_2_0_flash_lite_001", 79 ], + [ + "google_gemini_2_5_flash_lite_preview_06_17", + 78 + ], [ "deepseek_deepseek_chat", 77 @@ -101,8 +101,8 @@ "tasks": { "t0": { "deepseek_deepseek_chat_v3_0324": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -125,8 +125,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -151,8 +151,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -176,8 +176,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -201,8 +201,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -227,8 +227,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -252,8 +252,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -276,8 +276,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -301,8 +301,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -326,8 +326,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -351,8 +351,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -376,8 +376,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -401,8 +401,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -427,8 +427,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -452,8 +452,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -477,8 +477,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -503,8 +503,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -529,8 +529,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -555,8 +555,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -580,8 +580,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -606,8 +606,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -630,8 +630,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -654,8 +654,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -679,8 +679,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -704,8 +704,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -729,8 +729,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -754,8 +754,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -779,8 +779,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -803,8 +803,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -827,8 +827,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -851,8 +851,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -876,8 +876,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -900,8 +900,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -924,8 +924,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -948,8 +948,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -972,8 +972,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -996,8 +996,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -1020,8 +1020,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1045,8 +1045,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1069,8 +1069,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -1094,8 +1094,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -1118,8 +1118,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -1142,8 +1142,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1166,8 +1166,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -1190,8 +1190,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1214,8 +1214,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1238,8 +1238,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1262,8 +1262,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1286,8 +1286,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -1310,8 +1310,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1334,8 +1334,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1358,8 +1358,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1382,8 +1382,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -1406,8 +1406,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -1430,8 +1430,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -1454,8 +1454,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1478,8 +1478,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1503,8 +1503,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1527,8 +1527,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1551,8 +1551,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1575,8 +1575,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -1599,8 +1599,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1623,8 +1623,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -1647,8 +1647,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1671,8 +1671,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1695,8 +1695,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1719,8 +1719,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -1743,8 +1743,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1767,8 +1767,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1791,8 +1791,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1815,8 +1815,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1840,8 +1840,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1864,8 +1864,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -1889,8 +1889,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1913,8 +1913,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -1937,8 +1937,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1962,8 +1962,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -1986,8 +1986,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -2010,8 +2010,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -2035,8 +2035,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2059,8 +2059,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2083,8 +2083,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2107,8 +2107,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2132,8 +2132,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2156,8 +2156,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2180,8 +2180,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2204,8 +2204,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2228,8 +2228,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2252,8 +2252,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2276,8 +2276,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2300,8 +2300,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -2324,8 +2324,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -2348,8 +2348,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2372,8 +2372,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2396,8 +2396,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2421,8 +2421,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2445,8 +2445,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -2469,8 +2469,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -2494,8 +2494,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2518,8 +2518,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2542,8 +2542,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2566,8 +2566,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2590,8 +2590,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -2614,8 +2614,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2639,8 +2639,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -2664,8 +2664,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2689,8 +2689,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -2713,8 +2713,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2737,8 +2737,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2762,8 +2762,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2786,8 +2786,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -2810,8 +2810,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2834,8 +2834,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2858,8 +2858,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2882,8 +2882,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -2906,8 +2906,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2930,8 +2930,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -2954,8 +2954,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -2978,8 +2978,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -3002,8 +3002,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3026,8 +3026,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3050,8 +3050,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3074,8 +3074,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -3098,8 +3098,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3122,8 +3122,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3146,8 +3146,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3170,8 +3170,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3194,8 +3194,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3218,8 +3218,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -3242,8 +3242,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3266,8 +3266,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -3290,8 +3290,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3314,8 +3314,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -3338,8 +3338,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -3362,8 +3362,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3386,8 +3386,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3410,8 +3410,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3434,8 +3434,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3459,8 +3459,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -3483,8 +3483,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -3508,8 +3508,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3532,8 +3532,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3556,8 +3556,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3580,8 +3580,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -3604,8 +3604,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -3628,8 +3628,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -3653,8 +3653,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -3677,8 +3677,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3701,8 +3701,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -3725,8 +3725,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3750,8 +3750,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3774,8 +3774,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3798,8 +3798,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": false, @@ -3822,8 +3822,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3846,8 +3846,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3871,8 +3871,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": false, @@ -3895,8 +3895,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3919,8 +3919,8 @@ "deepseek_deepseek_chat_v3_0324": false, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": false, "inception_mercury_coder_small_beta": true, @@ -3943,8 +3943,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3967,8 +3967,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -3991,8 +3991,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -4015,8 +4015,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": false, "google_gemini_2_0_flash_lite_001": false, + "google_gemini_2_5_flash_lite_preview_06_17": false, "google_gemini_2_5_flash_preview": false, - "google_gemini_2_5_flash_preview_05_20": false, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -4039,8 +4039,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": true, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, @@ -4063,8 +4063,8 @@ "deepseek_deepseek_chat_v3_0324": true, "google_gemini_2_0_flash_001": true, "google_gemini_2_0_flash_lite_001": true, + "google_gemini_2_5_flash_lite_preview_06_17": true, "google_gemini_2_5_flash_preview": true, - "google_gemini_2_5_flash_preview_05_20": true, "google_gemini_flash_1_5": false, "google_gemma_3_27b_it": true, "inception_mercury_coder_small_beta": true, diff --git a/reports/total.md b/reports/total.md index 9965083..783b3d8 100644 --- a/reports/total.md +++ b/reports/total.md @@ -2,11 +2,11 @@ - google_gemini_2_0_flash_001: 87% - openai_o3_mini_high: 85% -- google_gemini_2_5_flash_preview_05_20: 84% - deepseek_deepseek_chat_v3_0324: 82% - google_gemini_2_5_flash_preview: 82% - deepseek_deepseek_chat_free: 79% - google_gemini_2_0_flash_lite_001: 79% +- google_gemini_2_5_flash_lite_preview_06_17: 78% - deepseek_deepseek_chat: 77% - google_gemma_3_27b_it: 77% - openai_gpt_4o_mini: 77%
МодельУспешность
google_gemini_2_0_flash_00187%
openai_o3_mini_high85%
google_gemini_2_5_flash_preview_05_2084%
deepseek_deepseek_chat_v3_032482%
google_gemini_2_5_flash_preview82%
deepseek_deepseek_chat_free79%
google_gemini_2_0_flash_lite_00179%
google_gemini_2_5_flash_lite_preview_06_1778%
deepseek_deepseek_chat77%
google_gemma_3_27b_it77%
openai_gpt_4o_mini77%