Skip to content

Add FP8 support for Llama3.1-8b#83

Draft
blzheng wants to merge 3 commits intomingfeima:cpu_opt_ww11from
blzheng:beilei/llama3_fp8
Draft

Add FP8 support for Llama3.1-8b#83
blzheng wants to merge 3 commits intomingfeima:cpu_opt_ww11from
blzheng:beilei/llama3_fp8

Conversation

@blzheng
Copy link
Collaborator

@blzheng blzheng commented Jun 18, 2025

Motivation

This PR is for the DMR LZ KPI POC. It adds FP8 support for Llama3.1-8B using the recipe from INC, and also supports prompt files in the format shown in this example.

Modifications

Checklist

CaoE pushed a commit to CaoE/sglang that referenced this pull request Oct 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant