naive_speculate.infer.lm.qwen3¶
Provide Qwen3 model implementation of LanguageModel.
Qwen3LanguageModel
¶
Bases: LanguageModel
Qwen3LanguageModel wraps huggingface Qwen3 models, implementing LanguageModel.
Attributes:
| Name | Type | Description |
|---|---|---|
hf_model |
Qwen3ForCausalLM
|
The underlying huggingface Qwen3 model. |
forward(query_token_ids, kv_cache)
¶
Forward the underlying huggingface model.
Expects kv_cache to be an instance of DynamicCache.
Refers to LanguageModel.forward for more details.
Raises:
| Type | Description |
|---|---|
ValueError
|
If the model forward output does not contain logits. |