Provide SampleStrategy and VerifyStrategy for token generation and verification.
SampleStrategy
Bases: StrEnum
Sampling strategies for token generation.
Attributes:
| Name |
Type |
Description |
RANDOM |
|
Sample tokens probabilistically according to the token distribution over vocabulary.
|
GREEDY |
|
Always select the token with the highest probability (argmax).
|
VerifyStrategy
Bases: StrEnum
Verification strategies for speculative decoding.
Attributes:
| Name |
Type |
Description |
GREEDY_MATCH |
|
Verify drafted tokens using greedy matching.
|
SPECULATIVE_SAMPLING |
|
Verify drafted tokens using speculative sampling.
|