vllm.worker
Modules:
| Name | Description |
|---|---|
cache_engine | CacheEngine class for managing the KV cache. |
enc_dec_model_runner | |
model_runner | |
model_runner_base | |
neuron_model_runner | |
neuron_worker | A Neuron worker class. |
neuronx_distributed_model_runner | |
pooling_model_runner | |
utils | Worker-related helper functions. |
worker | A GPU worker class. |
worker_base | |