vllm.benchmarks
Modules:
| Name | Description |
|---|---|
datasets | This module defines a framework for sampling benchmark requests from various |
latency | Benchmark the latency of processing a single batch of requests. |
lib | Benchmark library utilities. |
serve | Benchmark online serving throughput. |
throughput | Benchmark offline inference throughput. |