Skip to content

You are viewing the latest developer preview docs. Click here to view docs for the latest stable release.

vLLM

vllm.v1.core

Initializing search

Home
User Guide
Developer Guide
API Reference
CLI Reference
Community

vLLM

Home
User Guide
User Guide
- Getting Started
  Getting Started
  - Quickstart
  - Installation
    
    Installation
    
    GPU
    
    CPU
    
    Google TPU
    
    Intel Gaudi
    
    AWS Neuron
- Examples
  Examples
- General
  General
- Inference and Serving
  Inference and Serving
  - Offline Inference
  - OpenAI-Compatible Server
  - Data Parallel Deployment
  - Troubleshooting distributed deployments
  - Expert Parallel Deployment
  - Parallelism and Scaling
  - Integrations
    Integrations
    
    LangChain
    
    LlamaIndex
- Deployment
  Deployment
  - Using Docker
  - Using Kubernetes
  - Using Nginx
  - Frameworks
    Frameworks
    
    Anyscale
    
    Anything LLM
    
    AutoGen
    
    BentoML
    
    Cerebrium
    
    Chatbox
    
    Dify
    
    dstack
    
    Haystack
    
    Helm
    
    LiteLLM
    
    Lobe Chat
    
    LWS
    
    Modal
    
    Open WebUI
    
    Retrieval-Augmented Generation
    
    SkyPilot
    
    Streamlit
    
    NVIDIA Triton
  - Integrations
    Integrations
    
    KServe
    
    KubeAI
    
    KubeRay
    
    Llama Stack
    
    llmaz
    
    Production stack
- Training
  Training
  - Reinforcement Learning from Human Feedback
  - Transformers Reinforcement Learning
- Configuration
  Configuration
- Models
  Models
  - Supported Models
  - Generative Models
  - Pooling Models
  - Extensions
    Extensions
    
    Loading Model weights with fastsafetensors
    
    Loading models with Run:ai Model Streamer
    
    Loading models with CoreWeave's Tensorizer
  - Hardware Supported Models
    Hardware Supported Models
    
    TPU
- Features
  Features
Developer Guide
Developer Guide
- General
  General
- Model Implementation
  Model Implementation
- CI
  CI
  - CI Failures
  - Update PyTorch version on vLLM OSS CI/CD
- Design Documents
  Design Documents
API Reference
API Reference
CLI Reference
CLI Reference
- vllm serve
- vllm chat
- vllm complete
- vllm run-batch
- vllm bench
  vllm bench
Community
Community
- Contact Us
- Meetups
- Sponsors
- Blog
- Forum
- Slack

Table of contents

core

vllm.v1.core

Modules:

Name	Description
`block_pool`
`encoder_cache_manager`
`kv_cache_coordinator`
`kv_cache_manager`
`kv_cache_utils`	KV-Cache Utilities.
`sched`
`single_type_kv_cache_manager`

August 26, 2025

Made with Material for MkDocs