How to Serve MiniCPM5-2B with vLLM or SGLang

01

MiniCPM5-2B is a self-hosted API path

MiniCPM5-2B is not currently documented by OpenBMB as a separately billed hosted commercial API. The verified server path is self-hosting the downloadable model through supported runtimes such as vLLM or SGLang.

Both runtimes are listed by OpenBMB as OpenAI-server deployment backends, so an application that already speaks the familiar chat-completions schema can sit in front of a local or private MiniCPM5-2B deployment.

02

Serve MiniCPM5-2B with vLLM

OpenBMB's current deployment table lists vLLM for BF16/FP16 OpenAI-server use. The exact installation requirements belong to the current vLLM cookbook, so recheck the upstream guide instead of pinning an old version from a launch-day draft.

Once the server is running, validate the model identifier, chat template, streaming behavior and the exact request fields your application sends before placing the endpoint behind production traffic.

03

Use SGLang for the documented tool-calling path

OpenBMB currently marks SGLang as the recommended backend for tool or function calling. Its model card documents a server command using python -m sglang.launch_server --model-path openbmb/MiniCPM5-2B --port 30000 --tool-call-parser minicpm5, with auto also shown as an alternative parser setting.

MiniCPM5-2B emits XML-style tool calls, and OpenBMB says SGLang's built-in MiniCPM5 parser converts them into OpenAI-compatible tool_calls. That is an interoperability feature, not a guarantee that every tool schema or multi-step agent loop will behave correctly.

04

Validate agent behavior before production

Test your exact tool schemas, malformed-call recovery, multi-step behavior, tool-result injection, long-context behavior and concurrency rather than relying on aggregate benchmark tables. OpenBMB's tool-use and agent scores are vendor-published evaluations.

05

Context, security and infrastructure cost

The model card lists a 131,072-token native context window, but maximum context is not necessarily the best operating point. KV-cache memory, latency and throughput can change substantially as active context grows, so start smaller and measure.

Infrastructure cost depends on the hardware, runtime, context size and concurrency you choose. Put authentication, network controls, logging and rate limits around any externally reachable endpoint, and do not expose an unauthenticated local server directly to the public internet.

For the model's release details, license and benchmark boundaries, return to the MiniCPM5-2B overview. If you want desktop or on-device use rather than an API server, use the local GGUF, Ollama, LM Studio and MLX guide.