Guides

Task-oriented documentation: how to measure PocketLLM, how to call it, and how to release it. Start with Benchmarking and reporting rules before quoting any performance number from this site — it defines the measurement conventions the rest of the documentation assumes.

Guide	What it covers
Benchmarking and reporting rules	The prefill/decode split, what every result record must contain, and the hardware and invocation details a comparable number needs.
Serving latency metrics (vLLM convention)	TTFT, TPOT, ITL and E2EL as vLLM defines them, the pocket_* series that back each one, and how they relate to the prefill/decode convention.
PocketLLM API and backend guide	The single user-facing API, backend selection, and the batch/scheduler surface — documented in RelicLLM, where the multi-card execution planes it also covers now live.
PyPI release guide	The single source of truth for releasing pocketllm to PyPI, including the Test PyPI dry run.
Building and loading the CUDA extensions	Which .so the loader resolves and in what order, and the two silent wrong-build failures and the three checks that separate them. The extensions compile in relic-core, not here.
Ascend SoC generations	Why 910B and 910B1–910B4 are different chips, how to read Short_SoC_version, and why they cannot share kernels.

For install and build instructions, see Getting started. For per-model runtimes see Model guides.