What it is
onnx-model-chunking is a toolkit for intelligent bisection, analytics, and quality checks on large ONNX models. The point is to deploy models like Qwen2, Llama, and Mistral on devices that cannot keep the full graph in memory.
Why it matters
A model that only runs if you own the biggest GPU is not a local-first model. Chunking at the graph level — not “hope quantization is enough” — is how you get a large network onto constrained hardware without pretending the memory problem disappeared.
How it works
The README describes automatic split-point selection from weight distribution and layer structure, plus a benchmarking path (including a stateful/fair-split proof-of-concept documented in the tree). Helper modules cover chunking utilities and pipeline utilities so the split is repeatable instead of a one-off notebook.
The public artifact is the ONNX splitter.
Status
Public at antmikinka/onnx-model-chunking.