AegeanWorks

ModelDuck is a local-first GGUF inference server and hardware-aware model optimization platform. Running an LLM locally is not simply about downloading weights. Choosing the right quantization, context length, KV cache, CPU/GPU allocation, and sampling configuration can dramatically change performance and output quality. ModelDuck addresses this problem through automated benchmarking and adaptive tuning. It evaluates models against reproducible coding, reasoning, code comprehension, and long-context tasks, measures their resource consumption and performance, and identifies configurations that balance quality, speed, and memory usage. This submission demonstrates the optimization of Meta's Llama 3.2 3B Instruct alongside other open-weight models on consumer hardware, without requiring a cloud inference service.