https://mathscantor.github.io/tags/llama-cpp/ Recent content in Llama-Cpp on Mathscantor's Cybersecurity Blog Hugo en <a href="https://creativecommons.org/licenses/by-nc/4.0/" target="_blank" rel="noopener">CC BY-NC 4.0</a> Mon, 25 May 2026 10:51:00 +0800 https://mathscantor.github.io/posts/spinning-up-local-llms/ Mon, 25 May 2026 10:51:00 +0800 https://mathscantor.github.io/posts/spinning-up-local-llms/ <h2 id="1-introduction">1. Introduction</h2> <p>I have an old desktop with an NVIDIA RTX 2070 SUPER (8GB VRAM) sitting around, quietly collecting dust and judging me. So naturally, I decided to give it a job: run a local LLM server with <a href="https://github.com/ggml-org/llama.cpp">llama.cpp</a>, then wire it up to an AI coding agent.</p> <p>No cloud tokens, no monthly bill anxiety, no sending prompts halfway across the planet. Just one old GPU, a quantized model, and a little bit of stubbornness.</p>