Coding Agent Server on a Free GPU
Runs Qwen2.5-Coder-7B on a Colab GPU with llama.cpp and exposes it through a token-protected private tunnel, so local coding tools (Aider, Continue, Cline) get a fast open-weight model without paid APIs.
Overview
A Colab notebook that turns a free notebook GPU into a private coding-model server for local tools such as Aider, Continue and Cline.
The Problem
- A capable coding model needs a strong GPU that a student laptop doesn't have.
- Paid coding APIs add up quickly during heavy use.
How It Works
- Check the GPU runtime (L4 / A100 / T4).
- Build llama-cpp-python with CUDA support.
- Download the 4.7 GB GGUF model once to Google Drive so later sessions skip it.
- Start a FastAPI server with an OpenAI-style endpoint, protected by a bearer token.
- Open a private tunnel and point local tools at the tunnel URL.
Highlights
- Existing tools work unchanged because the endpoint follows the OpenAI format.
- New tunnel URL and token every session keep it private.
- Restoring the server in a new session takes three cells.