How I Built a Custom LLM
for CS Education with Blix.ai

Sayan Dutta

Full-Stack Developer & AI Enthusiast · IIT Patna

Creator of Blix.ai — a privacy-first, self-hosted AI tutor for CS students, currently in closed beta with IIT Patna classmates. This is the complete technical story of how it was built, what failed, and what actually works.

Every CS student has been there: it's 1am, you're staring at a question about semaphore implementation, your professor is unreachable, Stack Overflow gives you five contradictory answers, and the textbook explanation assumes you already understand the thing it's supposed to be explaining.

That gap — between needing a clear, contextual answer and getting one — is what Blix.ai exists to close. Not by wrapping GPT-4 with a system prompt (though that's a valid approach), but by training a model specifically on the subjects CS students actually struggle with: Operating Systems, DBMS, Data Structures & Algorithms, Computer Networks, AI/ML fundamentals, and more.

This post is the complete technical account of how that happened — what I built, what broke, what I'd do differently, and why the hardest problem wasn't the model at all.

Why Not Just Use GPT-4?

This is always the first question. And it's a fair one — frontier models are excellent and getting better. But there are three reasons a fine-tuned domain-specific model made sense for this project:

• Cost at scale. GPT-4o pricing makes a free, high-usage educational tool economically unsustainable without significant funding. A self-hosted quantised model runs on a single VPS for a fixed monthly cost, regardless of query volume.
• Depth vs breadth. General models are trained to be helpful across everything. That breadth means shallower knowledge per domain. A model trained specifically on OS theory, DBMS normalisation, and algorithm complexity has more targeted, reliable knowledge in those areas — and doesn't hallucinate textbook authors or fabricate theorem names.
• Privacy. Students sometimes share their actual assignment questions or exam prep notes. A self-hosted model means that data never leaves the server. No vendor sees the queries.

The Build: Four Phases

Phase 1: Dataset Curation

The quality of a fine-tuned model is almost entirely determined by the quality of its training data. I spent more time on this phase than any other, and I'd spend even more if I were doing it again.

Sources

The dataset was assembled from five categories of sources:

Total after deduplication and quality filtering: ~50,000 instruction-response pairs across six subjects, formatted as {"instruction": "...", "response": "..."} with a subject tag on each sample.

The Cleaning Pipeline

def is_quality_sample(sample: dict) -> bool:
    instr  = sample["instruction"]
    resp   = sample["response"]

    # length gates
    if len(instr.split()) < 8:   return False  # too vague
    if len(resp.split())  < 30:  return False  # too short to be useful
    if len(resp.split())  > 800: return False  # likely padded / off-topic

    # reject samples with uncertainty markers
    uncertainty = ["i'm not sure", "i think", "maybe",
                   "i believe", "not certain"]
    if any(p in resp.lower() for p in uncertainty):
        return False

    # reject if response doesn't address the instruction topic
    instr_keywords = extract_keywords(instr)
    overlap = keyword_overlap(instr_keywords, resp)
    if overlap < 0.15: return False

    return True

Phase 2: Base Model & Fine-Tuning

Choosing the Base Model

The constraints were clear: the model needed to run on a single consumer-grade VPS (4 vCPU, 16GB RAM, no GPU) at inference time. That ruled out anything above 7B parameters without aggressive quantisation.

I evaluated three candidates — a 7B Llama-2 variant, Mistral 7B, and Phi-2 (2.7B). Mistral 7B with 4-bit GPTQ quantisation hit the best balance of response quality, instruction following, and inference speed (~2.1 tokens/sec on the VPS, which streams acceptably).

LoRA Fine-Tuning Setup

Full fine-tuning a 7B model was out of the question on available compute. LoRA (Low-Rank Adaptation) lets you fine-tune a fraction of the parameters — typically 1–3% — while keeping the rest frozen. The trainable adapter weights are tiny (~50MB) and can be merged back into the base model after training.

from peft import LoraConfig, get_peft_model

lora_config = LoraConfig(
    r=16,                    # rank — higher = more capacity
    lora_alpha=32,           # scaling factor
    target_modules=[         # which layers to adapt
        "q_proj", "v_proj",
        "k_proj", "o_proj"
    ],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(base_model, lora_config)
model.print_trainable_parameters()
# trainable params: 41,943,040 / 7,241,748,480 — 0.58%

Training ran on Google Colab's free TPU tier for initial experiments, then on a rented A100 instance (via Vast.ai) for the full run. Total compute cost for the full fine-tuning: approximately $18.

Phase 3: The Hardest Problem — Alignment

Fine-tuning on Q&A pairs improved the model's CS knowledge substantially. But it introduced a new problem that was worse than ignorance: confident wrongness.

A fine-tuned model that has seen many OS questions will, when faced with an unfamiliar edge case, generate a fluent, confident, textbook-sounding answer — that is completely wrong. A student trusting that answer is worse off than a student who got no answer at all.

DPO: Teaching the Model to Prefer Honesty

I used Direct Preference Optimisation (DPO) — a simpler alternative to RLHF — to teach the model to prefer correct, hedged answers over confident wrong ones. DPO requires preference pairs: for each prompt, a "chosen" response and a "rejected" response.

{
  "prompt": "Can a deadlock occur with only one process?",

  "chosen": "No, a deadlock requires at least two processes
    competing for resources held by each other.
    A single process can block waiting for a resource
    it holds itself (self-deadlock in some contexts),
    but the classical four Coffman conditions require
    multiple processes.",

  "rejected": "Yes, a deadlock can occur with a single process
    if it requests a resource it already holds and
    the system does not support recursive locking.
    This is the most common form of deadlock in
    single-threaded applications."
}

I collected ~4,000 preference pairs — roughly half from cases where the SFT model gave confident wrong answers (I verified these manually against textbooks), and half synthetically generated with GPT-4 and verified.

Phase 4: Inference Server & Architecture

The model doesn't help anyone sitting in a notebook. The beta testing architecture currently serving Blix.ai to a closed group of IIT Patna students looks like this:

Streaming with Server-Sent Events

Waiting 8–12 seconds for a full response is unusable. Streaming tokens as they're generated makes the same latency feel fast. The FastAPI endpoint uses Python generators and SSE to push tokens to the browser in real time:

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio

app = FastAPI()

async def token_stream(prompt: str):
    # Check Redis cache first
    cached = await redis.get(cache_key(prompt))
    if cached:
        yield f"data: {cached}\n\n"
        return

    full_response = []
    # Stream tokens from the quantised model
    for token in model.generate_stream(prompt):
        full_response.append(token)
        yield f"data: {token}\n\n"
        await asyncio.sleep(0)   # yield to event loop

    # Cache the full response async
    await redis.setex(cache_key(prompt),
                     3600,
                     "".join(full_response))

@app.post("/ask")
async def ask(req: AskRequest):
    return StreamingResponse(
        token_stream(req.prompt),
        media_type="text/event-stream"
    )

RAG for Grounding

For complex multi-step questions — like tracing through a specific scheduling algorithm with a given process table — the fine-tuned model alone sometimes drifted. I added a lightweight RAG layer: a FAISS vector store containing chunked sections from the core textbooks. For queries above a complexity threshold, relevant chunks are retrieved and prepended to the prompt as context.

This combination — fine-tuned domain knowledge plus retrieval-augmented grounding for edge cases — reduced the confident hallucination rate from approximately 12% in the SFT-only model to under 3% in beta testing.

The Challenges I Didn't Expect

What the Numbers Look Like Now

After six months of running Blix.ai in closed beta with a group of IIT Patna classmates, here's where things stand:

• Hallucination rate (verified wrong confident answers): ~2.8% — down from 12% in the initial SFT-only model
• Cache hit rate: ~41% — classmates ask very similar CS theory questions
• Average response latency (uncached): 9.4 seconds to first token, then streaming
• Beta group satisfaction (informal survey, n=47): 78% rated answers "more useful than a Google search" for conceptual questions
• Monthly VPS cost during beta: ~$22 — sustaining the closed beta for free for all testers

What's Next: Blix.ai v2

Version 2 is currently in development, with a public launch planned for February 2027. The main improvements planned for v2:

• Larger, cleaner dataset — targeting 150k pairs with stricter quality gates and more synthetic verified data across newer topics (LLM fundamentals, cloud architecture, system design)
• Multi-turn conversation memory — the current model treats every question as independent. v2 will maintain session context so students can ask follow-up questions naturally
• Better "I don't know" calibration — using confidence scoring at inference time to decide when to retrieve rather than generate
• Structured output for code questions — a separate generation mode for questions that should produce executable code with test cases

The code for Blix.ai — the data pipeline, fine-tuning scripts, inference server, and frontend — is open-source on GitHub. If you're building something similar for another domain, it's a reasonable starting point. Contributions and issue reports are welcome.

"Quality Education for All" — UN SDG4. That's the north star. A CS student in a tier-3 city with no TA access should have the same quality of conceptual help as someone at a well-resourced institution. Blix.ai is one small step toward that.

Sayan Dutta

Full-Stack Developer & AI Enthusiast · IIT Patna

GitHub Get in touch