Fine-tuning tiny models as the decision brain: the candidate list and where the training data comes from

Since [part one](/blog/quant-not-a-gambling-machine) the "self-hosted decision model" has been the protagonist without ever answering the basic question: where does this model come from? This post closes that loop: take tiny open-weight models, fine-tune them on data your own system produces, and grow a decision brain. To be clear about what this is — a research direction, not a tutorial. The list will age; the method will not.

Why tiny models are enough

Because deciding is a narrow task. The input is structured market state — an indicator array plus a few lines of Market Context; the output is signal JSON with confidence. Not open chat, no human knowledge to memorize — just stable judgment in an extremely narrow domain. On narrow tasks, a small model fed on proprietary data beats a general model that knows a bit of everything — doubly true in markets. Two practical accounts: 0.5B–4B models deploy on free ModelScope or Colab tiers, and a quantized 7B runs on a consumer GPU; inference is sub-second, and the data never leaves the box.

The candidate list (0.5B–7B, all of it perishable, today's prices)

Qwen3-0.6B / 1.7B / 4B — strong in Chinese and English, Apache 2.0, the fullest tool-calling ecosystem: default base and best value. DeepSeek-R1-Distill-Qwen-1.5B / 7B, R1-Distill-Llama-8B — reasoning distilled in, suited to decision paths that need a chain of thought, at the cost of more reasoning tokens. Llama-3.2-1B / 3B — the safe English base. Phi-4-mini — Microsoft; per-parameter reasoning that punches above its weight. Gemma-3-1B / 4B — the Google line, clean multilingual. SmolLM3-1.7B — HuggingFace's genuinely tiny model, good for testing ideas. Worth dissecting (not deploying): FinGPT, InvestLM, DISC-FinLLM, XuanYuan — financial fine-tunes whose value is in how they define the task, not how they score.

Where the training data comes from — the most elegant loop in this design

From the decision memory itself. The daily rows — [market state | AI call | actual outcome | correct?] — are a labeled dataset waiting to happen: verified-correct calls become SFT input-output pairs; a right call and a wrong call in the same scenario become a DPO chosen/rejected pair. The longer the system runs, the thicker the training set — the system manufactures its own upgrade material. This is the hypothesis I most want to test in this whole research direction: whether a one-person company can raise its own decision model on its own operating data.

Method sketch

Formalize the task: input = indicator array + Market Context text; output = JSON `{signal, confidence, sizing, rationale}`. Train with QLoRA: a 1.7B runs overnight on a free Colab T4; a 0.6B takes an afternoon. Serve with vLLM constrained decoding (JSON-schema guided) to weld the output format shut — small models hallucinate little in narrow domains, but the schema must be ironclad. Evaluation ignores perplexity; it reads out-of-sample decision accuracy and backtest Sharpe — the same iron law as Phase 4 in the [roadmap](/blog/quant-roadmap-and-console).

Three pitfalls, written down in advance

One: regime overfitting — train on a bull-market-only memory and the model learns "always long." Decision memory must accumulate across market states; slower is fine. Two: do not fine-tune early — before the memory base has a few hundred outcome-labeled calls, fine-tuning is a castle in the air. Run Phase 2 first. Three: the small model does not take the fall — fine-tuning buys narrow-domain stability, not regime awareness. That is Strategy Health's and version control's job, not the model's.

The series pauses here. Five posts, one through-line: run it first, bank your own data, then raise your own brain. Between here and that sentence being proven stands one Phase 1 dashboard.