Series · 12 parts

Small Models on One DGX Spark

Five open-weight models, eight real tasks, one desk-side box. A measured answer to the question every team asks: which of our large-model calls could a small local model take over?

Every team using a frontier model eventually asks the same thing: how much of this could a small open model do on our own hardware? Benchmarks don’t answer it, because leaderboard tasks aren’t your tasks, and a model that’s “close on average” can fail in exactly the way that hurts you.

So we ran the experiment. We took five open-weight models under 100B parameters, served them one at a time on a single NVIDIA DGX Spark (GB10, 121 GB unified memory), and gave them eight narrow, realistic tasks. Each task had a pass rule written down before any model ran. Every result was compared with Claude Opus 5.5 and DeepSeek-V4.1-Flash, and every surprising score was checked by reading the traces.

3tasks small-model ready: screening, report structuring, tool-calling agents
1task that needs tuning: numeric extraction from trial papers
4tasks that still need a large model, two of them only narrowly
≈ 62 hof GPU time on one box, over 8 days

The verdict on one page

Best small model against the best large-model reference, per task

Primary metric for each task, 0–100 (F1 scores shown ×100). Status: whether the best local model met both pass bars.

Best small modelLarge-model reference (Opus; DeepSeek for E1 and E4)
Small model ready
L1 Screening99.0
100.0
L2 Structuring100.0
100.0
E4 Tool loops90.0
88.7
Needs tuning
L4 Extraction74.2
77.9
Needs a large model
E3 Enrichment96.0
98.1
E2 Log triage86.1
91.8
L3 Guideline Q&A81.7
90.8
E1 Coding72.5
95.0
050100

The pattern in one sentence: small models are ready for closed, well-specified work, such as judging one item against explicit criteria, filling a schema or following a tool policy. They fall short where the answer has to be complete, the work is long and open-ended, or the benchmark encodes conventions the model can’t see.

The three verdicts are about model size, not where the model runs. “Needs a large model” doesn’t mean “needs an API”: large open-weight models such as DeepSeek-V4.1-Flash or GLM 5.3 can be deployed locally too, given more hardware than one desk-side box.

How to read this series

Start with part 1 for the results and the practical guidelines, and part 2 if you want to run something similar yourself. Each task post (parts 3–12) stands on its own and follows the same shape: what the task is and why it matters, a real input with the real model outputs, the results against the pass bar, what the traces showed, and what it means if you do this work.

The series: from the verdict to the traces

I.Start here

The overall verdict, and how the study was run on a single machine.

  1. 01 Which Jobs Can a Small Local Model Take Over? Results from Eight Real TasksFive open models under 100B parameters, eight tasks, one DGX Spark, against Claude Opus 5.5 and DeepSeek-V4.1-Flash. Three tasks are small-model ready, one needs tuning, four still need a large model. 11 min
  2. 02 62 GPU-Hours on One Box: How We Ran a Small-Model Evaluation on a DGX SparkProcess: 5 models, 8 tasks and a LoRA on one DGX Spark took 8 days 10 hours, about 62 GPU-hours and about $95 of API spend. About 54 hours were lost to idle time after jobs finished. 14 min

II.Getting the models running

Before any task: can each model be served fast enough, without silently truncating prompts or breaking tool calls?

  1. 03 Before Any Benchmark: Can Five Open Models Run Well on One DGX Spark?T01: all five models kept a 32k-token prompt intact and finished a 5-call tool chain. The four MoE models decode at 52–66 tok/s; the dense Qwen3.8-27B manages 21. 9 min

III.Life-science and clinical tasks

Reading medical literature, structuring scan reports, answering from guidelines and extracting trial numbers.

  1. 04 Can a Local Model Screen a Systematic Review? Yes — Here's the RecipeL1: five open models ranked 3,603 abstracts from four medical systematic reviews. The best put 99% of the papers humans included in the top half, within a point of Claude Opus 5.5. 7 min
  2. 05 Turning Scan Reports into JSON Is Solved. Spotting Contradictions Isn'tL2: every local model turned 300 synthetic PET/CT and SPECT reports into valid JSON at field F1 ≥ 0.98. Catching planted contradictions ranged from 88% (Qwen3.8, Gemma 4) to 18% (Nemotron). 8 min
  3. 06 Rarely Wrong, Often Incomplete: Small Models Answering from Clinical GuidelinesL3: the best local model, Qwen3.8-27B, fully answered 81.7% of answerable guideline questions against 90.8% for Claude Opus 5.5. Local models are wrong as rarely as the references; they drop the caveats. 9 min
  4. 07 When the Answer Key Is the Ceiling: Pulling Numbers out of Trial PapersL4: no model, small or large, reached 90% on extracting trial results for meta-analysis. Qwen3.6-35B-A3B scored 74.2%, 1.6 points behind Claude Opus 5.5, and the answer key itself caps the score. 9 min

IV.Engineering and enterprise tasks

Fixing real bugs in real repositories, triaging logs, cleaning product data and running customer-service agents.

  1. 08 Fixing Real Bugs in Real Repos: Where Small Models Still Trail by 22.5 PointsE1: five local models drove a coding agent on 40 real SWE-bench issues. The best, Qwen3.6-35B-A3B, fixed 72.5%; DeepSeek-V4.1-Flash fixed 95%. The widest small-vs-large gap in the study. 10 min
  2. 09 Log Triage: Severity Is Solved, Fault Labels Are a ConventionE2: Qwen3.8-27B classifies log lines at macro-F1 0.861, 5.7 points behind Claude Opus 5.5. Three local models rate alert severity at 98.8–100%, and the gap is mostly lines the experts left untagged. 9 min
  3. 10 2,360 Offers, 40 Categories: Small Models Land Two Points Short on Product DataE3: Gemma 4 filed 96.0% of 2,360 web-shop offers in the right category, for under a cent per thousand. Claude Opus 5.5 scored 98.1%, so the rule says it still needs a large model, by 0.08 points. 8 min
  4. 11 A 3B-Active Model That Matches DeepSeek at Running Customer-Service AgentsE4: on 150 simulated customer-service tasks, Qwen3.6-35B-A3B passed 90.0% against 88.7% for DeepSeek-V4.1-Flash. The strongest local result in the study; two other models looped or invented ids. 9 min

V.Can fine-tuning close the gap?

A LoRA on the one task that landed on “needs tuning”, and why it learned the wrong lesson.

  1. 12 The LoRA That Learned the Wrong Lesson: Fine-Tuning on Synthetic PapersPhase 4: a LoRA on Qwen3.6-35B-A3B lifted synthetic-paper extraction from 90.0% to 94.6%, but real L4 went from 74.2% to 73.8%. It learned to answer more often, including where the answer key is empty. 9 min

Published so far