Research · Thread 05
Domain LLM fine-tune
Teaching a general-purpose model the vocabulary of one market, on one GPU.
QLoRA is a way of adapting a large language model by training a small set of extra weights instead of retraining the whole thing, which is what brings the job down from a server rack to a single graphics card. The question it made affordable to ask was whether a domain-tuned model actually beats a base model with retrieval for a narrow, terminology-heavy vertical where the entities are rare. BARNES Dubai LLM is the production answer for Dubai luxury real estate, trained on national registry and brokerage records rather than on scraped listings.
Findings
Domain fine-tune vs base + RAG
QLoRA made a 7-13B fine-tune tractable on one GPU. BARNES Dubai LLM v8 was the production test: does a domain tune actually beat base + RAG for Dubai real estate?
Yes, on terminology and named-entity recall.
A multi-source property corpus
15,369 pairs: DLD transactions, buildings, brokers, developers, plus UK/FR/SG comparables. Tool-calling agents wrap it with inventory and valuation lookups.
The corpus taught the model the team's judgement, not just the registries' wording.
Paper to shipped
- QLoRA: Efficient Finetuning of Quantized LLMs (Dettmers, Pagnoni, Holtzman, Zettlemoyer, NeurIPS 2023)
4-bit NormalFloat + double quantisation + paged optimisers, the toolkit that brought 33B+ fine-tunes onto a single workstation GPU.
View abstract
QLoRA is an efficient finetuning approach that reduces memory usage enough to finetune a 65B-parameter model on a single 48GB GPU while preserving full 16-bit-finetune task performance. QLoRA backpropagates gradients through a frozen 4-bit quantized pretrained language model into Low-Rank Adapters (LoRA), and introduces three contributions: a new 4-bit NormalFloat data type, double quantization, and paged optimisers to manage memory spikes.
- 15,369 instruction pairs, multi-source Dubai corpus
Built from national registry transactions and brokerage inventory rather than scraped listings, then wrapped in tool calls so the model looks a figure up rather than recalling it.
- BARNES Dubai LLM v8 shipped
FastAPI service, integrated with WhatsApp and tested by brokers through 2026, though not put into day-to-day use in the field. The answer it gave to 'can a domain LLM beat base + RAG for our vertical' was 'yes, on terminology and named-entity recall'.
