All research

Research · Thread 05

Domain LLM fine-tune

Teaching a general-purpose model the vocabulary of one market, on one GPU.

QLoRA is a way of adapting a large language model by training a small set of extra weights instead of retraining the whole thing, which is what brings the job down from a server rack to a single graphics card. The question it made affordable to ask was whether a domain-tuned model actually beats a base model with retrieval for a narrow, terminology-heavy vertical where the entities are rare. BARNES Dubai LLM is the production answer for Dubai luxury real estate, trained on national registry and brokerage records rather than on scraped listings.

Findings

Paper to shipped

  1. Paper

    QLoRA: Efficient Finetuning of Quantized LLMs (Dettmers, Pagnoni, Holtzman, Zettlemoyer, NeurIPS 2023)

    4-bit NormalFloat + double quantisation + paged optimisers, the toolkit that brought 33B+ fine-tunes onto a single workstation GPU.

    View abstract
    QLoRA is an efficient finetuning approach that reduces memory usage enough to finetune a 65B-parameter model on a single 48GB GPU while preserving full 16-bit-finetune task performance. QLoRA backpropagates gradients through a frozen 4-bit quantized pretrained language model into Low-Rank Adapters (LoRA), and introduces three contributions: a new 4-bit NormalFloat data type, double quantization, and paged optimisers to manage memory spikes.
  2. Implementation

    15,369 instruction pairs, multi-source Dubai corpus

    Built from national registry transactions and brokerage inventory rather than scraped listings, then wrapped in tool calls so the model looks a figure up rather than recalling it.

  3. Shipped

    BARNES Dubai LLM v8 shipped

    FastAPI service, integrated with WhatsApp and tested by brokers through 2026, though not put into day-to-day use in the field. The answer it gave to 'can a domain LLM beat base + RAG for our vertical' was 'yes, on terminology and named-entity recall'.

Contact

Pavlo Puzikov, a black and white studio portrait in a pinstripe suit.

Pavlo Puzikov

Marketing and innovation: immersive 3D, applied AI, generative content

Write to

pavlopuzikov@gmail.com

Open to
Freelance projects, remote. Full-time roles, onsite or hybrid in the EU.
Based in
Bulgaria
Most recent role
Marketing and Innovation Lead, BARNES Dubai, 2026. Product Innovation Researcher there from 2025.
Education
BCom, Marketing and Management, University of Auckland, 2020 to 2023
Languages
English, Russian