Labels

QLoRA on One GPU: Validate the Dataset and Prove the Run Before Deployment

QLoRA can reduce the memory needed to adapt a language model, but “single GPU” is not a hardware guarantee. Whether a run fits depends on the model, sequence length, batch size, optimizer, gradient checkpointing, GPU architecture, library versions, and temporary memory peaks.

The reliable workflow is therefore not “copy a notebook and press Train.” Validate the conversation data, make a clean train/evaluation split, confirm the model’s license and chat template, start with a short smoke run, and record the exact environment before committing hours of GPU time.

✅ This guide provides two separate pieces: a CPU-testable dataset preflight tool and a current QLoRA training entry point whose GPU results you must measure on your own machine.

Understand what QLoRA changes—and what it does not

The QLoRA paper combines a frozen 4-bit quantized base model with trainable LoRA adapters. Its key memory techniques include 4-bit NormalFloat, double quantization, and paged optimizers. The paper demonstrated that very large models could be adapted on a single 48GB GPU under its tested settings.

That result does not mean every 7B or 8B model fits every 16GB or 24GB card. Activations, attention, sequence length, adapter targets, and framework overhead still consume memory. An exact VRAM promise copied from a different model and dataset is not evidence for your run.

The base model also stays frozen; QLoRA does not make low-quality examples useful or prove that the adapted model is better. Evaluation must be defined separately from training loss.

Decide whether fine-tuning is the right intervention

Before preparing a dataset, write down the behavior you want to change.

Need Try first
Current facts or private documents retrieval with citations
A fixed output shape structured output or constrained decoding
A repeatable tone or response pattern prompting, then supervised fine-tuning if needed
New domain behavior shown by many examples supervised fine-tuning may help

Fine-tuning changes model behavior; it is not a dependable database for frequently changing facts. If the model needs today’s inventory or an exact policy paragraph, retrieve that information at request time instead of hoping the weights remember it.

📌 Define an evaluation set and a failure threshold before training. Otherwise, a lower training loss can feel like progress even when held-out answers become less useful.

Use conversational JSONL with an explicit contract

TRL’s SFTTrainer supports conversational examples stored as a list of messages. One JSON object per line keeps the dataset streamable and reviewable:

{"messages":[{"role":"system","content":"Answer using the approved support policy."},{"role":"user","content":"How do I reset the device?"},{"role":"assistant","content":"Open Settings, choose Device, then select Reset."}]}

The preflight tool in this tutorial accepts exactly one messages field. It allows an optional first system message, then requires alternating user and assistant turns ending with an assistant answer. Empty content and unknown roles are rejected.

This strict shape is a starting contract, not a universal schema. Tool calls, images, chosen/rejected pairs, or completion-only loss need a different contract and matching trainer configuration.

Validate before downloading a model

Save the validator as qlora_preflight.py. Its central check is deliberately independent of PyTorch and Hugging Face libraries:

ALLOWED_ROLES = {"system", "user", "assistant"}


def validate_example(example, line_number):
    if not isinstance(example, dict) or set(example) != {"messages"}:
        raise ValueError(f"line {line_number}: expected one 'messages' field")
    messages = example["messages"]
    if not isinstance(messages, list) or len(messages) < 2:
        raise ValueError(f"line {line_number}: messages must contain at least two turns")

    cleaned = []
    for index, message in enumerate(messages):
        if not isinstance(message, dict) or set(message) != {"role", "content"}:
            raise ValueError(f"line {line_number}, message {index}: expected role and content")
        role = message["role"]
        content = message["content"]
        if role not in ALLOWED_ROLES:
            raise ValueError(f"line {line_number}, message {index}: unsupported role {role!r}")
        if role == "system" and index != 0:
            raise ValueError(f"line {line_number}: system message is only allowed first")
        if not isinstance(content, str) or not content.strip():
            raise ValueError(f"line {line_number}, message {index}: content is blank")
        cleaned.append({"role": role, "content": content.strip()})

The full function also verifies the dialogue order and requires an assistant answer at the end. Errors contain the source line number so a bad record can be fixed instead of silently dropped.

This validator does not detect factual errors, private data, licensing problems, unsafe instructions, or subtly duplicated answers. Those require domain review.

Remove exact duplicates before splitting

If the same example appears in both training and evaluation, the reported evaluation loss can reward memorization. The preflight script canonicalizes each JSON object, hashes it with SHA-256, removes exact duplicates, sorts by hash, and then creates a deterministic split.

def canonical_example(example):
    return json.dumps(
        example,
        ensure_ascii=False,
        sort_keys=True,
        separators=(",", ":"),
    )


def example_hash(example):
    return hashlib.sha256(
        canonical_example(example).encode("utf-8")
    ).hexdigest()

Sorting hashes makes the split repeatable even if input lines are reordered. The report records the train and evaluation hashes, counts, and the number of removed exact duplicates under exact_duplicates_removed.

Run it before installing GPU libraries:

py qlora_preflight.py raw_examples.jsonl data --eval-ratio 0.2

It writes data/train.jsonl, data/eval.jsonl, and data/split-report.json.

Hash separation only catches exact duplicates. Near-duplicate prompts, paraphrased answers, repeated source documents, and multiple answers derived from the same customer conversation can still leak across the split. Group those by their source ID before splitting in a production dataset.

Review rights, privacy, and the base model card

Training data needs a documented origin and permitted use. Remove secrets and unnecessary personal information before it reaches a GPU instance, experiment tracker, checkpoint, or shared model repository.

For the base model, read its model card and license rather than assuming every Hub repository permits the same uses. Confirm whether access is gated, whether redistribution is allowed, which chat template the tokenizer expects, and which languages and domains the model card says were evaluated.

⚠️ A public dataset or model repository is not automatically cleared for every commercial or personal-data use. Record the dataset source, license, review owner, removal decisions, and base-model revision beside each run.

Install in a fresh environment and capture versions

Create a separate environment, then install the current training stack:

py -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install --upgrade torch transformers datasets accelerate peft trl bitsandbytes
python -m pip freeze > environment-lock.txt

Compatibility changes across PyTorch, CUDA, Transformers, TRL, PEFT, Accelerate, and bitsandbytes. An unversioned installation command is useful for a new experiment, but environment-lock.txt is what lets you record the environment that actually ran.

Before training, check nvidia-smi, the CUDA version reported by PyTorch, the GPU name and memory, free disk space, and whether the selected bitsandbytes backend supports the device.

Configure 4-bit loading and LoRA explicitly

The current Hugging Face documentation exposes quantization through BitsAndBytesConfig. For training a 4-bit base, it recommends NF4; PEFT’s quantization guide shows targeting all linear layers for QLoRA-style adaptation.

use_bf16 = torch.cuda.is_bf16_supported()

quantization = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16 if use_bf16 else torch.float16,
)

adapter = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules="all-linear",
)

These are starting settings, not an optimal recipe. Adapter rank, alpha, dropout, target modules, and learning rate should be evaluated for the selected model and task.

Refuse a model without a reviewed chat template

The tokenizer’s chat template turns message objects into the tokens seen during training. If training uses one format and inference uses another, the model can receive different role markers and end-of-sequence behavior.

The reference script stops if the tokenizer has no template:

tokenizer = AutoTokenizer.from_pretrained(model_id)
if tokenizer.chat_template is None:
    raise RuntimeError(
        "The selected tokenizer has no chat template; define and review one first"
    )
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

Do not paste a template from a different model just to make the exception disappear. Use the model’s documented format or define one deliberately, then test a rendered example and verify the end-of-sequence token.

Start with a short, observable training run

The reference train_qlora.py uses current SFTConfig and SFTTrainer parameters:

arguments = SFTConfig(
    output_dir=output_dir,
    max_length=max_length,
    per_device_train_batch_size=1,
    per_device_eval_batch_size=1,
    gradient_accumulation_steps=8,
    gradient_checkpointing=True,
    learning_rate=2e-4,
    num_train_epochs=1,
    bf16=use_bf16,
    fp16=not use_bf16,
    eval_strategy="steps",
    eval_steps=50,
    save_steps=50,
    save_total_limit=2,
    logging_steps=10,
    report_to="none",
)

trainer = SFTTrainer(
    model=model_id,
    args=arguments,
    train_dataset=dataset["train"],
    eval_dataset=dataset["test"],
    processing_class=tokenizer,
    peft_config=adapter,
    quantization_config=quantization,
)

Set the model and data paths through environment variables:

$env:MODEL_ID = "the exact Hub model ID you reviewed"
$env:DATA_DIR = "data"
$env:OUTPUT_DIR = "outputs/adapter"
$env:MAX_LENGTH = "1024"
py train_qlora.py

Begin with a tiny reviewed subset or a small max_steps smoke configuration. Watch peak GPU memory, throughput, training and evaluation loss, checkpoint size, and a fixed set of qualitative prompts. Stop on non-finite loss, unexpected truncation, missing evaluation, or repeated out-of-memory failures.

Reducing batch size may not be enough. Sequence length often dominates activation memory, and merging an adapter into an unquantized base can require substantially more memory than training the adapter. Plan merging and inference as separate capacity tests.

Test what can be tested without a GPU

The preflight test suite runs with Python’s standard library:

py -m unittest -v test_qlora_preflight.py

Five tests passed while preparing this guide. They cover whitespace normalization, blank answers, invalid role order, exact-duplicate removal with non-overlapping hashes, and line-numbered JSON errors. The training entry point also passed Python bytecode compilation.

Those checks do not prove that a particular model fits your GPU, that the full dependency stack imports correctly on your CUDA installation, or that fine-tuning improves the task. No live QLoRA run or quality score is claimed here.

Evaluate behavior before serving anything

Keep the base model and adapted model available, then compare them on the same held-out prompts. Record at least:

  • task-specific correctness judged by a written rubric;
  • refusal and safety behavior relevant to the application;
  • format compliance;
  • latency and peak memory;
  • regressions on general prompts; and
  • representative failures, not only successful examples.

Training loss is an optimization signal, not a deployment approval. A smaller loss can coexist with copied answers, brittle formatting, hallucinations, or degraded safety behavior.

Do not expose the training process’s minimal development server directly to the internet. Production serving needs authentication, request limits, input bounds, timeouts, monitoring, safe model serialization, and a rollback path. Adapter merging, quantization for inference, and API deployment deserve their own tested release process.

Primary documentation and research

The useful promise of QLoRA is narrower than “fine-tune any big model on one gaming GPU.” It gives you a memory-efficient adaptation method. A trustworthy workflow still needs licensed data, a clean split, a compatible model, measured hardware limits, held-out evaluation, and a deployment gate that can say no.

Labels:

Comments

Post a Comment