NLP & LLM Interview Questions

BERT (encoder-only): bidirectional, reads full sequence at once. Pretrained with Masked Language Modeling. Best for: classification, NER, QA, embeddings. GPT (decoder-only): left-to-right, causal attention mask. Pretrained with next-token prediction. Best for: text generation, summarization, chat. BERT knows context from both directions → richer representations for understanding. GPT generates autoregressively → natural for open-ended generation.

RLHF (Reinforcement Learning from Human Feedback) aligns LLMs with human preferences in 3 steps: 1. SFT (Supervised Fine-Tuning): fine-tune base LLM on human-written demonstrations. 2. Reward Model: humans rank K model outputs; train a reward model to predict preferences. 3. PPO: optimize SFT model to maximize reward model score + KL penalty to stay close to SFT baseline. DPO (Direct Preference Optimization) skips the reward model — directly optimizes on preference pairs. Simpler, stable, now preferred over PPO.

Hallucination: generating confident but false information. Mitigated by RAG, grounding, chain-of-thought. Knowledge cutoff: can't access events after training data cutoff. Mitigated by RAG, tool use (web search). Context window limits: can't process arbitrarily long documents. Mitigated by chunking, summarization, hierarchical retrieval. Prompt injection: malicious inputs override system prompt instructions. Mitigated by input validation, sandboxing. Sycophancy: model agrees with user even when wrong. Mitigated by adversarial RLHF, debate techniques.

RAG (Retrieval-Augmented Generation): retrieve relevant docs from a vector DB, inject into prompt. + No training cost, instantly updatable, verifiable (can cite sources) − Adds retrieval latency, limited by context window, requires good retrieval Fine-tuning: bake knowledge into model weights. + No retrieval overhead, can change model style/behavior − Expensive, hard to update, still hallucinates Use RAG when: knowledge changes frequently, answers need citations, private/enterprise data. Use fine-tuning when: changing response format/tone, specific domain vocabulary, latency-critical.

LoRA (Low-Rank Adaptation) fine-tunes LLMs by injecting trainable low-rank matrices into existing layers: W' = W₀ + ΔW = W₀ + B·A Where A ∈ R^{r×d}, B ∈ R^{d×r}, r << d (rank, e.g., 8 or 16) Only A and B are trained — typically 0.1-1% of original parameters. Achieves results comparable to full fine-tuning at a tiny fraction of the compute cost. QLoRA extends this with 4-bit quantization of the frozen base model — enables fine-tuning 7B-70B models on a single consumer GPU.