LLM Questions

 

1. Prompt Engineering

Definition:
Prompt engineering is the process of writing clear and structured instructions so the LLM generates accurate and relevant responses.

Important Points:

• A good prompt usually contains Role + Task + Context + Constraints + Expected Output.
• Better prompts generally produce better responses.

2. Temperature

Definition:
Temperature controls the randomness or creativity of the LLM's output.

Important Points:

• 0.0–0.3 → More deterministic and consistent (good for QA, coding, test cases).
• 0.4–0.7 → Balanced creativity.
• 0.8–1.0+ → More creative but less predictable (good for stories, brainstorming).
• Lower temperature = More consistent output.
• Higher temperature = More diverse output.

3. Tokens

Definition:
Tokens are the basic units of text that an LLM processes.

Important Points:

• Both input and output consume tokens.
• API cost is usually based on token usage.
• More tokens = Higher cost and slower response.

4. Tokenizer

Definition:
A tokenizer converts text into tokens before the LLM processes it.

Important Points:

• Tokenizer → Text → Tokens.
• After generation, tokens are converted back into readable text.
• Different models may use different tokenizers.

5. Context Window

Definition:
The context window is the maximum number of tokens an LLM can process in one request.

Important Points:

• Includes:
  • System Prompt
  • User Prompt
  • Conversation History
  • Retrieved RAG Chunks
  • LLM Response
• If the limit is exceeded, older content may be dropped or the request may fail.

6. Chunking

Definition:
Chunking is the process of splitting large documents into smaller pieces before creating embeddings.

Important Points:

• Improves retrieval accuracy.
• Reduces token usage.
• Common chunk size: 300–1000 tokens (depends on the application).
• Usually performed by the application using text-splitting libraries.

7. Embeddings

Definition:
Embeddings are vector representations of text that capture semantic meaning.

Important Points:

• Similar meaning → Similar vectors.
• Used for semantic search, not text generation.
• Created by an embedding model, not the LLM itself.

8. FAISS / ChromaDB

Definition:
FAISS and ChromaDB are vector databases that store embeddings and perform similarity search.

Important Points:

• They store vectors, not raw PDFs for retrieval.
• They compare vectors to find the most relevant chunks.
• They do not generate answers. 

9. Similarity Search

Definition:
Similarity search compares the user's query embedding with stored document embeddings to find the closest matches.

Important Points:

• Usually based on Cosine Similarity (or similar distance metrics).
• Searches by meaning, not exact keywords.
• Returns the closest matching chunks. 

10. Top-K Retrieval

Definition:
Top-K retrieval returns the K most relevant chunks from the vector database.

Important Points:

• Common values: K = 3–5.
• Too low → Missing context.
• Too high → Higher token usage, more cost, slower response, and more irrelevant context.

11. RAG (Retrieval-Augmented Generation)

Definition:
RAG retrieves relevant document chunks and provides them to the LLM to generate more accurate answers.

Important Points:

• Pipeline: Chunking → Embeddings → Vector DB → Similarity Search → Top-K → LLM.
• Reduces hallucinations.
• Useful for company documents, manuals, and knowledge bases.

12. Hallucinations

Definition:
Hallucination is when an LLM generates incorrect or unsupported information that sounds convincing.

Important Points:

• Happens because LLMs predict the next token rather than verify facts.
• Reduced using:
  • RAG
  • Better chunking
  • Good embedding model
  • Appropriate Top-K
  • Strong system prompts
• Cannot be eliminated completely. 

Question	One-line Answer
What creates embeddings?	Embedding Model
Who performs chunking?	Application/Text Splitter
Who stores vectors?	FAISS/ChromaDB
Who performs similarity search?	FAISS/ChromaDB
Who generates the final answer?	LLM
Who converts text into tokens?	Tokenizer
What determines API cost?	Total input + output tokens
What reduces hallucinations?	RAG + Good Prompt + Good Retrieval
Does RAG remove hallucinations completely?	No, it only reduces them.
What is a good Top-K value?	Usually 3–5.

Final Interview Mantra (Remember This Flow)

User Question

      │

      ▼

Tokenizer

      │

      ├──► Embedding Model ─► Vector ─► FAISS ─► Top-K Chunks

      │

      ▼

LLM (receives system prompt + user prompt + retrieved chunks)

      │

      ▼

Final Answer