Blog Init

Principles of Building AI Agents: Prompting A Large Language Model (LLM)

This post is for engineers and builders who want to move beyond “prompt hacking” and actually understand how modern LLMs think, reason, and respond.

Where Sam Bhagwat’s Principles of Building AI Agents [0] meets my own curiosity. This blog is Part 1 of 8 Part Blog series on AI where I share my key takeaways from the book with external resources, examples and my personal analysis.

By the end of this post, you’ll know:

  1. How today’s LLMs evolved and why transformers changed everything
  2. How to choose the right model (not just the biggest one)
  3. Why reasoning models behave differently and how to prompt them properly
  4. Practical prompting techniques that actually work in production

A Brief History

2000s: Rules, Search, and Narrow Intelligence

2010s: Deep Learning Takes Over

Later on in 2017, 8 researchers from Google came up with the renowned research paper - Attention Is All You Need [1] this is what started the modern LLM and AI revolution. This paper introduced the Transformer Architecture [2] for generating a text where a large language model (LLM) was given a set of “tokens” and its task was to predict the next “token”.

This led to models like GPT and BERT which later influenced modern systems such as Google's Gemini family. The transformer architecture also become the standard for not just text generation but also for image generation (Vision Transformers).

Choosing A Provider and Model

When you start to work on building an AI application one of the first choices you will make is which model to use.

1. Hosted vs Open-Source

When building AI Applications start with hosted providers like OpenAI, Anthropic, Google Gemini. This will help you hit the ground up and running faster and later on you can switch the provider before deploying.

2. Model Size: Accuracy vs cost/latency

When building agents, choosing the right model size involves a critical tradeoff between accuracy and operational costs. For a detailed comparison of current State of the Art (SoTA) versus efficient models across major providers like OpenAI, Google, and Anthropic, see Appendix A: Model Size Comparison Table. SoTA models are more expensive, accurate and often slower, LLM providers also provide smaller models, which are faster, cheaper but less accurate.

3. Context Window Size

Context Window is a LLM’s working memory. It signifies how many tokens can a model have. As per rough estimates 1M tokens means ~750k words which is around 2k-2.5k pages of a textbook. Larger the context size of a model, more is its capability to retain information and not lose context in larger conversations.

The models with larger context windows are capable of sustaining long-form conversations by retaining the context, being able to reason with users, and respond more consistently without hallucinations.

4. Reasoning Models and Techniques

Reasoning models as the name suggests are the ones which have the persona to logic internally before giving a response. In most of the cases reasoning models are ideal for planning and research tasks. These models take seconds, or minutes to give a response depending on the complexity of the task and will return a response all at once.

Latest reasoning models are getting faster without sacrificing on the accuracy and are actually able to think through the task in logical steps almost like a human. Reasoning models often use chain-of-thought internally.

Chain of Thought (CoT)[4] enables LLMs to tackle complex arithmetic, common sense and symbolic reasoning tasks. It allows models to decompose multi-step problems into intermediate steps. This technique encourages models to generate intermediate reasoning steps, while reasoning models are trained to internalize and optimize such multi-step reasoning, often without explicitly exposing it due to safety, IP protection, preventing internal prompt leakage reasons.

Newer techniques like Chain of Draft[5] replaces verbose, step-by-step explanations used in CoT with concise intermediate drafts which capture only the essential reasoning needed to solve a problem. This is very similar to how humans go about reasoning tasks, using short mental models instead of full explanations. Due to the CoD’s focus on minimal but informative reasoning traces it dramatically reduces verbosity and CoD is able to achieve similar or better accuracy than CoT, while using as little as ~7.6% of the tokens, this ultimately leads to lower latency and cost.

Chain of Preference Optimization[6] (CPO) is a fine-tuning method that teaches a model to prefer better reasoning steps by learning from Tree-of-Thought (ToT) search results. CoT generates a single reasoning path. While this is fast and cheap, but at times this reasoning path is often not optimal. On the other hand, ToT explores many possible reasoning paths using search (branching, backtracking, evaluation) to find better reasoning paths but this is very expensive to run at inference time. Instead of running ToT at inference time, ToT is ran offline, good vs bad reasoning paths are collected. Model is fine tuned so that its Chain-of-Thought naturally flows through the better (more optimal) paths.

Aspect CoT ToT CoT + CPO
Reasoning quality Medium High High
Inference cost Low Very high Low
Training complexity Low N/A Higher (offline)
Production-ready ✅ ❌ ✅

Example

tl;dr -

The latest SoTA models like GPT 5.2, Gemini 3 Pro, Opus 4.5 have shown that if you give them enough context and good examples, they are capable of delivering super smart, high quality answers to tough questions. It is a well documented fact that more you help the LLM by providing them the right context up-front, better their reasoning gets.

Reasoning models are like report generators**[7]**, give them lots of context through multi-shot prompting[8]. By providing a few well-crafted examples in your prompt, you can dramatically improve the accuracy, consistency, and quality of LLMs.

The way you are supposed to work with these reasoning models is by not instructing them on the how, only on the what. Then let reasoning model take over and plan and resolve its own steps. This is what autonomous reasoning is for, and can actually be much faster than if you were to manually review and chat as the human in the loop with reasoning models.

Image Credit: o1 isn’t a chat model blog post by swyx [7]

Providers and Models (Jan 2026)

Writing Great Prompts

Writing good prompts is one of the most foundational skill which you will need to master in order to leverage the true potential of these frontier models. LLMs will follow instructions, only when you know how to specify them well.

Give LLMs more examples

There are three basic techniques to prompting [9]

  1. Zero-Shot: The “YOLO” approach. The simplest form of prompting, no examples are provided and the model must rely entirely on its pre-trained knowledge.

Example

**Input**

Classify the sentiment of the following text as positive, negative, or neutral.
Text: I think the vacation was okay.
Sentiment:

**AI Output**

Neutral
  1. One-Shot: Ask a question then provide one example with input and output to guide the model.

Example

**Input**

Classify the sentiment of the following text as positive, negative, or neutral.
Text: I think the vacation was okay.
Sentiment:

**AI Output**

Neutral
  1. Few-Shot: Give multiple examples for more precise control over the output. This helps the model recognize patterns and handle more complex tasks. With more examples, the model gains a better understanding of the task leading to improved accuracy and consistency

Example

**Input**

Classify the sentiment of the following text as positive, negative, or neutral.

Text: The product is terrible. Sentiment: Negative

Text: Super helpful, worth it Sentiment: Positive

Text: It doesnt work! Sentiment:

**AI Output**

Negative

Prompting Rules of Thumb

A “seed crystal” approach to prompting

At times when you are starting to work on a new problem, you might not be sure how to start. In such scenarios you can ask the LLM to generate a prompt for you.

E.g.

You are an expert prompt engineer.

Here is an example of a high-quality prompt I like:

---
Prompt:
"You are a senior backend engineer. Explain database indexing to a junior developer using a real-world analogy, then give one production pitfall."
---

Using the same clarity, structure, and specificity, generate a new prompt that asks ChatGPT to explain how a rate limiter works for a system design interview.

Typically we should ask the model to generate the prompt, where we intend to run the prompt. Claude is best at generating prompts for Claude, Gemini is best for generating prompts for Gemini and so on.

Using the System Prompt

When we access models through API, the models have the ability to set a system prompt. E.g. give the model characteristics that you want it to have, you are defining the persona of your responses by setting system prompt. This will be in addition to the specific “user prompt” that gets passed in. Note that doing so, usually won’t help you improve the accuracy of the responses you get, just the persona or tone in which you get the responses will be changed.

Example

You are a staff software engineer. Your task is to critique the architecture and code that is presented to you from software engineering first principles and best practices.

Weird Formatting Tricks

AI models can be sensitive to formatting, this is how you can use these to your advantage.

Here is an example of a great prompt.

In Part 2, we’ll move from prompts to agents:

Thanks a lot if you read till here.

Appendix A: Model Size Comparison Table

References

  1. Principles of Building AI Agents - 2nd Edition - Sam Bhagwat - Link
  2. Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017). Link
  3. Transformer Model Visually Explained - Link
  4. Leading AI Models: Context Window Comparison 2026 - Link
  5. Wei, Jason, et al. "Chain-of-thought prompting elicits reasoning in large language models." Advances in neural information processing systems 35 (2022): 24824-24837. Link
  6. Xu, Silei, et al. "Chain of draft: Thinking faster by writing less." arXiv preprint arXiv:2502.18600 (2025). Link
  7. Zhang, Xuan, et al. "Chain of preference optimization: Improving chain-of-thought reasoning in llms." Advances in Neural Information Processing Systems 37 (2024): 333-356. Link
  8. “o1 isn’t a chat model” - Ben Hylak - Link
  9. Multi-Shot Prompting to Guide Claude’s Behavior by Anthropic - Link
  10. Examples in Prompts: From Zero-Shot to Few-Shot - Link