Unit 3: Chain-of-Thought, Agents & ReAct

Chain-of-Thought (CoT) Prompting

Definition & Origin

Chain-of-Thought (CoT) prompting is a technique that encourages large language models to show their step-by-step reasoning process before arriving at a final answer. This approach was introduced by Google researchers in 2022 and has since become a fundamental strategy for improving LLM performance on complex tasks.

Implementation Strategies

Strategies: Explicit vs. Implicit Instructions

• Explicit CoT: Directly instruct the model to "think step by step" or "show your reasoning process." Example: "Solve this problem step by step."
• Implicit CoT: Embed reasoning demonstrations within the prompt without explicitly telling the model to reason. The model follows the pattern set by examples.
• Demonstrative Examples: Provide worked examples showing the intermediate reasoning steps, then ask the model to solve a similar problem following the same pattern.

Zero-Shot CoT vs. Few-Shot CoT

Benefits

• Improved Accuracy: Studies show up to 40% improvement on multi-step reasoning tasks compared to standard prompting.
• Enhanced Interpretability: The reasoning trace provides transparency into how the model arrived at its conclusion, making errors easier to identify and correct.
• Better Generalization: Models trained or prompted with CoT show improved performance on out-of-distribution problems.

Limitations

• Model-Dependent: CoT effectiveness varies significantly across model sizes — smaller models may produce coherent-looking but incorrect reasoning chains.
• Verbose Outputs: The step-by-step format increases output length, leading to higher token costs and latency.
• Error Propagation: If an early step is incorrect, the entire reasoning chain can be compromised.

graph TD
    A[Complex Question] --> B[Break into Steps]
    B --> C[Step 1: Identify Key Information]
    C --> D[Step 2: Apply Relevant Knowledge]
    D --> E[Step 3: Reason Through Logic]
    E --> F[Step 4: Draw Conclusion]
    F --> G[Final Answer]
    style A fill:#0891b2,stroke:#0e7490,color:#fff,font-weight:bold
    style G fill:#22c55e,stroke:#4ade80,color:#fff,font-weight:bold
    classDef process fill:#06b6d4,stroke:#22d3ee,color:#fff,font-size:18px,font-weight:bold
    classDef decision fill:#0891b2,stroke:#0e7490,color:#fff,font-size:18px,font-weight:bold
    classDef success fill:#22c55e,stroke:#4ade80,color:#fff,font-size:18px,font-weight:bold
                    

AI Agents — Capabilities & Applications

Core Capabilities

Applications

Challenges

• Ethics & Bias: Agents may amplify existing biases in training data or make ethically questionable decisions when operating autonomously.
• Privacy & Security: Tool access creates attack surfaces; agents handling sensitive data require robust security measures and access controls.
• Interpretability: Understanding why an agent took a particular action sequence remains difficult, especially with multi-step reasoning chains.
• Scalability: Managing hundreds or thousands of concurrent agents requires efficient resource allocation and coordination mechanisms.

Reason and Act (ReAct) Framework

Overview

The ReAct framework was proposed by Yao et al. (2022) and introduces a paradigm that interleaves reasoning traces with actions. Unlike purely reactive systems or purely deliberative ones, ReAct alternates between thinking and doing at each step.

The ReAct Flow

1. INPUT: The agent receives a question or task from the user.
2. REASONING: The model generates a thought explaining what it knows, what it needs, and what action to take next.
3. ACTION: The agent executes an action (e.g., search, calculation, API call).
4. OBSERVATION: The agent processes the result of the action and feeds it back into the reasoning loop.
5. RESPONSE: Once the goal is met, the agent produces a final answer.

Advantages

graph LR
    A[Input] --> B[Reason]
    B --> C[Act]
    C --> D[Observe]
    D --> E{Goal Met?}
    E -->|No| B
    E -->|Yes| F[Respond]
    style A fill:#0891b2,stroke:#0e7490,color:#fff,font-weight:bold
    style B fill:#6366f1,stroke:#818cf8,color:#fff,font-weight:bold
    style C fill:#0ea5e9,stroke:#38bdf8,color:#fff,font-weight:bold
    style D fill:#f59e0b,stroke:#fbbf24,color:#1e1b4b,font-weight:bold
    style F fill:#22c55e,stroke:#4ade80,color:#fff,font-weight:bold
    classDef decision fill:#0891b2,stroke:#0e7490,color:#fff,font-size:18px,font-weight:bold
    classDef process fill:#06b6d4,stroke:#22d3ee,color:#fff,font-size:18px,font-weight:bold
    classDef success fill:#22c55e,stroke:#4ade80,color:#fff,font-size:18px,font-weight:bold
    classDef warning fill:#f59e0b,stroke:#fbbf24,color:#1e1b4b,font-size:18px,font-weight:bold
    classDef info fill:#8b5cf6,stroke:#a78bfa,color:#fff,font-size:18px,font-weight:bold
                    

OpenAI Functions (LLMs as API)

API Access & HTTP Requests

OpenAI's API allows developers to interact with large language models programmatically via HTTP requests. Understanding the API architecture is essential for building reliable AI-powered applications.

Key Endpoints

Essential Parameters

Best Practices

• Security: Never expose API keys in client-side code. Use environment variables and server-side proxies. Implement rate limiting and input validation.
• Efficiency: Use streaming responses for better user experience. Batch similar requests. Cache frequent responses where appropriate.
• Error Handling: Implement retry logic with exponential backoff for rate limits (429) and server errors (500). Handle timeout gracefully.
• Monitoring: Track token usage, latency, error rates, and costs. Set up alerts for anomalous patterns or budget thresholds.

graph TD
    A[Client Application] --> B[HTTP Request]
    B --> C[API Gateway]
    C --> D[Authentication]
    D --> E[Rate Limiting]
    E --> F[Model Inference]
    F --> G[Response Formatting]
    G --> H[JSON Response]
    H --> A
    style A fill:#0891b2,stroke:#0e7490,color:#fff,font-weight:bold
    style F fill:#6366f1,stroke:#818cf8,color:#fff,font-weight:bold
    style H fill:#22c55e,stroke:#4ade80,color:#fff,font-weight:bold
    classDef process fill:#06b6d4,stroke:#22d3ee,color:#fff,font-size:18px,font-weight:bold
    classDef decision fill:#0891b2,stroke:#0e7490,color:#fff,font-size:18px,font-weight:bold
    classDef success fill:#22c55e,stroke:#4ade80,color:#fff,font-size:18px,font-weight:bold
    classDef info fill:#8b5cf6,stroke:#a78bfa,color:#fff,font-size:18px,font-weight:bold
                    

Agent Toolkits & LangChain

OpenAI Functions vs. ReAct — Comparison

Agent Toolkits

Customising Standard Agents

• prefix / suffix: Add system instructions before or after the default prompt to customize agent behavior and personality.
• max_iterations: Limit the number of reasoning-action loops to control costs and prevent infinite cycles.
• verbose: Enable detailed logging of agent thought processes for debugging and monitoring.
• callback_manager: Hook into agent lifecycle events (tool calls, errors, completions) for logging, analytics, and custom behavior.

Summary Matrix — All Frameworks