Key Takeaways
- AI agent = LLM + tools + loop. LLM decides which tool to use, executes, observes result, decides next action.
- LangGraph is a framework for building agentic workflows using local or cloud LLMs.
- Key components: LLM (Ollama), tools (web search, code execution, file access), memory (conversation history), planning (reasoning loops).
- Local agents are slower than cloud (LLM reasoning takes time) but private and customizable.
- As of April 2026, local agents work best for tasks that benefit from reasoning over speed.
How Does an AI Agent Work?
An agent follows this loop: (1) observe state/context, (2) LLM reasons about best action, (3) execute action (tool call), (4) observe result, (5) repeat until done.
Example: Research agent given task "Compare Llama 3.2 vs Qwen 2.5 on coding tasks".
- Observation: Task received.
- Reasoning: Need to find benchmarks, search for HumanEval scores.
- Action: Use web_search tool to find "Llama 3.2 HumanEval benchmark".
- Observation: Retrieved text with scores.
- Action: Search for "Qwen 2.5 HumanEval".
- Reasoning: Both models found. Qwen is faster, Llama is more general.
- Final Action: Synthesize answer and return.
What Is the Difference Between Agents and Chains?
| Aspect | Chains | Agents |
|---|---|---|
| Decision-making | Predetermined sequence | Dynamic, LLM decides |
| Loops | No loops | Reasoning loop (repeat until done) |
| Error recovery | Manual error handling | LLM can recover from failures |
| Use case | Fixed workflows (summarize → email) | Complex reasoning (research, automation) |
| Complexity | Simple, predictable | Complex, unpredictable behavior |
LangGraph Architecture
LangGraph defines agents as directed acyclic graphs (DAGs) with nodes (states) and edges (transitions).
- State: Information agent holds (context, observations, decisions).
- Nodes: Functions that process state (LLM reasoning, tool execution).
- Edges: Transitions between nodes (conditional based on LLM output).
- Tools: Functions the LLM can call (web search, code execution, database queries).
What Tools Can Agents Use?
Agents are only as capable as their tools. Common tools:
- Web search: Search the internet for information (duckduckgo, Google, Bing).
- Code execution: Run Python code and return results.
- File operations: Read/write files, list directories.
- Database queries: Query local or remote databases.
- Document retrieval: Search RAG vector database for documents.
- Calculator: Perform arithmetic and symbolic math.
- Email: Send messages (with caution, verify permissions).
- API calls: Interact with external services.
How Do Agents Reason and Plan?
Agent reasoning depends on the LLM model size and prompt quality.
- Small models (3–7B): Limited reasoning. Work best with deterministic tasks (tool lookup, classification).
- Medium models (13–30B): Decent reasoning. Can handle 2–3 step reasoning chains.
- Large models (70B+): Strong reasoning. Can solve complex problems with multi-step planning.
Prompting technique: Chain-of-Thought (CoT) helps agents think through steps before deciding.
# Example: CoT reasoning prompt for agent
system_prompt = """
You are a research agent. Break complex tasks into steps:
1. Identify what information you need
2. Call appropriate tools to gather information
3. Analyze results and determine next steps
4. Return the final answer with sources
Always reason step-by-step before calling tools.
"""Common Local Agent Patterns
- Research agent: Searches documents and web, synthesizes findings.
- Code agent: Writes and executes code to solve problems.
- Planning agent: Breaks complex tasks into subtasks, delegates to other agents.
- Conversational agent: Maintains memory, answers questions, learns from feedback.
- Workflow automation: Reads emails, executes tasks, sends confirmations.
Common Agent Implementation Mistakes
- Too many tools: Agent gets confused with too many options. Limit to 5–10 relevant tools.
- Poor tool descriptions: LLM won't use tools correctly if descriptions are vague. Write clear, specific descriptions.
- Infinite loops: Agent can get stuck in reasoning loops. Add max iteration limit (e.g., 10 steps).
- No error handling: Tool calls may fail. Have agent handle failures gracefully.
- Using small models: 3B models cannot reason well enough for complex agents. Use 13B+ for autonomous agents.
Common Questions About Local Agents
How much faster are cloud agents vs local agents?
Cloud agents: ~1 sec per reasoning step. Local agents: ~3–5 sec per step (depends on model size). Local adds latency but eliminates API costs.
Can local agents access the internet?
Yes, if you provide a web_search tool. Agents can browse any publicly accessible content.
How do I ensure an agent doesn't break things (e.g., delete files)?
Use sandboxed tool environments (Docker containers). Restrict file/network access. Log all tool calls for audit trails.
Can I run multiple agents in parallel?
Yes. Use async frameworks (FastAPI) to handle concurrent agent requests. Each gets its own LLM instance.
Sources
- LangGraph Documentation — github.com/langchain-ai/langgraph
- LangChain Agents — python.langchain.com/docs/modules/agents/
- Agent Reasoning Papers — arxiv.org (search "agents reasoning")