Overview
IBM Technology explains the evolution from standalone AI models to compound AI systems and AI agents. This is essential context for understanding modern AI architectures — and their security implications.
Key Takeaways
The Problem with Monolithic Models
Standalone LLMs are limited to their training data. They can't answer questions like "how many vacation days do I have?" because they lack access to private databases. This limitation drives the need for more sophisticated architectures.
Compound AI Systems
These systems integrate models with external tools (databases, APIs, calculators) to solve complex problems. Instead of training a model on everything, you connect it to the right resources. More modular, more adaptable.
Programmatic vs. Agentic Control
- Programmatic — Humans define the control logic. The system follows a predefined path. Works well for narrow, well-defined problems but breaks when queries deviate from expected patterns.
- Agentic — The LLM controls the logic. It can reason, break down problems, plan, and adjust based on what it learns. More flexible but less predictable.
Three Components of AI Agents
- Reasoning — The LLM plans and strategizes to solve problems
- Acting (Tools) — External tools execute parts of the plan: web search, calculators, APIs, other models
- Memory — Access to past interactions and internal thought processes for personalization and improved problem-solving
The ReAct Framework
ReAct (Reasoning and Acting) is a popular agent configuration pattern:
- Receive a query
- Plan an approach
- Call external tools to act
- Observe results
- Iterate until a final answer is achieved
When to Use Each Approach
- Programmatic systems — Efficient for narrow, well-defined problems with predictable queries
- Agentic systems — Better for complex tasks with varied queries where the system needs to explore multiple solution paths
The video concludes that compound AI systems will become increasingly agentic, with humans in the loop for accuracy.
Practitioner Notes
If you're in healthcare security, this video is foundational to understanding your emerging risk surface:
Tool access is where the security story changes
When IBM describes agents using "external tools (web search, calculators, other language models, APIs)," that's your attack surface expanding. Every tool an agent can access is a potential vector. In healthcare, those tools might connect to EHRs, scheduling systems, or patient databases. The agent's permissions become your security boundary.
Memory creates persistence — and risk
Agent memory means conversation history and context persist across interactions. For healthcare, that raises questions: What's being stored? Where? For how long? Is PHI ending up in agent memory? This is a data governance issue that most organizations haven't addressed yet.
The ReAct loop is where prompt injection lives
That iterative cycle — plan, act, observe, adjust — is exactly where prompt injection attacks become dangerous. If an attacker can influence what the agent "observes" (through poisoned tool outputs or manipulated data sources), they can redirect the entire reasoning chain. This is why input validation at every step matters.
"Human in the loop" isn't automatic
The video mentions humans in the loop for accuracy. In practice, that has to be designed in — and in healthcare, it's often a regulatory requirement. Think about where approval gates need to exist in your agent workflows, especially for actions that affect patient care or access PHI.
Agentic flexibility vs. auditability
Programmatic systems are predictable and auditable — you know exactly what path they'll take. Agentic systems are flexible but harder to audit. When a compliance officer asks "why did the AI do that?", you need to be able to answer. Agent observability and logging become critical.
Continue Learning
This is the third resource in the AI Foundations learning path.