Function Response Poisoning

Function Response Poisoning is an attack technique in which an adversary manipulates the output returned by an AI function or external service, causing the language model to trust malicious, inaccurate, or misleading data when generating responses or executing downstream actions.

AI assistants increasingly depend on structured function calls to retrieve data from enterprise applications, APIs, and external services. Function Response Poisoning targets the data returned by these trusted functions rather than the function call itself.

By compromising or manipulating function responses, attackers can influence AI reasoning, inject false information, trigger unauthorized workflows, or deceive users into making incorrect decisions.

Modern AI systems assume that trusted function responses are accurate and authoritative. This assumption creates an opportunity for attackers to manipulate returned data through compromised APIs, vulnerable middleware, malicious plugins, or intercepted communications.

Because the AI model treats the response as legitimate context, poisoned outputs can alter future reasoning, affect business decisions, or trigger automated enterprise actions.

Unlike Function Parameter Injection, which targets inputs, Function Response Poisoning targets the trusted outputs consumed by the AI.

1. The AI invokes a legitimate enterprise function.
2. An attacker compromises or manipulates the returned response.
3. The AI accepts the response as trusted information.
4. The poisoned data influences model reasoning.
5. Users or automated workflows act on inaccurate information.

• AI Function Calling APIs
• Enterprise AI Assistants
• Business Automation Platforms
• CRM Systems
• ERP Platforms
• Internal Enterprise APIs
• AI Agents

• Manipulated AI decision making
• Distribution of false or misleading information
• Unauthorized workflow execution
• Financial loss due to incorrect automation
• Reduced trust in AI-assisted operations
• Compliance and audit concerns

Redbot Security evaluates Function Response Poisoning by introducing controlled modifications to trusted function outputs during enterprise AI security assessments.

Testing includes:

• Response integrity validation
• API output manipulation
• Business logic verification
• Trust boundary testing
• Workflow integrity analysis
• AI decision consistency evaluation

Monitor for:

• Unexpected changes in function responses
• Inconsistent API output
• Data integrity failures
• AI decisions conflicting with authoritative records
• Abnormal workflow behavior
• Unexpected downstream automation

AML.T0011 – AI System Abuse AML.T0051 – Prompt Injection (supporting technique where applicable)

LLM05 – Supply Chain Vulnerabilities LLM06 – Excessive Agency LLM09 – Misinformation