Datadog tests show AI agents still write insecure code without security rules
Research by Datadog Security Labs finds three major AI coding models produce vulnerable code unless teams explicitly provide pre-set security guidelines.
Key points
- Sonnet 5, Composer 2.5, and GPT 5.5 all produced code with an insecure direct object reference vulnerability when tested without specific security rules.
- The study used Project CodeGuard, an open-source framework, to provide security guidelines to the AI agents during the testing process.
- Datadog Security Labs conducted this research to measure whether pre-set security rules significantly improve the security output of AI coding agents.
Datadog Security Labs found that three popular AI coding models still generate vulnerable code when teams do not provide them with explicit security rules. The research highlights a persistent risk in relying on artificial intelligence for software development without additional safeguards.
The risk
The study tested Sonnet 5, Composer 2.5, and GPT 5.5 using identical prompts to see if pre-set security rules improved code safety. Without these rules, all three models produced code containing an insecure direct object reference vulnerability. This flaw allows any authenticated user to access or modify files belonging to other users. The researchers noted this was a recurring architectural blind spot across all iterations in default mode.
Who is exposed
Software developers and engineering teams using AI coding agents are at risk if they rely on default model settings. The finding applies to users of Claude Code, Cursor, and Codex, which correspond to the tested models. Teams that do not implement specific security guidelines or frameworks for their AI tools may inadvertently introduce access control flaws into their applications.
Cost and compliance
The source material does not provide specific data on financial costs or regulatory compliance impacts associated with these vulnerabilities. The research focuses on the technical presence of security flaws rather than their business consequences. No monetary figures or compliance framework violations are mentioned in the Datadog Security Labs report.
What should you do?
Teams should review their AI coding agent configurations to ensure security rules are actively applied. Developers should not assume default model settings provide adequate security protections for production code. Implementing established security frameworks can help guide AI agents toward safer coding practices.
The background in plain language
AI coding agents generate code based on user prompts and training data. Without explicit instructions, they may overlook security best practices like access controls. Security rules act as constraints that force the AI to consider safety requirements. Teams usually need to define these rules clearly to prevent common vulnerabilities. Check whether your development workflow includes mandatory security guidelines for AI-generated code.
Background: AI agents
AI agents are software systems that perceive their environment, make decisions, and execute actions to achieve specific goals. Unlike standard models, they maintain internal state and interact with external tools, creating unique security risks around tool execution, memory retention, and autonomous decision loops that require specialized controls.
Read the full guide: AI Agents Explained: Answers to Security and Architecture Questions
What to do and how to stay safe: Datadog
- Implement pre-set security rules or frameworks like Project CodeGuard for AI coding agents.
- Review AI-generated code for access control flaws, especially insecure direct object references.
- Use static application security testing tools to audit code produced by AI agents.
- Ensure your development environment includes protections against malicious package installations.
Step-by-step guide: Browser security settings are the architectural walls of your digital life
General security guidance from the Vector Tribune newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Which AI coding models were tested in the Datadog study?
The study tested Sonnet 5, Composer 2.5, and GPT 5.5 using corresponding tools like Claude Code, Cursor, and Codex.
What vulnerability did the models produce without security rules?
All three models produced code with an insecure direct object reference vulnerability, allowing unauthorized file access.
Did the study find that security rules improved code safety?
The study aimed to measure the impact of security rules, noting that default modes produced vulnerable code without them.
