Key points
- Datadog Security Labs tested three major AI coding models using identical prompts.
- All three models produced code containing insecure direct object reference vulnerabilities.
- The study examined whether pre-set security rules improve the safety of generated code.
Datadog Security Labs found that three leading AI coding agents generated insecure code when tested without explicit security instructions. The research team evaluated Sonnet 5, Composer 2.5, and GPT 5.5 to determine if standard prompts produce safe software. According to the lab, all three models created applications with significant access control flaws. This finding highlights the ongoing challenge of relying on artificial intelligence for secure software development without additional safeguards.
How it unfolded
- Researchers at Datadog Security Labs designed an experiment to test whether pre-set security rules improve code safety.
- The team used Project CodeGuard, an open-source framework, to provide security guidelines for the coding agents.
- They ran Sonnet 5, Composer 2.5, and GPT 5.5 using identical prompts across different platforms like Claude, Cursor, and Codex.
- The study compared outputs from default mode against runs with specific security constraints applied to the models.
- Datadog Code Security tools analyzed the results for static application security testing and software composition analysis issues.
Who is affected
Organizations using AI coding agents to develop software are directly impacted by these findings. System administrators and security teams must understand that default model behaviors may not prioritize application security. The study specifically identified insecure direct object reference vulnerabilities in the generated code. This means authenticated users could potentially access or modify files belonging to other users. Any team relying on these models without custom security rules faces similar risks in their development pipelines.
What happens next
Datadog Security Labs plans to continue this research series over the coming months. The initial focus was on comparing plan mode to default mode for secure coding output. This second phase examined the impact of adding specific security rules to the prompt. The lab acknowledged that AI outputs are nondeterministic, meaning results may vary between runs. They used Supply Chain Firewall to protect their testing environment from malicious package installations during the experiment.
The background in plain language
Teams usually need to define clear security boundaries when using AI tools for development. Coding agents do not inherently understand access control requirements unless explicitly instructed. Without specific rules, models may generate code that allows any logged-in user to view or edit data. Security frameworks help translate complex application security concepts into simple instructions for these tools. Administrators should verify that their development processes include mandatory security checks for AI-generated code.
Background: Static application security testing (SAST)
Static application security testing scans your source code for vulnerabilities without running the application. For small teams, it reduces risk by finding errors early. Use open-source tools integrated into your build pipeline and delegate complex logic reviews to specialized experts.
Read the full guide: Static Application Security Testing for Small Teams: A Practical Framework
What to do and how to stay safe: Datadog
- Review current AI coding policies to ensure they require explicit security instructions for access control.
- Test generated code with static application security testing tools before deployment to catch common vulnerabilities.
- Implement software composition analysis to identify insecure dependencies introduced by AI coding agents.
- Monitor for insecure direct object reference flaws in applications built with artificial intelligence assistance.
Step-by-step guide: Mobile Device Security for Small Business: Control Without Cost
General security guidance from the Circuit Observer newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Which AI models were tested in this study?
Datadog Security Labs tested Sonnet 5, Composer 2.5, and GPT 5.5 using identical prompts and security parameters.
What specific vulnerability did the models produce?
All three models generated code containing insecure direct object reference vulnerabilities, allowing unauthorized access to files.
Did the study find that security rules improve code safety?
The study aimed to determine the impact of pre-set security rules, but the provided text focuses on the baseline insecure results.
