Datadog tests reveal Sonnet 5, Composer 2.5, and GPT 5.5 generate vulnerable code lacking security prompts

Datadog Security Labs published findings showing that AI coding agents generate insecure code when operating without specific security rules. The research focused on Sonnet 5, Composer 2.5, and GPT 5.5 to determine if pre-set rules improve output security. This affects organizations relying on these tools for software development.

What was the scope of the testing?

The researchers tested Sonnet 5, Composer 2.5, and GPT 5.5 using identical prompts across all models. They compared default mode performance against runs using Project CodeGuard, an open-source framework providing security rules for coding agents. The goal was to measure the impact of explicit security guidelines on code quality.

What vulnerabilities were identified?

In default mode without security rules, all three models produced code with insecure direct object reference vulnerabilities. This architectural blind spot allowed any authenticated session to access or annotate files belonging to other users. The vulnerability stemmed from the prompt not specifying document access controls.

How were the tests conducted?

The team used Datadog Code Security to identify software composition analysis, static application security testing, and infrastructure-as-code vulnerabilities. They also employed Claude’s code_review skill with Sonnet 5 to audit the generated code. Supply Chain Firewall was used to prevent malicious package installation during the testing process.

What does this mean for operations?

The findings suggest that relying on default AI coding agent behavior introduces significant security risks. Organizations must evaluate whether their current workflows include explicit security constraints. The nondeterministic nature of AI means results may vary, but the baseline risk remains high without intervention.

Questions to ask your suppliers

Security teams should ask vendors whether their AI coding tools enforce security rules by default or require manual configuration. Determine who within the organization is responsible for defining and updating these security guidelines. Check if the supplier provides visibility into how the model handles access control and data privacy. Ensure that application security teams are involved in reviewing the output of these tools before deployment.

What to do and how to stay safe: Datadog

• Review current AI coding agent configurations to ensure security rules are explicitly defined.
• Audit generated code for insecure direct object reference vulnerabilities and similar issues.
• Implement static application security testing to catch vulnerabilities before deployment.
• Train developers on the importance of specifying access controls in prompts.

Step-by-step guide: Browser Security Settings for Small Business Teams

General security guidance from the TechSec Feed newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Which AI models were tested in this study?

The study tested Sonnet 5, Composer 2.5, and GPT 5.5 for secure code output.

What vulnerability was found in the default mode?

All models produced insecure direct object reference vulnerabilities in default mode.

Did the study confirm a fix for these issues?

The study evaluated the impact of security rules but did not confirm a permanent fix.