Do better prompts lead to more secure AI-written code?

A published user study found that participants who trusted an AI assistant less and engaged more with their prompts, for example by rephrasing them, produced code with fewer vulnerabilities. That points to active prompting and skepticism helping. It is one study with one older model, so use it as a hint, not a rule.

What the study reported

The paper, "Do Users Write More Insecure Code with AI Assistants?", reports four findings in its abstract. Participants with access to an assistant built on OpenAI's codex-davinci-002 wrote significantly less secure code than those without access. They were also more likely to believe their code was secure. Participants who trusted the AI less and engaged more with the language and format of their prompts, such as rephrasing or adjusting temperature, wrote code with fewer security vulnerabilities. The authors also released their interface for similar studies.

What it suggests

Two behaviors seem to matter: doubt and iteration. A user who accepts the first answer treats the assistant as an authority. A user who rephrases, questions, and adjusts is treating it as a draft generator.

For teams that is a cultural point as much as a technical one. Reviewers who ask "why is this safe?" tend to catch more than reviewers who ask "does this look right?".

What it does not show

• It is one study, with a particular model, tasks, and participant group. Models have changed a great deal since.
• It does not say which prompts are best. It describes engagement, not a formula.
• Correlation. People who engaged more may also have been more careful in general.

Do not quote it as proof that any specific prompting trick yields secure code. For a broader look at the evidence, see what studies show about AI assistants and code security.

Habits worth taking from it

1. State security requirements in the prompt. Say what input is untrusted and what must be checked, rather than hoping the model infers it.
2. Ask for the reasoning. "Explain how this handles malicious input" gives you something to verify.
3. Rephrase and compare. If two phrasings give different code, look closely at the difference.
4. Verify outside the model. Tests, static analysis, and review. Confidence in the answer is not evidence.
5. Review AI-written changes on purpose. Use a checklist, such as the pull request review checklist for AI-written code.

For team leads

If you roll out coding assistants, train people to treat output as a proposal. A short onboarding note, a review checklist, and a rule that the author must be able to explain the code do more than a ban or a blanket endorsement.

How to test it on your own team

You do not have to rely on a study from another group. Pick one small, security relevant task, such as a login handler or a file upload endpoint. Have two or three developers do it with an assistant, then review the results with a short checklist and count the problems found. Ask each person how many times they rephrased or challenged the assistant. A rough comparison on your own stack, even with a handful of people, tells you more about your habits than a headline number. Repeat after you change tools or guidance.

Quick check

Sources

• Do Users Write More Insecure Code with AI Assistants? (arXiv)

Last reviewed: 19 October 2026.