iCreatAI News
Skip to content
Pydantic AI Articles· Michael Pfaffenberger·· 18 days agoFeaturedAI score74

Code mode: make your agents 2x faster and 2x cheaper on long tasks

Code mode: make your agents 2x faster and 2x cheaper on long tasks

AI summary

A benchmark of 5,276 graded runs across eight models on five small Pydantic AI apps found that code mode, where the model writes a program that calls tools inside a sandbox, cut cost by 58% and wall time by 51% on long tasks while raising success by 16 points. Gains grew with task length: at 200 items code mode was 47% to 75% cheaper and 42% to 67% faster, whereas at 5 items it cost 21% to 38% more on three of five tasks. The author attributes the win not to fewer round trips, since models already batch calls, but to less content per request, and runs the model's code in Monty, a Rust Python interpreter with no host access by default and 30-second, 256 MiB limits.

Why it matters

Offers a detailed, methodology-heavy benchmark with per-model cost, latency and correctness numbers, plus an honest list of where code mode costs more.

Source: Pydantic AI Articles · pydantic.dev