Serving Prefill and Decode for Agentic AI (On Tenstorrent Blackhole)

Tenstorrent describes how its Galaxy systems, built on Blackhole chips, serve both prefill and decode phases on the same hardware so operators can shift capacity between them as agentic workloads change. The post explains how agent loops produce growing, largely cacheable context, models how faster decode only partly shortens dependent-call latency, and contrasts prefill's compute-bound long prompts with decode's memory-bound small batches. It cites a Wormhole FlashAttention kernel comparison (9 to 44 times faster than a basic implementation) as a measured example of data-placement control.