OpenAI Unveils Jalapeño AI Chip Benchmarks at Hot Chips Conference
At the Hot Chips conference on Tuesday, OpenAI shared a detailed presentation on Jalapeño, unveiling the initial benchmark metrics for its dedicated artificial intelligence inference system. Tested against leading hardware on the Semianalysis InferenceX benchmark, the processor demonstrated higher throughput per kilowatt alongside higher token generation rates per user.
What Happened
OpenAI revealed performance data comparing its custom Jalapeño processor against existing top-tier inference processors, notably an Nvidia Blackwell system. According to findings tested on the InferenceX benchmark from Semianalysis, Jalapeño surpassed current state-of-the-art systems in energy efficiency and per-user output.
Speaking on a press call, Richard Ho, the head of hardware at OpenAI, described the metrics as a notable progression over current industry baselines. Ho stated that the chip is engineered to process more artificial intelligence tasks per unit of electrical power while delivering faster response times, providing high capacity for concurrent users alongside low processing latency.
Key Highlights
- Performance Metrics: Outperformed current state-of-the-art processors on Semianalysis’s InferenceX benchmark, registering superior throughput per kilowatt and more tokens per user.
- Comparative Baseline: Evaluated directly against an Nvidia Blackwell setup.
- Development Partnership: Built in close partnership with Broadcom, with assistance provided by OpenAI’s own AI models during development.
- Platform Scope: Designed as a multigenerational hardware initiative aimed at aligning models, chips, software products, and memory development.
- Bottleneck Reduction: Specifically architected to curb latency across communication and prefill phases of inference.
Why This Matters
Inference processing often encounters performance hurdles during specific operational stages. OpenAI explained that Jalapeño utilizes a full-stack design to directly target phases that traditionally slow down processing, particularly data communication and the prefill stage.
In a blog post detailing the architecture, OpenAI noted that the system is engineered to restrict data transfer and reduce communication lag. This design keeps the model state—including the KV cache utilized during token generation—stored locally while engaging the appropriate configuration of networking, memory, and compute resources for each phase of inference.
What to Watch Next
OpenAI originally announced the Jalapeño project in October. Following this technical presentation, the rollout schedule points toward a gradual multi-year implementation.
Richard Ho indicated that the processor is projected to begin deployment in limited volumes toward the end of 2026. A broader, more substantial rollout of the hardware platform is anticipated to follow in 2027, during which time competing hardware designs in the market may also evolve.
Frequently Asked Questions
What is OpenAI Jalapeño?
Jalapeño is a specialized artificial intelligence processor developed by OpenAI in collaboration with Broadcom, specifically designed for running AI model inference at scale.
What did the benchmark tests show?
According to results from Semianalysis’s InferenceX benchmark, Jalapeño produced more tokens per user and higher operational throughput per kilowatt compared to an Nvidia Blackwell system.
When will Jalapeño be deployed?
According to OpenAI hardware lead Richard Ho, Jalapeño is scheduled for very small volume deployment by late 2026, followed by a larger deployment phase in 2027.
Source: TechCrunch reporting on OpenAI statements at the Hot Chips conference.