Semiconductor Engineering

Reducing HBM ECC Controller Overhead For AI Inference (RPI, IBM)

Researchers at Rensselaer Polytechnic Institute and IBM T.J. Watson Research Center published a technical paper titled “REACH: Controller-Managed Long-Span ECC for HBM AI Inference.” Abstract Excerpt: ...
Semiconductor Engineering

Hybrid HBM-HBF Architecture in LLM Inference (University of Oxford)

Researchers at the University of Oxford published a technical paper titled “Hardware-Managed Heterogeneous High-Bandwidth ...
InfoQ

NVIDIA Dynamo Planner Brings SLO-Driven Automation to Multi-Node LLM Inference

A monthly overview of things you need to know as an architect or aspiring architect. Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with ...
Ars Technica

OpenAI and Broadcom announce chip designed for LLM inference at scale

OpenAI, the company behind ChatGPT and Codex and the models those tools use, and Broadcom, an established silicon supplier, have announced a new chip, called Jalapeño, designed specifically for large ...
VentureBeat

DeepSeek open sources DSpark, a new framework to speed up LLM inference by up to 85%

Even as the geopolitical conversation around AI continues to grow more fraught following the U.S. government's actions to limit the new models from Anthropic and OpenAI, Chinese open source darling ...

Gimlet Labs nabs $300M for its disaggregated inference platform

Gimlet Labs Inc., a startup that helps developers speed up their inference workloads, has raised $300 million in funding at a ...