Meta has deployed an AI agent that automatically optimizes the hidden layer of code running AI models on diverse hardware, achieving 60% faster inference speeds on its ads ranking system in hours instead of weeks of manual engineering work. The system, called KernelEvolve, represents a fundamental shift in how companies approach the bottleneck that's been quietly holding back AI infrastructure scaling .

What's the Real Problem That KernelEvolve Solves?

Behind every AI model running on a server lies a layer of highly optimized low-level code called kernels. These are small programs that translate high-level model operations into instructions a specific chip can execute efficiently. Think of them as the translator between what a programmer writes and what a graphics processing unit (GPU), custom chip, or processor actually understands .

The problem has become exponential. Meta now operates a diverse hardware fleet including NVIDIA GPUs, AMD GPUs, Meta's custom MTIA silicon chips, and CPUs. Each hardware type has different memory architectures, instruction sets, and execution models. A kernel optimized for one platform may perform poorly or fail entirely on another. Even within a single hardware family, successive generations introduce architectural changes that require different optimization strategies .

The total number of kernels Meta needs scales with the product of three factors: hardware types and generations multiplied by model architectures multiplied by the number of operators. This creates thousands of unique kernel configurations that must be written, tested, and maintained. Hand-tuning each kernel by human experts simply doesn't scale anymore .

How Does KernelEvolve Actually Work?

Unlike typical AI systems that generate code once and call it done, KernelEvolve treats kernel optimization as a continuous search problem. It explores hundreds of alternative kernel implementations to identify solutions that often match or exceed human expert performance. A purpose-built job harness evaluates each candidate kernel, feeds diagnostics back to the large language model (LLM), and drives a continuous search over hundreds of alternatives .

The results speak for themselves. In Meta's production environment, KernelEvolve improved ads model inference throughput by over 60% on NVIDIA GPUs and achieved over 25% training throughput improvement for an ads model on Meta's custom MTIA silicon chips. What would have taken human kernel experts weeks of profiling, optimizing, and cross-hardware debugging was compressed into hours of automated search and evaluation .

Why This Matters for the Entire AI Industry

KernelEvolve optimizes across public and proprietary hardware, generating kernels in high-level domain-specific languages (DSLs) like Triton, Cute DSL, and FlyDSL, as well as low-level languages including CUDA, HIP, and MTIA C++. This broad applicability means the approach isn't limited to Meta's specific hardware stack .

The timing is critical. As AI models grow more complex and the hardware landscape diversifies, the kernel optimization bottleneck has become one of the most significant constraints on hardware enablement and performance tuning. This directly slows model iteration cycles that drive advances in machine learning technology and its real-world applications. By automating this process, Meta is removing a major friction point in the AI development pipeline .

How Companies Can Benefit From Automated Kernel Optimization

• Faster Hardware Integration: New GPU generations and custom silicon chips can be integrated into production systems in days rather than months, since kernel optimization no longer requires weeks of expert engineering time.
• Reduced Engineering Bottlenecks: Kernel experts are freed from repetitive optimization work to focus on architectural innovations and novel operator designs that push AI capabilities forward.
• Improved Model Performance: Automated search discovers optimization strategies that human engineers might miss, often exceeding the performance of manually tuned kernels across diverse hardware platforms.
• Cost Efficiency at Scale: Faster inference and training throughput directly reduces the computational resources required to serve billions of daily AI requests, lowering infrastructure costs.
• Hardware Flexibility: Companies can maintain diverse hardware portfolios without the engineering overhead of manually optimizing kernels for each platform and generation.

Meta's approach represents a fundamental shift in how the industry thinks about the relationship between AI software and hardware. Where kernel development was once a manual, expert-driven process that struggled to keep pace with hardware and model evolution, KernelEvolve makes it continuous and automated, adapting as each changes .

The system is already optimizing code that serves trillions of daily inference requests in Meta's production environment. As Meta continues to diversify its AI hardware portfolio, the ability to rapidly generate optimized kernels for new chips substantially reduces the engineering effort required to integrate heterogeneous hardware for both training and inference .

More technical details are available in the paper "KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta," which will appear at the 53rd International Symposium on Computer Architecture (ISCA) 2026 .