LibTorch ABI Stable Plans 2026

1

TL;DR

  • A limited subset of stable ABIs in libtorch exist, and we encourage you to use them!

  • We plan to land some more features and tools to improve user experience soon

  • We’re providing white glove services to help migrate libraries like torchvision and vLLM this year

Context

PyTorch’s C++ bits (libtorch.so) aren’t ABI stable. This means that when PyTorch releases a new version, a library that depends on libtorch.so (like torchvision) needs a separate build to work with the new version of PyTorch. We’ve worked in the past year to lift this restriction by introducing a new limited subset of stable ABIs in libtorch. If your library uses only APIs from this stable subset, you will be ABI stable with LibTorch, meaning you can build your binary on one torch version and run on another torch version!

The following documents our goals and plans with “LibTorch ABI Stable”, or Project H-Beam as I like to refer to it. We will discuss the current status, the goals we aim to achieve in 2026, and some concrete milestones for how we plan to get there. We will not be going into the technical details in this post–for that, please take a look at our docs!

Current Status - You Can Experience It Today!

As of today (2026.05.07), LibTorch has a wide enough subset of stable APIs that a good number of modern C++ custom operator libraries can migrate to become ABI stable with LibTorch. Yes, you read that right, if you’re curious about getting onboarded, there’s nothing stopping you from looking into it today! (Check out our docs and custom ops tutorial. There are also PyTorch conference talks that may be more visual, if I say so myself ;)) In fact, several ecosystem libraries have already migrated and are ABI stable: Flash-Attn-3, xformers, torchao, torchaudio. In other words, if you’re finding yourself recompiling FA3 from source, know that you may be being silly! If you build FA3 with torch 2.9+, you only need to build the wheel once (!); that same wheel will run fine with any later torch version.

It may sound like the job is complete. We are not so confident, so we’d like to share our vision for “LibTorch ABI Stable”.

Goals & Vision

Ultimately, why we even embarked along this journey began with one motivation: we noticed a need for ABI stable libraries in the community, especially as the ecosystem spiderwebbed deeper. We wanted to help fill those gaps and create a nice experience for custom op library writers.

We started by picking a library and brainstormed what it would require to make that library ABI stable with LibTorch. We realized we had to answer three questions.

  1. Do we offer a sufficient set of ABI stable APIs that library writers need? => Feature Completion

  2. How easy is it to write a library that would be ABI stable with LibTorch? => User Experience, "self-serve"ness

  3. Which ecosystem libraries are important to make ABI stable? => Enablement

These three prongs are how we will discuss our goals and milestones going forward, so let’s define them a bit more:

Feature Completion - do we have a sufficient set of APIs supported through the C shim and torch/headeronly?

  • This includes all of what we consider “stable API” coverage: high-level C++ headers, C shims, torch/headeronly utils

  • TORCH_TARGET_VERSION - hardening of versioning logic

  • testing - e.g., sufficient testing to prevent regressions

  • build - e.g., Windows works, ROCm works

User Experience - how easy is it to onboard to become stable? How “self-serve” is the process?

  • Docs/tutorials

  • Finetuning the Claude skill(s) and any thing that can be made deterministic should be. e.g., we should have a deterministic AST-based script that would convert a library’s unstable API usages to stable, and the skill can handle the trickier parts where you need to call a different API, adapt calling code, use torch_call_dispatcher, etc.

Enablement - which libraries are we helping to onboard + will include in our roadmap?

  • Specific goals of enablement

    1. There is a need for ABI stable libraries in the community that is not getting met for whatever reason, so we are stepping in to fill those gaps
    2. The act of enabling helps us measure “self-serve”-ness. i.e., This answers “How good is our user experience?”
    3. Gauge coverage of our stable ABI surface. Once we know that migrating XYZ library is possible, then we mark ourselves as “coverage complete”.
    4. Validate the dynamic effects of the system. For example, we want to ascertain that stable libraries sitting atop other stable libraries interact properly.

  • Note we expect that points 3 and 4 above will be served by side effect from what we need to do for 1 and 2.

  • Which libraries?

    • vLLM - ecosystem library offering a collection of kernels upon which many other libraries depend on (in progress)

    • modernized torchvision - stalwart partner library to pytorch that CV use cases depend on

    • sglang - ecosystem library offering a collection of kernels upon which many other libraries depend on (we are still exploring)

We will mark Project H-Beam as complete when we’ve accomplished our original motivation: we have filled the gaps where there is a need to make an ecosystem library ABI stable with LibTorch. We will do that by expanding the stable surface, doing a bit of enablement, and continuously improving the experience so that library writers can easily write LibTorch ABI stable libraries themselves.

Note that the marked completion of this project is very dependent on the actual libraries we deem require involved migration, as the APIs they use will guide our stable surface expansion, and each enablement process will inform how to improve our user experience. Once all such libraries are migrated, parts of the feature completion and user experience prongs will naturally close out too. We do not see ourselves creating and maintaining a stable ABI surface covering all of LibTorch; that is NOT the goal.

That said, the limited number of libraries mentioned here do not encompass all the libraries that we think would benefit from becoming ABI stable! By no means! There are several library migrations underway (huggingface kernels, torchcodec, kvcached) that simply don’t require direct involvement from us–and we love that. Ideally, all the library building blocks in the ecosystem can be nicely detangled from each other. The end of Project H-Beam should only mean the start of more ABI stable libraries in the ecosystem, where library authors can easily get there without our help. In the meantime, of course, please don’t be shy and pester us with any concerns and feature requests you have!

Concrete Milestones

With the above in mind, here is how we’re breaking down what needs to be done:

  • Migrate vLLM (completely stable by Q3, targeting 2.13, 2.14)

    • Migrate each of their dependencies (stable by 2.13)

      • FlashMLA, QUTLASS, their forks for FA2 and FA3

      • Convince each dependency owner, or create my own stable fork.

    • Migrate the CUDA wheel (stable by 2.11, in progress by @mikaylagawarecki, will be taken over by @cleonard530)

    • Migrate the ROCm wheel (stable by 2.13)

    • Migrate the CPU wheel (stable by 2.13/2.14, not yet scoped)

  • Improve the Claude skills (drafts by end of May 2026, continuously iterate after)

    • Skill to determine whether a library is migrateable + where there is missing coverage. This skill should list out the steps it would take to make a library ABI stable.

    • Skill to do the actual migration, maybe taking in output from the previous skill + dispatching to proper subskills defined below. This skill should execute the subskills sequentially with proper testing in between.

    • Sub-Skill to migrate C++ meta functions to Python

    • Sub-Skill to refactor a library into ABI stable and unstable parts

    • Sub-Skill to migrate pybind bindings to TORCH_LIBRARY registrations

    • Sub-Skill to migrate unstable API calls → stable calls.

      • Turn this into a deterministic script

  • Harden versioning (by end of H1)

    • Add needed lints and tests to prevent regressions

  • Improve docs and tutorials (ongoing)

  • Explore migrating sglang (can be stable by 2.12/2.13 latest, but need to align)

    • Get on board with the maintainers that they want this

    • Migrate each of their dependencies

    • Migrate the kernels

  • Work with @NicolasHug to migrate torchvision (stable by 2.14)

6 Likes