The future of C++ model deployment

Cross post What's the difference between torch.export / torchserve / executorch / aotinductor? - #7 by desertfire here, to partly answer your question on how an AOTInductor generated .so can work with Torchscript. If your use case is to have the generated .so work with PyTorch eager, I have recently added some pybind utils to help with that.

For the longer-term, we do view export+AOTInductor as a migration path for TorchScript. We understand control-flow support is something Torchscript users really like and we are working on supporting that in the export+AOTInductor path. Of course, there are still lots of other work to be done to make the new path truly mature enough to work as an out-of-box solution.

1 Like