What is the recommend serialization format when considering the upcoming pt2?

compiler torchscript
1

I have post this question on What is the recommend serialization format when considering the upcoming pt2? - deployment - PyTorch Forums, but there is no reply. So I repeat it here: begging for help: :joy:

There are two serialization format in the current: torchscript and onnx.

torchscript seems will be deprecated in the long term: The nuances of PyTorch Graph Capture - #9 by penguinwu - compiler - PyTorch Dev Discussions, and after using torchscript for a period of time, we found that torchscript is not easy to use.

ONNX has a relatively larger number of users, but onnx can’t be loaded and run by torch directly, which lead to potential numeric issues.

My question is: is there a new serialization format in pt2? If not, which format you guys recommend for model serialization?

2 Likes
2

torchscript seems will be deprecated in the long term

To be clear, torchscript graph capture will be deprecated in the long term. However, one very reasonable thing that you can do is to use TorchDynamo to capture the graph, and then export to Torchscript. That will be supported for the foreseeable future. It should also hopefully resolve some of the usability issues you had with Torchscript :slight_smile:

2 Likes