The "Ideal" PyTorch FLOP Counter (with __torch_dispatch__)

performance
9

I think this counter estimates MACs, i.e. multiply-and-accumulate.
To compute FLOPs, you have to 2x this.
Making this fix also correctly lines up the numbers with XLA’s flop counter

Here’s an updated gist with that fix: Horace's flop counter, but with flops metric fixed correctly · GitHub

1 Like