After that, I got this error. @awgu please have look if possible
RuntimeError: setStorage: sizes [4, 256], strides [256, 1], storage offset 0, and itemsize 4 requiring a storage size of 4096 are out of bounds for storage of size 0
When I only use fully_shard(model, **fsdp_kwargs) it works well, but it’s meaningless right?