Perhaps it’s a very basic question but I was not able to reconcile my observations with my understanding.
If I tie weights of an entire model to a flat buffer by doing something similar to this.
Imagine params are supplied by model.parameters().
Will weights hold the reference to the flat buffer or flat buffer stays separate in the memory.
In my observation the value of flat_buffer stays the same during training loop while model weights change as expected. Clearly weights did not get the reference.
Any suggestions to achieve that effect. i.e map all the weights of the model to flat buffer such that weights keep pointing to the flat buffer?
Perhaps something to that effect has been done in the gradient_view_as feature ?