Hi team,
We have proposed this issue to support out-of-tree rendezvous backend to the elastic pytorch. We have raised a PR and it is under review.
Once this PR is merged, we are planning to contribute additional handlers to the community starting with Redis rendezvous backend. We are looking for recommendation from the community on where to add the out-of-tree plugins. Do you think it’s a good idea to add the new plugins in the Pytorch repo itself? Please provide your thoughts!
For things like this, it’s generally a balance between value to the community and maintenance costs. I think the best place to put a Redis plugin would be in a separate repo maintained by you. This follows a similar pattern to how other non-pytorch projects that use entry_points extensions operate (i.e. fsspec).
if we do see adoption for it then that would be something we can upstream into the main repo.
I realize that’s not the best for discoverability so I think we should add a “Community Supported Backend” section to the elastic/rendezvous documentation and link to any external repos