symmetric memory all2all integration for EP #2100
Description
Hi team,
I find https://github.com/pytorch/torchtitan/tree/main/torchtitan/experiments/moe_symm_mem_kernels. But seems there is no progress update for a while according to
| Experiment | Test Status | Owners |
|---|---|---|
| moe_symm_mem_kernels | TBA | @kwen2501 |
Is there a plan for the integration? Is there any known issue that stops the release?
Thanks!