-
Notifications
You must be signed in to change notification settings - Fork 815
[slimtensor] integration into backend #16565
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: gh/gasoonjia/101/base
Are you sure you want to change the base?
Conversation
Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/) [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16565
Note: Links to docs will display an error until the docs builds have been completed. ❌ 17 New Failures, 2 Unrelated FailuresAs of commit 7d6a571 with merge base a093fe4 ( NEW FAILURES - The following jobs have failed:
BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/) ghstack-source-id: 333239044 Pull Request resolved: #16565
Pull Request resolved: #16565 ghstack-source-id: 335005909 @exported-using-ghexport Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/)
Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/) [ghstack-poisoned]
Pull Request resolved: #16565 ghstack-source-id: 335280573 @exported-using-ghexport Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/)
Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/) [ghstack-poisoned]
Pull Request resolved: #16565 ghstack-source-id: 335418194 @exported-using-ghexport Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/)
Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/) [ghstack-poisoned]
Pull Request resolved: #16565 perf maintains as before. {F1984962152} ghstack-source-id: 336200461 @exported-using-ghexport Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/)
Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/) [ghstack-poisoned]
Pull Request resolved: #16565 perf maintains as before. {F1984962152} ghstack-source-id: 336233120 @exported-using-ghexport Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/)
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #16565 * #16551 * #16469 * #16457 * #16455 * #16454 * #16453 * #16452 * #16451 * #16450 * #16449 * #16448 * #16447 * #16446 * __->__ #16724 Copy CUDAGuard and CUDAStreamGuard from cuda/runtime/ to aoti/slim/cuda/ to support slimtensor requirement while get rid of potential circular dependency: - cuda_backend/main_functionalities -> aoti/slimtensor -> cuda_backend/cuda_guard This change: - copy guard.h, guard.cpp and test files from backend/cuda_backend to backend/aoti/slim/cuda/ Differential Revision: [D91056808](https://our.internmc.facebook.com/intern/diff/D91056808/)
…v2 (#16446) Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #16565 * #16551 * #16469 * #16457 * #16455 * #16454 * #16453 * #16452 * #16451 * #16450 * #16449 * #16448 * #16447 * __->__ #16446 * #16724 Add SlimTensor-based implementations of AOTI shim functions for tensor creation: 1. `aoti_torch_create_tensor_from_blob_v2()` - Creates a non-owning SlimTensor that wraps existing memory using the `from_blob()` factory Both functions support CPU and CUDA devices and handle all 7 SlimTensor dtypes. Also add `memory_slim.h` and `memory_slim.cpp` with SlimTensor-based shim implementations for working on new API while not impact the current pipeline. Will use memory_slim.{h/cpp} to replace current memory.{h/cpp} when everything has been set up. Differential Revision: [D90126247](https://our.internmc.facebook.com/intern/diff/D90126247/)
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #16565 * #16551 * #16469 * #16457 * #16455 * #16454 * #16453 * #16452 * #16451 * #16450 * #16449 * #16448 * __->__ #16447 * #16446 * #16724 Add SlimTensor-based implementations of AOTI shim functions for tensor creation: `aoti_torch_create_tensor_from_blob_v2()` - Creates a non-owning SlimTensor that wraps existing memory using the `from_blob()` factory Both functions support CPU and CUDA devices and handle all 7 SlimTensor dtypes. Changes: - Add `memory_slim.h` and `memory_slim.cpp` with SlimTensor-based shim implementations - Add `runtime_shims_slim` library target to TARGETS with `CUDA_AVAILABLE=1` preprocessor flag - Add `cuda_shim_slim_cpp_unittest()` function for SlimTensor test targets Differential Revision: [D90126244](https://our.internmc.facebook.com/intern/diff/D90126244/)
larryliu0820
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review automatically exported from Phabricator review in Meta.
This diff makes cuda backend actually use slimtensor. It: updates cuda_backends to create slimtensor from given etensor removed duplicate etensor-driven shim layers under cuda_backend update cmake logic in both cuda backend and aoti backend Perf maintains the same. Shows as before. <img width="3092" height="1902" alt="image" src="https://github.com/user-attachments/assets/6061576b-0d4b-4b20-ac8d-5f45493737d8" /> Worth to notice that we are still keeping two sets of common shims, one is etensor-based and for metal backend, the other is slimtensor-based which used by cuda backend, to not impact metal backend work. When Metal backend finishs the migration, we should delete the duplicate common shims and only keep slimtensor-based one. Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/) [ghstack-poisoned]
Pull Request resolved: #16565 This diff makes cuda backend actually use slimtensor. It: 1. updates cuda_backends to create slimtensor from given etensor 2. removed duplicate etensor-driven shim layers under cuda_backend 3. update cmake logic in both cuda backend and aoti backend Perf maintains the same. Shows as before. {F1984982156} Worth to notice that currently we keeps two sets of common shims, one is etensor-based and for metal backend, the other is slimtensor-based which used by cuda backend, to not impact metal backend work. When Metal backend finishs the migration, we should delete the duplicate common shims and only keep slimtensor-based one. ghstack-source-id: 336538676 @exported-using-ghexport Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/)
This diff makes cuda backend actually use slimtensor. It: updates cuda_backends to create slimtensor from given etensor removed duplicate etensor-driven shim layers under cuda_backend update cmake logic in both cuda backend and aoti backend Perf maintains the same. Shows as before. <img width="3092" height="1902" alt="image" src="https://github.com/user-attachments/assets/6061576b-0d4b-4b20-ac8d-5f45493737d8" /> Worth to notice that we are still keeping two sets of common shims, one is etensor-based and for metal backend, the other is slimtensor-based which used by cuda backend, to not impact metal backend work. When Metal backend finishs the migration, we should delete the duplicate common shims and only keep slimtensor-based one. Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/) [ghstack-poisoned]
Pull Request resolved: #16565 This diff makes cuda backend actually use slimtensor. It: 1. updates cuda_backends to create slimtensor from given etensor 2. removed duplicate etensor-driven shim layers under cuda_backend 3. update cmake logic in both cuda backend and aoti backend Perf maintains the same. Shows as before. {F1984982156} Worth to notice that currently we keeps two sets of common shims, one is etensor-based and for metal backend, the other is slimtensor-based which used by cuda backend, to not impact metal backend work. When Metal backend finishs the migration, we should delete the duplicate common shims and only keep slimtensor-based one. ghstack-source-id: 336658381 @exported-using-ghexport Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/)
This diff makes cuda backend actually use slimtensor. It: updates cuda_backends to create slimtensor from given etensor removed duplicate etensor-driven shim layers under cuda_backend update cmake logic in both cuda backend and aoti backend Perf maintains the same. Shows as before. <img width="3092" height="1902" alt="image" src="https://github.com/user-attachments/assets/6061576b-0d4b-4b20-ac8d-5f45493737d8" /> Worth to notice that we are still keeping two sets of common shims, one is etensor-based and for metal backend, the other is slimtensor-based which used by cuda backend, to not impact metal backend work. When Metal backend finishs the migration, we should delete the duplicate common shims and only keep slimtensor-based one. Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/) [ghstack-poisoned]
Pull Request resolved: #16565 This diff makes cuda backend actually use slimtensor. It: 1. updates cuda_backends to create slimtensor from given etensor 2. removed duplicate etensor-driven shim layers under cuda_backend 3. update cmake logic in both cuda backend and aoti backend Perf maintains the same. Shows as before. {F1984982156} Worth to notice that currently we keeps two sets of common shims, one is etensor-based and for metal backend, the other is slimtensor-based which used by cuda backend, to not impact metal backend work. When Metal backend finishs the migration, we should delete the duplicate common shims and only keep slimtensor-based one. ghstack-source-id: 336675369 @exported-using-ghexport Differential Revision: [D90606409](https://our.internmc.facebook.com/intern/diff/D90606409/)
This diff makes cuda backend actually use slimtensor.
It:
updates cuda_backends to create slimtensor from given etensor
removed duplicate etensor-driven shim layers under cuda_backend
update cmake logic in both cuda backend and aoti backend
Perf maintains the same. Shows as before.
Worth to notice that we are still keeping two sets of common shims, one is etensor-based and for metal backend, the other is slimtensor-based which used by cuda backend, to not impact metal backend work.
When Metal backend finishs the migration, we should delete the duplicate common shims and only keep slimtensor-based one.
Stack from ghstack (oldest at bottom):
Differential Revision: D90606409