Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Hardware][TPU] Implement tensor parallelism with Ray #5871

Merged
merged 72 commits into from
Jul 27, 2024
Merged

[Hardware][TPU] Implement tensor parallelism with Ray #5871

merged 72 commits into from
Jul 27, 2024

Conversation

WoosukKwon
Copy link
Collaborator

@WoosukKwon WoosukKwon commented Jun 26, 2024 •
edited
Loading

This PR implements Ray TPU executor for distributed inference support on TPU.

@WoosukKwon WoosukKwon added the tpu Related to Google TPUs label Jun 26, 2024
@WoosukKwon WoosukKwon changed the title [Hardware][TPU] Support tensor parallelism with Ray [Hardware][TPU] Implement tensor parallelism with Ray Jun 26, 2024
@WoosukKwon
Copy link
Collaborator Author

WoosukKwon commented Jun 26, 2024 •
edited
Loading

For this PR, I will merge it after getting reviews. :)

The changes outside the TPU backend was reviewed in #6812 and #6813.

@WoosukKwon WoosukKwon added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 21, 2024
This was referenced Jul 26, 2024
Merged
Merged
@WoosukKwon WoosukKwon merged commit 52f07e3 into main Jul 27, 2024
69 of 71 checks passed
@WoosukKwon WoosukKwon deleted the tpu-n branch July 27, 2024 03:54
@dtrifiro dtrifiro mentioned this pull request Aug 5, 2024
Closed
kylesayrs pushed a commit to neuralmagic/vllm that referenced this pull request Aug 17, 2024
@WoosukKwon @kylesayrs
[Hardware][TPU] Implement tensor parallelism with Ray (vllm-project#5871
876faef 
)
Alvant pushed a commit to compressa-ai/vllm that referenced this pull request Oct 26, 2024
@WoosukKwon @Alvant
[Hardware][TPU] Implement tensor parallelism with Ray (vllm-project#5871
08963df 
)

Signed-off-by: Alvant <[email protected]>
KuntaiDu pushed a commit to KuntaiDu/vllm that referenced this pull request Nov 20, 2024
@WoosukKwon
[Hardware][TPU] Implement tensor parallelism with Ray (vllm-project#5871
aad051c 
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
ready ONLY add when PR is ready to merge/full CI is needed tpu Related to Google TPUs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@WoosukKwon