Drop Spark 3.2 support #8491
zhztheplayer
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I've heard more and more people are upgrading their Spark clusters from 3.2 to higher versions. So perhaps it's necessary for us to discuss about dropping Spark 3.2 support from our main branch.
I have been auditing on the feasibility of this change because we can have chance to solve a long-existing issue #540 after the upgradation. The issue doesn't impact the stability of Gluten but could make the columnar batch management code complex to maintain, and could slightly impact performance. There might be other bonuses so feel free to supplement here.
Beta Was this translation helpful? Give feedback.
All reactions