Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preparation for non-UTC nightly CI [skip ci] #9885

Merged
merged 4 commits into from
Nov 29, 2023
Merged

Preparation for non-UTC nightly CI [skip ci] #9885

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2acf141
Preparation for non-UTC nightly CI
Nov 29, 2023
b00e177
Add comments
Nov 29, 2023
c678314
Select time zone according to day of week
Nov 29, 2023
c5f14cf
update comments
Nov 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions jenkins/spark-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,20 @@ run_pyarrow_tests() {
./run_pyspark_from_build.sh -m pyarrow_test --pyarrow_test
}

run_non_utc_time_zone_tests() {
# select one time zone according to date
non_utc_time_zones=("Asia/Shanghai" "Iran")
time_zones_length=${#non_utc_time_zones[@]}
current_date=$(date +%Y%m%d)
Copy link
Collaborator

@pxLi pxLi Nov 29, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: may use the date +%w to get day of week, so schedule is fixed unlike depends on date YearMD number

echo "Current date is: ${current_date}"
time_zone_index=$((current_date % time_zones_length))
time_zone="${non_utc_time_zones[${time_zone_index}]}"
echo "Run Non-UTC tests, time zone is ${time_zone}"

# run tests
TZ=${time_zone} ./run_pyspark_from_build.sh
}

# TEST_MODE
# - DEFAULT: all tests except cudf_udf tests
# - DELTA_LAKE_ONLY: Delta Lake tests only
Expand Down Expand Up @@ -321,6 +335,11 @@ if [[ "$TEST_MODE" == "DEFAULT" || "$TEST_MODE" == "PYARROW_ONLY" ]]; then
run_pyarrow_tests
fi

# Non-UTC time zone tests
if [[ "$TEST_MODE" == "NON_UTC_TZ" ]]; then
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please also add comment to explain the MODE in above TEST_MODE section thx

run_non_utc_time_zone_tests
fi

popd
stop-worker.sh
stop-master.sh