Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[QST] RAPIDS Shuffle Manager (Use Mellanox ConnectX5) Opposite Worker's Executor has been Exited forcefully. #9843

Closed
leehaoun opened this issue Nov 23, 2023 · 5 comments
Closed

[QST] RAPIDS Shuffle Manager (Use Mellanox ConnectX5) Opposite Worker's Executor has been Exited forcefully. #9843

leehaoun opened this issue Nov 23, 2023 · 5 comments
Labels
question Further information is requested

Comments

@leehaoun
Copy link

leehaoun commented Nov 23, 2023 •
edited
Loading

What is your question?
I want to use RAPIDS Shuffle Manager on my spark job.
My Spark job code completes successfully when there are no settings related to the ShuffleManager.
However, when I activate options related to ShuffleManager, the Executor on the remote PC connected via Mellanox optical cable
Exits immediately upon creation and does not activate.

My submit command is below.
(When I remove the contents corresponding to '--conf' here and submit, the remote Executor gets registered normally, and the job processes without any errors)

spark-submit 
--conf spark.shuffle.manager=com.nvidia.spark.rapids.spark341.RapidsShuffleManager
--conf spark.rapids.shuffle.mode=UCX 
--conf spark.shuffle.service.enabled=false --conf spark.dynamicAllocation.enabled=false 
--conf spark.executor.extraClassPath=${SPARK_RAPIDS_PLUGIN_JAR} 
--conf spark.executorEnv.UCX_ERROR_SIGNALS=
--conf spark.executorEnv.UCX_MEMTYPE_CACHE=n 
--packages org.apache.spark:spark-sql-kafka-0-10_2.12:3.2.0 
--jars ${SPARK_RAPIDS_PLUGIN_JAR} 
--files $SPARK_HOME/examples/src/main/scripts/getGpusResources.sh 
--master spark://30.0.0.2:7077 
/root/CNN_TensorFlow_Classify.py

My Spark-Job code is below.

from pyspark.ml.functions import predict_batch_udf
from pyspark.sql.types import ArrayType, FloatType, IntegerType
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, pandas_udf, PandasUDFType
from pyspark import SparkConf
from pyspark import SparkContext as sc
from pyspark.sql.functions import udf
from typing import Iterator
import tensorflow as tf
import pandas as pd
import cv2
import numpy as np

# Pandas DataFrame -> NumPy ndArray (for opencv2)
def PandasDFtoNumpy(pdf, color_scheme=cv2.COLOR_BGR2RGB) :
    results = []
    for i in range(0, len(pdf)):
        np_array = np.frombuffer(pdf['data'].values[i], dtype=np.uint8)
        results.append(cv2.cvtColor( np_array.reshape((pdf['height'].values[i], pdf['width'].values[i], pdf['nChannels'].values[i])),color_scheme ) )
    return results

# resize image to 224 x 224
@pandas_udf(ArrayType(ArrayType(ArrayType(FloatType()))))
def resize_image(iterator: Iterator[pd.DataFrame]) -> Iterator[pd.DataFrame]:
    result = []
    for pdf in iterator:
        # reshape the array
        cv2_images = PandasDFtoNumpy(pdf)
        for img in cv2_images:
            if img is None:
                result.append(None)
            else:
                gray=cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
                img = cv2.resize(gray, (224, 224))
                img = np.expand_dims(img, axis=1)  # 차원 확장
                result.append(img.tolist())
        yield pd.Series(result)

def predict_batch_fn():
    load_path = "/root/models/CNN_model"
    load_options = tf.saved_model.LoadOptions(experimental_io_device='/job:localhost')
    model = tf.keras.models.load_model(load_path, options=load_options)
    def predict(inputs: np.ndarray) -> np.ndarray:
        return model.predict(inputs)
    return predict

def printAll(data):
    print(str(type(data)))
    try:
        print(len(data))
    except TypeError:
        print("data 객체는 len() 함수를 지원하지 않습니다.")
    if hasattr(data, 'shape'):
        print(data.shape)
    else:
        print("data.shape 속성이 존재하지 않습니다.")
conf = SparkConf()
conf.set("spark.executor.cores", "8")
conf.set("spark.executor.memory", "128G")
conf.set("spark.executor.instances", "2")
# conf.set("spark.driver.maxResultSize", "200g")
conf.set("spark.rapids.memory.gpu.reserve","1500000000")

# executor당 할당받는 GPU 갯수 = 1로고정
conf.set("spark.executor.resource.gpu.amount", "1")
conf.set("spark.task.cpus", "1")

# GPU가 병렬처리를 하는 기준, gpu.amount를 executor core갯수의 역수로 설정해야함
conf.set("spark.task.resource.gpu.amount", "0.125")

conf.set("spark.rapids.sql.explain", "NOT_ON_GPU")
conf.set("spark.rapids.sql.concurrentGpuTasks", "4")
conf.set("spark.rapids.memory.pinnedPool.size", "16G")
conf.set("spark.sql.files.maxPartitionBytes", "512m")
conf.set("spark.plugins", "com.nvidia.spark.SQLPlugin")
conf.set("spark.rapids.sql.python.gpu.enabled","true")
conf.set("spark.shuffle.manager","com.nvidia.spark.rapids.spark341.RapidsShuffleManager")
conf.set("spark.rapids.shuffle.mode","UCX")
conf.set("spark.shuffle.service.enabled","false")
conf.set("spark.dynamicAllocation.enabled","false")
conf.set("spark.executorEnv.UCX_MEMTYPE_CACHE","n")
conf.set("spark.executorEnv.UCX_IB_RX_QUEUE_LEN","1024")
conf.set("spark.executorEnv.UCX_TLS","cuda_copy,cuda_ipc,rc,tcp")
conf.set("spark.executorEnv.UCX_RNDV_SCHEME","put_zcopy")
conf.set("spark.executorEnv.UCX_MAX_RNDV_RAILS","1")

spark = SparkSession.builder.appName("Linear_Regression").config(conf=conf).getOrCreate()   

classify = predict_batch_udf(predict_batch_fn,
                          return_type=ArrayType(FloatType()),
                          batch_size=512,
                          input_tensor_shapes=[[224, 224, 1]])

image_dir = "/root/imagenette2/train/n02102040"
df = spark.read.format("image").load(image_dir)
df.repartition(2)
df = df.select(resize_image("image").alias("data"))
predictions = df.select(classify("data").alias("pred"))
predictions.show()

# 예측값에서 가장 높은 인덱스를 추출하는 UDF 정의
get_max_index = udf(lambda arr: arr.index(max(arr)), IntegerType())
#predictions DataFrame에 get_max_index UDF를 적용하여 새로운 컬럼 "pred_index" 생성
predictions = predictions.withColumn("pred_index", get_max_index("pred"))
predictions.select("pred_index").write.format('com.databricks.spark.csv').mode('overwrite').option("header", "true").save("hdfs://30.0.0.2:9000/res")

Problem

When i submit this job, local executor is loaded well.
But, The Executor on the remote worker is forcefully terminated, leaving the following message

23/11/23 10:41:04 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint: Asked to remove non-existent executor 6
23/11/23 10:41:04 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/7 is now RUNNING
23/11/23 10:41:07 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/7 is now EXITED (Command exited with code 1)
23/11/23 10:41:07 INFO StandaloneSchedulerBackend: Executor app-20231123104048-0008/7 removed: Command exited with code 1
23/11/23 10:41:07 INFO BlockManagerMaster: Removal of executor 7 requested
23/11/23 10:41:07 INFO BlockManagerMasterEndpoint: Trying to remove executor 7 from BlockManagerMaster.
23/11/23 10:41:07 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint: Asked to remove non-existent executor 7
23/11/23 10:41:07 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20231123104048-0008/8 on worker-20231123101041-30.0.0.1-36617 (30.0.0.1:36617) with 8 core(s)
23/11/23 10:41:07 INFO StandaloneSchedulerBackend: Granted executor ID app-20231123104048-0008/8 on hostPort 30.0.0.1:36617 with 8 core(s), 128.0 GiB RAM
23/11/23 10:41:07 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/8 is now RUNNING
23/11/23 10:41:08 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0) (30.0.0.2, executor 1, partition 0, PROCESS_LOCAL, 20844 bytes) taskResourceAssignments Map(gpu -> [name: gpu, addresses: 0])
23/11/23 10:41:09 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/8 is now EXITED (Command exited with code 1)
23/11/23 10:41:09 INFO StandaloneSchedulerBackend: Executor app-20231123104048-0008/8 removed: Command exited with code 1
23/11/23 10:41:09 INFO BlockManagerMasterEndpoint: Trying to remove executor 8 from BlockManagerMaster.
23/11/23 10:41:09 INFO BlockManagerMaster: Removal of executor 8 requested
23/11/23 10:41:09 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint: Asked to remove non-existent executor 8
23/11/23 10:41:09 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20231123104048-0008/9 on worker-20231123101041-30.0.0.1-36617 (30.0.0.1:36617) with 8 core(s)
23/11/23 10:41:09 INFO StandaloneSchedulerBackend: Granted executor ID app-20231123104048-0008/9 on hostPort 30.0.0.1:36617 with 8 core(s), 128.0 GiB RAM
23/11/23 10:41:09 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/9 is now RUNNING
23/11/23 10:41:09 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 30.0.0.2:42879 (size: 18.6 KiB, free: 68.1 GiB)
23/11/23 10:41:11 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 30.0.0.2:42879 (size: 37.9 KiB, free: 68.1 GiB)
23/11/23 10:41:11 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/9 is now EXITED (Command exited with code 1)
23/11/23 10:41:11 INFO StandaloneSchedulerBackend: Executor app-20231123104048-0008/9 removed: Command exited with code 1
23/11/23 10:41:11 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20231123104048-0008/10 on worker-20231123101041-30.0.0.1-36617 (30.0.0.1:36617) with 8 core(s)
23/11/23 10:41:11 INFO BlockManagerMasterEndpoint: Trying to remove executor 9 from BlockManagerMaster.
23/11/23 10:41:11 INFO BlockManagerMaster: Removal of executor 9 requested
23/11/23 10:41:11 INFO StandaloneSchedulerBackend: Granted executor ID app-20231123104048-0008/10 on hostPort 30.0.0.1:36617 with 8 core(s), 128.0 GiB RAM
23/11/23 10:41:11 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint: Asked to remove non-existent executor 9
23/11/23 10:41:11 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/10 is now RUNNING
23/11/23 10:41:13 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/10 is now EXITED (Command exited with code 1)
23/11/23 10:41:13 INFO StandaloneSchedulerBackend: Executor app-20231123104048-0008/10 removed: Command exited with code 1
23/11/23 10:41:13 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20231123104048-0008/11 on worker-20231123101041-30.0.0.1-36617 (30.0.0.1:36617) with 8 core(s)
23/11/23 10:41:13 INFO BlockManagerMasterEndpoint: Trying to remove executor 10 from BlockManagerMaster.
23/11/23 10:41:13 INFO StandaloneSchedulerBackend: Granted executor ID app-20231123104048-0008/11 on hostPort 30.0.0.1:36617 with 8 core(s), 128.0 GiB RAM
23/11/23 10:41:13 INFO BlockManagerMaster: Removal of executor 10 requested
23/11/23 10:41:13 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint: Asked to remove non-existent executor 10
23/11/23 10:41:13 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/11 is now RUNNING
23/11/23 10:41:15 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/11 is now EXITED (Command exited with code 1)
23/11/23 10:41:15 INFO StandaloneSchedulerBackend: Executor app-20231123104048-0008/11 removed: Command exited with code 1
23/11/23 10:41:15 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20231123104048-0008/12 on worker-20231123101041-30.0.0.1-36617 (30.0.0.1:36617) with 8 core(s)
23/11/23 10:41:15 INFO BlockManagerMasterEndpoint: Trying to remove executor 11 from BlockManagerMaster.
23/11/23 10:41:15 INFO BlockManagerMaster: Removal of executor 11 requested
23/11/23 10:41:15 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint: Asked to remove non-existent executor 11
23/11/23 10:41:15 INFO StandaloneSchedulerBackend: Granted executor ID app-20231123104048-0008/12 on hostPort 30.0.0.1:36617 with 8 core(s), 128.0 GiB RAM
23/11/23 10:41:15 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/12 is now RUNNING
23/11/23 10:41:17 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/12 is now EXITED (Command exited with code 1)
23/11/23 10:41:17 INFO StandaloneSchedulerBackend: Executor app-20231123104048-0008/12 removed: Command exited with code 1
23/11/23 10:41:17 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20231123104048-0008/13 on worker-20231123101041-30.0.0.1-36617 (30.0.0.1:36617) with 8 core(s)
23/11/23 10:41:17 INFO BlockManagerMasterEndpoint: Trying to remove executor 12 from BlockManagerMaster.
23/11/23 10:41:17 INFO BlockManagerMaster: Removal of executor 12 requested
23/11/23 10:41:17 INFO StandaloneSchedulerBackend$StandaloneDriverEndpoint: Asked to remove non-existent executor 12
23/11/23 10:41:17 INFO StandaloneSchedulerBackend: Granted executor ID app-20231123104048-0008/13 on hostPort 30.0.0.1:36617 with 8 core(s), 128.0 GiB RAM
23/11/23 10:41:17 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20231123104048-0008/13 is now RUNNING

My detail environment - Master

OS : Ubuntu 22.04
cuda : 11.8
Spark : 2.12_3.4.1
JAVA : 1.8
rapids jar : rapids-4-spark_2.12-23.10.0.jar
Conda environment :

conda create --solver=libmamba -n rapids-23.10 -c rapidsai -c conda-forge -c nvidia  \
    rapids=23.10 python=3.10 cuda-version=11.8

GPU : RTX 3080 x 1
RAM : 314G
MLNX Driver : MLNX_OFED_LINUX-23.10-0.5.5.0-ubuntu22.04-x86_64
NV_Peer_Mem : 1.2
UCX : 1.14.1

My detail environment - Worker

OS : Ubuntu 22.04
cuda : 11.8
Spark : 2.12_3.4.1
JAVA : 1.8
rapids jar : rapids-4-spark_2.12-23.10.0.jar
Conda environment :

conda create --solver=libmamba -n rapids-23.10 -c rapidsai -c conda-forge -c nvidia  \
    rapids=23.10 python=3.10 cuda-version=11.8

GPU : RTX 3090 x 1
RAM : 1TB
MLNX Driver : MLNX_OFED_LINUX-23.10-0.5.5.0-ubuntu22.04-x86_64
NV_Peer_Mem : 1.2
UCX : 1.14.1
@leehaoun leehaoun added ? - Needs Triage Need team to review and classify question Further information is requested labels Nov 23, 2023
@abellina
Copy link
Collaborator

hello @leehaoun, it would be great to see executor logs as they should hopefully give more information on what is failing. Can you take a look at those logs and share any errors you see?

@leehaoun
Copy link
Author

@abellina Because the Executor wasn't created, there are no logs available for verification. (No record of Executor addition).

image

image

The EventLog in the HistoryServer is below.

{"Event":"SparkListenerLogStart","Spark Version":"3.4.1"}
{"Event":"SparkListenerResourceProfileAdded","Resource Profile Id":0,"Executor Resource Requests":{"cores":{"Resource Name":"cores","Amount":8,"Discovery Script":"","Vendor":""},"memory":{"Resource Name":"memory","Amount":131072,"Discovery Script":"","Vendor":""},"offHeap":{"Resource Name":"offHeap","Amount":0,"Discovery Script":"","Vendor":""},"gpu":{"Resource Name":"gpu","Amount":1,"Discovery Script":"","Vendor":""}},"Task Resource Requests":{"cpus":{"Resource Name":"cpus","Amount":1.0},"gpu":{"Resource Name":"gpu","Amount":0.125}}}
{"Event":"SparkListenerBlockManagerAdded","Block Manager ID":{"Executor ID":"driver","Host":"30.0.0.2","Port":35705},"Maximum Memory":384093388,"Timestamp":1700712652420,"Maximum Onheap Memory":384093388,"Maximum Offheap Memory":0}
{"Event":"SparkListenerEnvironmentUpdate","JVM Information":{"Java Home":"/usr/lib/jvm/java-8-openjdk-amd64/jre","Java Version":"1.8.0_382 (Private Build)","Scala Version":"version 2.12.17"},"Spark Properties":{"spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.python.gpu.enabled":"true","spark.executor.resource.gpu.amount":"1","spark.rapids.sql.concurrentGpuTasks":"4","spark.sql.files.maxPartitionBytes":"512m","spark.executor.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.driver.host":"30.0.0.2","spark.serializer.objectStreamReset":"100","spark.history.fs.logDirectory":"file:///root/eventLog","spark.eventLog.enabled":"true","spark.shuffle.manager":"com.nvidia.spark.rapids.spark341.RapidsShuffleManager","spark.driver.port":"41803","spark.rdd.compress":"True","spark.jars":"*********(redacted)","spark.rapids.sql.python.gpu.enabled":"true","spark.rapids.memory.gpu.reserve":"1500000000","spark.app.name":"Linear_Regression","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.concurrentGpuTasks":"4","spark.app.initial.file.urls":"*********(redacted)","spark.scheduler.mode":"FIFO","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.gpu.reserve":"1500000000","spark.executor.instances":"2","spark.rapids.memory.pinnedPool.size":"16G","spark.submit.pyFiles":"*********(redacted)","spark.app.submitTime":"1700712647352","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.pinnedPool.size":"16G","spark.files":"*********(redacted)","spark.app.startTime":"1700712650869","spark.executor.id":"driver","spark.task.resource.gpu.amount":"0.125","spark.driver.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.rapids.driver.user.timezone":"Asia/Seoul","spark.app.initial.jar.urls":"*********(redacted)","spark.submit.deployMode":"client","spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.master":"spark://30.0.0.2:7077","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.rapids.sql.explain":"NOT_ON_GPU","spark.executor.memory":"128G","spark.driver.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.eventLog.dir":"file:///root/eventLog","spark.executor.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.executor.cores":"8","spark.plugins":"com.nvidia.spark.SQLPlugin","spark.repl.local.jars":"*********(redacted)","spark.task.cpus":"1","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.explain":"NOT_ON_GPU","spark.sql.extensions":"com.nvidia.spark.rapids.SQLExecPlugin,com.nvidia.spark.udf.Plugin,com.nvidia.spark.rapids.optimizer.SQLOptimizerPlugin","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.driver.user.timezone":"Asia/Seoul","spark.app.id":"app-20231123131052-0002"},"Hadoop Properties":{"hadoop.service.shutdown.timeout":"30s","yarn.resourcemanager.amlauncher.thread-count":"50","yarn.sharedcache.enabled":"false","fs.s3a.connection.maximum":"96","yarn.nodemanager.numa-awareness.numactl.cmd":"/usr/bin/numactl","fs.viewfs.overload.scheme.target.o3fs.impl":"org.apache.hadoop.fs.ozone.OzoneFileSystem","fs.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem","yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms":"1000","yarn.timeline-service.timeline-client.number-of-async-entities-to-merge":"10","hadoop.security.kms.client.timeout":"60","hadoop.http.authentication.kerberos.principal":"HTTP/_HOST@LOCALHOST","mapreduce.jobhistory.loadedjob.tasks.max":"-1","yarn.resourcemanager.application-tag-based-placement.enable":"false","mapreduce.framework.name":"local","yarn.sharedcache.uploader.server.thread-count":"50","yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds.min":"3600","yarn.nodemanager.linux-container-executor.nonsecure-mode.user-pattern":"^[_.A-Za-z0-9][-@_.A-Za-z0-9]{0,255}?[$]?$","tfile.fs.output.buffer.size":"262144","yarn.app.mapreduce.am.job.task.listener.thread-count":"30","yarn.nodemanager.node-attributes.resync-interval-ms":"120000","yarn.nodemanager.container-log-monitor.interval-ms":"60000","hadoop.security.groups.cache.background.reload.threads":"3","yarn.resourcemanager.webapp.cross-origin.enabled":"false","fs.AbstractFileSystem.ftp.impl":"org.apache.hadoop.fs.ftp.FtpFs","fs.viewfs.overload.scheme.target.gs.impl":"com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS","hadoop.registry.secure":"false","hadoop.shell.safely.delete.limit.num.files":"100","mapreduce.job.acl-view-job":" ","fs.s3a.s3guard.ddb.background.sleep":"25ms","fs.s3a.retry.limit":"7","mapreduce.jobhistory.loadedjobs.cache.size":"5","fs.s3a.s3guard.ddb.table.create":"false","fs.viewfs.overload.scheme.target.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem","yarn.nodemanager.amrmproxy.enabled":"false","yarn.timeline-service.entity-group-fs-store.with-user-dir":"false","mapreduce.shuffle.pathcache.expire-after-access-minutes":"5","mapreduce.input.fileinputformat.split.minsize":"0","yarn.resourcemanager.container.liveness-monitor.interval-ms":"600000","yarn.resourcemanager.client.thread-count":"50","io.seqfile.compress.blocksize":"1000000","yarn.nodemanager.runtime.linux.docker.allowed-container-runtimes":"runc","fs.viewfs.overload.scheme.target.http.impl":"org.apache.hadoop.fs.http.HttpFileSystem","yarn.resourcemanager.nodemanagers.heartbeat-interval-slowdown-factor":"1.0","yarn.sharedcache.checksum.algo.impl":"org.apache.hadoop.yarn.sharedcache.ChecksumSHA256Impl","yarn.nodemanager.amrmproxy.interceptor-class.pipeline":"org.apache.hadoop.yarn.server.nodemanager.amrmproxy.DefaultRequestInterceptor","yarn.timeline-service.entity-group-fs-store.leveldb-cache-read-cache-size":"10485760","mapreduce.reduce.shuffle.fetch.retry.interval-ms":"1000","mapreduce.task.profile.maps":"0-2","yarn.scheduler.include-port-in-node-name":"false","yarn.nodemanager.admin-env":"MALLOC_ARENA_MAX=$MALLOC_ARENA_MAX","yarn.resourcemanager.node-removal-untracked.timeout-ms":"60000","mapreduce.am.max-attempts":"2","hadoop.security.kms.client.failover.sleep.base.millis":"100","mapreduce.jobhistory.webapp.https.address":"0.0.0.0:19890","yarn.node-labels.fs-store.impl.class":"org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore","yarn.nodemanager.collector-service.address":"${yarn.nodemanager.hostname}:8048","fs.trash.checkpoint.interval":"0","mapreduce.job.map.output.collector.class":"org.apache.hadoop.mapred.MapTask$MapOutputBuffer","yarn.resourcemanager.node-ip-cache.expiry-interval-secs":"-1","hadoop.http.authentication.signature.secret.file":"*********(redacted)","hadoop.jetty.logs.serve.aliases":"true","yarn.resourcemanager.placement-constraints.handler":"disabled","yarn.timeline-service.handler-thread-count":"10","yarn.resourcemanager.max-completed-applications":"1000","yarn.nodemanager.aux-services.manifest.enabled":"false","yarn.resourcemanager.system-metrics-publisher.enabled":"false","yarn.resourcemanager.placement-constraints.algorithm.class":"org.apache.hadoop.yarn.server.resourcemanager.scheduler.constraint.algorithm.DefaultPlacementAlgorithm","yarn.sharedcache.webapp.address":"0.0.0.0:8788","fs.s3a.select.input.csv.quote.escape.character":"\\\\","yarn.resourcemanager.delegation.token.renew-interval":"*********(redacted)","yarn.sharedcache.nm.uploader.replication.factor":"10","hadoop.security.groups.negative-cache.secs":"30","yarn.app.mapreduce.task.container.log.backups":"0","mapreduce.reduce.skip.proc-count.auto-incr":"true","fs.viewfs.overload.scheme.target.swift.impl":"org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem","hadoop.security.group.mapping.ldap.posix.attr.gid.name":"gidNumber","ipc.client.fallback-to-simple-auth-allowed":"false","yarn.nodemanager.resource.memory.enforced":"true","yarn.resourcemanager.system-metrics-publisher.timeline-server-v1.enable-batch":"false","yarn.client.failover-proxy-provider":"org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider","yarn.timeline-service.http-authentication.simple.anonymous.allowed":"true","ha.health-monitor.check-interval.ms":"1000","yarn.nodemanager.runtime.linux.runc.host-pid-namespace.allowed":"false","hadoop.metrics.jvm.use-thread-mxbean":"false","ipc.[port_number].faircallqueue.multiplexer.weights":"8,4,2,1","yarn.acl.reservation-enable":"false","yarn.resourcemanager.store.class":"org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore","yarn.app.mapreduce.am.hard-kill-timeout-ms":"10000","fs.s3a.etag.checksum.enabled":"false","yarn.nodemanager.container-metrics.enable":"true","ha.health-monitor.rpc.connect.max.retries":"1","yarn.timeline-service.client.fd-clean-interval-secs":"60","yarn.resourcemanager.nodemanagers.heartbeat-interval-scaling-enable":"false","yarn.resourcemanager.nodemanagers.heartbeat-interval-ms":"1000","hadoop.common.configuration.version":"3.0.0","fs.s3a.s3guard.ddb.table.capacity.read":"0","yarn.nodemanager.remote-app-log-dir-suffix":"logs","yarn.nodemanager.container-log-monitor.dir-size-limit-bytes":"1000000000","yarn.nodemanager.windows-container.cpu-limit.enabled":"false","yarn.nodemanager.runtime.linux.docker.privileged-containers.allowed":"false","file.blocksize":"67108864","hadoop.http.idle_timeout.ms":"60000","hadoop.registry.zk.retry.ceiling.ms":"60000","yarn.scheduler.configuration.leveldb-store.path":"${hadoop.tmp.dir}/yarn/system/confstore","yarn.sharedcache.store.in-memory.initial-delay-mins":"10","mapreduce.jobhistory.principal":"jhs/[email protected]","mapreduce.map.skip.proc-count.auto-incr":"true","fs.s3a.committer.name":"file","mapreduce.task.profile.reduces":"0-2","hadoop.zk.num-retries":"1000","yarn.webapp.xfs-filter.enabled":"true","fs.viewfs.overload.scheme.target.hdfs.impl":"org.apache.hadoop.hdfs.DistributedFileSystem","seq.io.sort.mb":"100","yarn.scheduler.configuration.max.version":"100","yarn.timeline-service.webapp.https.address":"${yarn.timeline-service.hostname}:8190","yarn.resourcemanager.scheduler.address":"${yarn.resourcemanager.hostname}:8030","yarn.node-labels.enabled":"false","yarn.resourcemanager.webapp.ui-actions.enabled":"true","mapreduce.task.timeout":"600000","yarn.sharedcache.client-server.thread-count":"50","hadoop.security.groups.shell.command.timeout":"0s","hadoop.security.crypto.cipher.suite":"AES/CTR/NoPadding","yarn.nodemanager.elastic-memory-control.oom-handler":"org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.DefaultOOMHandler","yarn.resourcemanager.connect.max-wait.ms":"900000","fs.defaultFS":"file:///","yarn.minicluster.use-rpc":"false","ipc.[port_number].decay-scheduler.decay-factor":"0.5","fs.har.impl.disable.cache":"true","yarn.webapp.ui2.enable":"false","io.compression.codec.bzip2.library":"system-native","yarn.webapp.filter-invalid-xml-chars":"false","yarn.nodemanager.runtime.linux.runc.layer-mounts-interval-secs":"600","fs.s3a.select.input.csv.record.delimiter":"\\n","fs.s3a.change.detection.source":"etag","ipc.[port_number].backoff.enable":"false","yarn.nodemanager.distributed-scheduling.enabled":"false","mapreduce.shuffle.connection-keep-alive.timeout":"5","yarn.resourcemanager.webapp.https.address":"${yarn.resourcemanager.hostname}:8090","yarn.webapp.enable-rest-app-submissions":"true","mapreduce.jobhistory.address":"0.0.0.0:10020","yarn.resourcemanager.nm-tokens.master-key-rolling-interval-secs":"*********(redacted)","yarn.is.minicluster":"false","yarn.nodemanager.address":"${yarn.nodemanager.hostname}:0","fs.abfss.impl":"org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem","fs.AbstractFileSystem.s3a.impl":"org.apache.hadoop.fs.s3a.S3A","mapreduce.task.combine.progress.records":"10000","yarn.resourcemanager.epoch.range":"0","yarn.resourcemanager.am.max-attempts":"2","yarn.nodemanager.runtime.linux.runc.image-toplevel-dir":"/runc-root","yarn.nodemanager.linux-container-executor.cgroups.hierarchy":"/hadoop-yarn","fs.AbstractFileSystem.wasbs.impl":"org.apache.hadoop.fs.azure.Wasbs","yarn.timeline-service.entity-group-fs-store.cache-store-class":"org.apache.hadoop.yarn.server.timeline.MemoryTimelineStore","yarn.nodemanager.runtime.linux.runc.allowed-container-networks":"host,none,bridge","fs.ftp.transfer.mode":"BLOCK_TRANSFER_MODE","ipc.server.log.slow.rpc":"false","ipc.server.reuseaddr":"true","fs.ftp.timeout":"0","yarn.resourcemanager.node-labels.provider.fetch-interval-ms":"1800000","yarn.router.webapp.https.address":"0.0.0.0:8091","yarn.nodemanager.webapp.cross-origin.enabled":"false","fs.wasb.impl":"org.apache.hadoop.fs.azure.NativeAzureFileSystem","yarn.resourcemanager.auto-update.containers":"false","yarn.app.mapreduce.am.job.committer.cancel-timeout":"60000","yarn.scheduler.configuration.zk-store.parent-path":"/confstore","yarn.nodemanager.default-container-executor.log-dirs.permissions":"710","yarn.app.attempt.diagnostics.limit.kc":"64","fs.viewfs.overload.scheme.target.swebhdfs.impl":"org.apache.hadoop.hdfs.web.SWebHdfsFileSystem","yarn.client.failover-no-ha-proxy-provider":"org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider","fs.s3a.change.detection.mode":"server","ftp.bytes-per-checksum":"512","yarn.nodemanager.resource.memory-mb":"-1","fs.AbstractFileSystem.abfs.impl":"org.apache.hadoop.fs.azurebfs.Abfs","yarn.timeline-service.writer.flush-interval-seconds":"60","fs.s3a.fast.upload.active.blocks":"4","yarn.resourcemanager.submission-preprocessor.enabled":"false","hadoop.security.credential.clear-text-fallback":"true","yarn.nodemanager.collector-service.thread-count":"5","ipc.[port_number].scheduler.impl":"org.apache.hadoop.ipc.DefaultRpcScheduler","fs.azure.secure.mode":"false","mapreduce.jobhistory.joblist.cache.size":"20000","fs.ftp.host":"0.0.0.0","yarn.timeline-service.writer.async.queue.capacity":"100","yarn.resourcemanager.fs.state-store.num-retries":"0","yarn.resourcemanager.nodemanager-connect-retries":"10","yarn.nodemanager.log-aggregation.num-log-files-per-app":"30","hadoop.security.kms.client.encrypted.key.cache.low-watermark":"0.3f","fs.s3a.committer.magic.enabled":"true","yarn.timeline-service.client.max-retries":"30","dfs.ha.fencing.ssh.connect-timeout":"30000","yarn.log-aggregation-enable":"false","yarn.system-metrics-publisher.enabled":"false","mapreduce.reduce.markreset.buffer.percent":"0.0","fs.AbstractFileSystem.viewfs.impl":"org.apache.hadoop.fs.viewfs.ViewFs","yarn.resourcemanager.nodemanagers.heartbeat-interval-speedup-factor":"1.0","mapreduce.task.io.sort.factor":"10","yarn.nodemanager.amrmproxy.client.thread-count":"25","ha.failover-controller.new-active.rpc-timeout.ms":"60000","yarn.nodemanager.container-localizer.java.opts":"-Xmx256m","mapreduce.jobhistory.datestring.cache.size":"200000","mapreduce.job.acl-modify-job":" ","yarn.nodemanager.windows-container.memory-limit.enabled":"false","yarn.timeline-service.webapp.address":"${yarn.timeline-service.hostname}:8188","yarn.app.mapreduce.am.job.committer.commit-window":"10000","yarn.nodemanager.container-manager.thread-count":"20","yarn.minicluster.fixed.ports":"false","hadoop.tags.system":"YARN,HDFS,NAMENODE,DATANODE,REQUIRED,SECURITY,KERBEROS,PERFORMANCE,CLIENT\n      ,SERVER,DEBUG,DEPRECATED,COMMON,OPTIONAL","yarn.cluster.max-application-priority":"0","yarn.timeline-service.ttl-enable":"true","mapreduce.jobhistory.recovery.store.fs.uri":"${hadoop.tmp.dir}/mapred/history/recoverystore","hadoop.caller.context.signature.max.size":"40","ipc.[port_number].decay-scheduler.backoff.responsetime.enable":"false","yarn.client.load.resource-types.from-server":"false","ha.zookeeper.session-timeout.ms":"10000","ipc.[port_number].decay-scheduler.metrics.top.user.count":"10","tfile.io.chunk.size":"1048576","fs.s3a.s3guard.ddb.table.capacity.write":"0","yarn.dispatcher.print-events-info.threshold":"5000","mapreduce.job.speculative.slowtaskthreshold":"1.0","io.serializations":"org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization","hadoop.security.kms.client.failover.sleep.max.millis":"2000","hadoop.security.group.mapping.ldap.directory.search.timeout":"10000","yarn.scheduler.configuration.store.max-logs":"1000","yarn.nodemanager.node-attributes.provider.fetch-interval-ms":"600000","fs.swift.impl":"org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem","yarn.nodemanager.local-cache.max-files-per-directory":"8192","hadoop.http.cross-origin.enabled":"false","hadoop.zk.acl":"world:anyone:rwcda","yarn.nodemanager.runtime.linux.runc.image-tag-to-manifest-plugin.num-manifests-to-cache":"10","mapreduce.map.sort.spill.percent":"0.80","yarn.timeline-service.entity-group-fs-store.scan-interval-seconds":"60","yarn.node-attribute.fs-store.impl.class":"org.apache.hadoop.yarn.server.resourcemanager.nodelabels.FileSystemNodeAttributeStore","fs.s3a.retry.interval":"500ms","yarn.timeline-service.client.best-effort":"false","yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled":"*********(redacted)","hadoop.security.group.mapping.ldap.posix.attr.uid.name":"uidNumber","fs.AbstractFileSystem.swebhdfs.impl":"org.apache.hadoop.fs.SWebHdfs","yarn.nodemanager.elastic-memory-control.timeout-sec":"5","fs.s3a.select.enabled":"true","mapreduce.ifile.readahead":"true","yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms":"300000","yarn.timeline-service.reader.webapp.address":"${yarn.timeline-service.webapp.address}","yarn.resourcemanager.placement-constraints.algorithm.pool-size":"1","yarn.timeline-service.hbase.coprocessor.jar.hdfs.location":"/hbase/coprocessor/hadoop-yarn-server-timelineservice.jar","hadoop.security.kms.client.encrypted.key.cache.num.refill.threads":"2","yarn.resourcemanager.scheduler.class":"org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler","yarn.app.mapreduce.am.command-opts":"-Xmx1024m","fs.s3a.metadatastore.fail.on.write.error":"true","hadoop.http.sni.host.check.enabled":"false","mapreduce.cluster.local.dir":"${hadoop.tmp.dir}/mapred/local","io.mapfile.bloom.error.rate":"0.005","fs.client.resolve.topology.enabled":"false","yarn.nodemanager.runtime.linux.allowed-runtimes":"default","yarn.sharedcache.store.class":"org.apache.hadoop.yarn.server.sharedcachemanager.store.InMemorySCMStore","ha.failover-controller.graceful-fence.rpc-timeout.ms":"5000","ftp.replication":"3","fs.getspaceused.jitterMillis":"60000","hadoop.security.uid.cache.secs":"14400","mapreduce.job.maxtaskfailures.per.tracker":"3","fs.s3a.metadatastore.impl":"org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore","io.skip.checksum.errors":"false","yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts":"3","yarn.timeline-service.webapp.xfs-filter.xframe-options":"SAMEORIGIN","fs.s3a.connection.timeout":"200000","yarn.app.mapreduce.am.webapp.https.enabled":"false","mapreduce.job.max.split.locations":"15","yarn.resourcemanager.nm-container-queuing.max-queue-length":"15","yarn.resourcemanager.delegation-token.always-cancel":"*********(redacted)","hadoop.registry.zk.session.timeout.ms":"60000","yarn.federation.cache-ttl.secs":"300","mapreduce.jvm.system-properties-to-log":"os.name,os.version,java.home,java.runtime.version,java.vendor,java.version,java.vm.name,java.class.path,java.io.tmpdir,user.dir,user.name","yarn.resourcemanager.opportunistic-container-allocation.nodes-used":"10","yarn.timeline-service.entity-group-fs-store.active-dir":"/tmp/entity-file-history/active","mapreduce.shuffle.transfer.buffer.size":"131072","yarn.timeline-service.client.retry-interval-ms":"1000","yarn.timeline-service.flowname.max-size":"0","yarn.http.policy":"HTTP_ONLY","fs.s3a.socket.send.buffer":"8192","fs.AbstractFileSystem.abfss.impl":"org.apache.hadoop.fs.azurebfs.Abfss","yarn.sharedcache.uploader.server.address":"0.0.0.0:8046","yarn.resourcemanager.delegation-token.max-conf-size-bytes":"*********(redacted)","hadoop.http.authentication.token.validity":"*********(redacted)","mapreduce.shuffle.max.connections":"0","yarn.minicluster.yarn.nodemanager.resource.memory-mb":"4096","mapreduce.job.emit-timeline-data":"false","yarn.nodemanager.resource.system-reserved-memory-mb":"-1","hadoop.kerberos.min.seconds.before.relogin":"60","mapreduce.jobhistory.move.thread-count":"3","yarn.resourcemanager.admin.client.thread-count":"1","yarn.dispatcher.drain-events.timeout":"300000","ipc.[port_number].decay-scheduler.backoff.responsetime.thresholds":"10s,20s,30s,40s","fs.s3a.buffer.dir":"${hadoop.tmp.dir}/s3a","hadoop.ssl.enabled.protocols":"TLSv1.2","mapreduce.jobhistory.admin.address":"0.0.0.0:10033","yarn.log-aggregation-status.time-out.ms":"600000","fs.s3a.accesspoint.required":"false","mapreduce.shuffle.port":"13562","yarn.resourcemanager.max-log-aggregation-diagnostics-in-memory":"10","yarn.nodemanager.health-checker.interval-ms":"600000","yarn.resourcemanager.proxy.connection.timeout":"60000","yarn.router.clientrm.interceptor-class.pipeline":"org.apache.hadoop.yarn.server.router.clientrm.DefaultClientRequestInterceptor","yarn.resourcemanager.zk-appid-node.split-index":"0","ftp.blocksize":"67108864","yarn.nodemanager.runtime.linux.sandbox-mode.local-dirs.permissions":"read","yarn.router.rmadmin.interceptor-class.pipeline":"org.apache.hadoop.yarn.server.router.rmadmin.DefaultRMAdminRequestInterceptor","yarn.nodemanager.log-container-debug-info.enabled":"true","yarn.resourcemanager.activities-manager.app-activities.max-queue-length":"100","yarn.resourcemanager.application-https.policy":"NONE","yarn.client.max-cached-nodemanagers-proxies":"0","yarn.nodemanager.linux-container-executor.cgroups.delete-delay-ms":"20","yarn.nodemanager.delete.debug-delay-sec":"0","yarn.nodemanager.pmem-check-enabled":"true","yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage":"90.0","mapreduce.app-submission.cross-platform":"false","yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms":"10000","yarn.nodemanager.container-retry-minimum-interval-ms":"1000","hadoop.security.groups.cache.secs":"300","yarn.federation.enabled":"false","yarn.workflow-id.tag-prefix":"workflowid:","fs.azure.local.sas.key.mode":"false","ipc.maximum.data.length":"134217728","fs.s3a.endpoint":"s3.amazonaws.com","mapreduce.shuffle.max.threads":"0","yarn.router.pipeline.cache-max-size":"25","yarn.resourcemanager.nm-container-queuing.load-comparator":"QUEUE_LENGTH","yarn.resourcemanager.resource-tracker.nm.ip-hostname-check":"false","hadoop.security.authorization":"false","mapreduce.job.complete.cancel.delegation.tokens":"*********(redacted)","fs.s3a.paging.maximum":"5000","nfs.exports.allowed.hosts":"* rw","yarn.nodemanager.amrmproxy.ha.enable":"false","fs.AbstractFileSystem.gs.impl":"com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS","mapreduce.jobhistory.http.policy":"HTTP_ONLY","yarn.sharedcache.store.in-memory.check-period-mins":"720","hadoop.security.group.mapping.ldap.ssl":"false","fs.s3a.downgrade.syncable.exceptions":"true","yarn.client.application-client-protocol.poll-interval-ms":"200","yarn.scheduler.configuration.leveldb-store.compaction-interval-secs":"86400","yarn.timeline-service.writer.class":"org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineWriterImpl","ha.zookeeper.parent-znode":"/hadoop-ha","yarn.resourcemanager.submission-preprocessor.file-refresh-interval-ms":"60000","yarn.nodemanager.log-aggregation.policy.class":"org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AllContainerLogAggregationPolicy","mapreduce.reduce.shuffle.merge.percent":"0.66","hadoop.security.group.mapping.ldap.search.filter.group":"(objectClass=group)","yarn.resourcemanager.placement-constraints.scheduler.pool-size":"1","yarn.resourcemanager.activities-manager.cleanup-interval-ms":"5000","yarn.nodemanager.resourcemanager.minimum.version":"NONE","mapreduce.job.speculative.speculative-cap-running-tasks":"0.1","yarn.admin.acl":"*","ipc.[port_number].identity-provider.impl":"org.apache.hadoop.ipc.UserIdentityProvider","yarn.nodemanager.recovery.supervised":"false","yarn.sharedcache.admin.thread-count":"1","yarn.resourcemanager.ha.automatic-failover.enabled":"true","yarn.nodemanager.container-log-monitor.total-size-limit-bytes":"10000000000","mapreduce.reduce.skip.maxgroups":"0","mapreduce.reduce.shuffle.connect.timeout":"180000","yarn.nodemanager.health-checker.scripts":"script","yarn.resourcemanager.address":"${yarn.resourcemanager.hostname}:8032","ipc.client.ping":"true","mapreduce.task.local-fs.write-limit.bytes":"-1","fs.adl.oauth2.access.token.provider.type":"*********(redacted)","mapreduce.shuffle.ssl.file.buffer.size":"65536","yarn.resourcemanager.ha.automatic-failover.embedded":"true","yarn.nodemanager.resource-plugins.gpu.docker-plugin":"nvidia-docker-v1","fs.s3a.s3guard.consistency.retry.interval":"2s","fs.s3a.multipart.purge":"false","yarn.scheduler.configuration.store.class":"file","yarn.resourcemanager.nm-container-queuing.queue-limit-stdev":"1.0f","mapreduce.job.end-notification.max.attempts":"5","mapreduce.output.fileoutputformat.compress.codec":"org.apache.hadoop.io.compress.DefaultCodec","yarn.nodemanager.container-monitor.procfs-tree.smaps-based-rss.enabled":"false","ipc.client.bind.wildcard.addr":"false","yarn.resourcemanager.webapp.rest-csrf.enabled":"false","ha.health-monitor.connect-retry-interval.ms":"1000","yarn.nodemanager.keytab":"/etc/krb5.keytab","mapreduce.jobhistory.keytab":"/etc/security/keytab/jhs.service.keytab","fs.s3a.threads.max":"64","yarn.nodemanager.runtime.linux.docker.image-update":"false","mapreduce.reduce.shuffle.input.buffer.percent":"0.70","fs.viewfs.overload.scheme.target.abfss.impl":"org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem","yarn.dispatcher.cpu-monitor.samples-per-min":"60","hadoop.security.token.service.use_ip":"*********(redacted)","yarn.nodemanager.runtime.linux.docker.allowed-container-networks":"host,none,bridge","yarn.nodemanager.node-labels.resync-interval-ms":"120000","hadoop.tmp.dir":"/tmp/hadoop-${user.name}","mapreduce.job.maps":"2","mapreduce.jobhistory.webapp.rest-csrf.custom-header":"X-XSRF-Header","mapreduce.job.end-notification.max.retry.interval":"5000","yarn.log-aggregation.retain-check-interval-seconds":"-1","yarn.resourcemanager.resource-tracker.client.thread-count":"50","yarn.nodemanager.containers-launcher.class":"org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher","yarn.rm.system-metrics-publisher.emit-container-events":"false","yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size":"10000","yarn.resourcemanager.ha.automatic-failover.zk-base-path":"/yarn-leader-election","io.seqfile.local.dir":"${hadoop.tmp.dir}/io/local","fs.s3a.s3guard.ddb.throttle.retry.interval":"100ms","fs.AbstractFileSystem.wasb.impl":"org.apache.hadoop.fs.azure.Wasb","mapreduce.client.submit.file.replication":"10","mapreduce.jobhistory.minicluster.fixed.ports":"false","fs.s3a.multipart.threshold":"128M","yarn.resourcemanager.webapp.xfs-filter.xframe-options":"SAMEORIGIN","mapreduce.jobhistory.done-dir":"${yarn.app.mapreduce.am.staging-dir}/history/done","ipc.server.purge.interval":"15","ipc.client.idlethreshold":"4000","yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage":"false","mapreduce.reduce.input.buffer.percent":"0.0","yarn.nodemanager.runtime.linux.docker.userremapping-gid-threshold":"1","yarn.nodemanager.webapp.rest-csrf.enabled":"false","fs.ftp.host.port":"21","ipc.ping.interval":"60000","yarn.resourcemanager.history-writer.multi-threaded-dispatcher.pool-size":"10","yarn.resourcemanager.admin.address":"${yarn.resourcemanager.hostname}:8033","file.client-write-packet-size":"65536","ipc.client.kill.max":"10","mapreduce.reduce.speculative":"true","hadoop.security.key.default.bitlength":"128","mapreduce.job.reducer.unconditional-preempt.delay.sec":"300","yarn.nodemanager.disk-health-checker.interval-ms":"120000","yarn.nodemanager.log.deletion-threads-count":"4","fs.s3a.committer.abort.pending.uploads":"true","yarn.webapp.filter-entity-list-by-user":"false","yarn.resourcemanager.activities-manager.app-activities.ttl-ms":"600000","ipc.client.connection.maxidletime":"10000","mapreduce.task.io.sort.mb":"100","yarn.nodemanager.localizer.client.thread-count":"5","io.erasurecode.codec.rs.rawcoders":"rs_native,rs_java","io.erasurecode.codec.rs-legacy.rawcoders":"rs-legacy_java","yarn.sharedcache.admin.address":"0.0.0.0:8047","yarn.resourcemanager.placement-constraints.algorithm.iterator":"SERIAL","yarn.nodemanager.localizer.cache.cleanup.interval-ms":"600000","hadoop.security.crypto.codec.classes.aes.ctr.nopadding":"org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec, org.apache.hadoop.crypto.JceAesCtrCryptoCodec","mapreduce.job.cache.limit.max-resources-mb":"0","fs.s3a.connection.ssl.enabled":"true","yarn.nodemanager.process-kill-wait.ms":"5000","mapreduce.job.hdfs-servers":"${fs.defaultFS}","yarn.app.mapreduce.am.webapp.https.client.auth":"false","hadoop.workaround.non.threadsafe.getpwuid":"true","fs.df.interval":"60000","ipc.[port_number].decay-scheduler.thresholds":"13,25,50","fs.s3a.multiobjectdelete.enable":"true","yarn.sharedcache.cleaner.resource-sleep-ms":"0","yarn.nodemanager.disk-health-checker.min-healthy-disks":"0.25","hadoop.shell.missing.defaultFs.warning":"false","io.file.buffer.size":"65536","fs.viewfs.overload.scheme.target.wasb.impl":"org.apache.hadoop.fs.azure.NativeAzureFileSystem","hadoop.security.group.mapping.ldap.search.attr.member":"member","hadoop.security.random.device.file.path":"/dev/urandom","hadoop.security.sensitive-config-keys":"*********(redacted)","fs.s3a.s3guard.ddb.max.retries":"9","fs.viewfs.overload.scheme.target.file.impl":"org.apache.hadoop.fs.LocalFileSystem","hadoop.rpc.socket.factory.class.default":"org.apache.hadoop.net.StandardSocketFactory","yarn.intermediate-data-encryption.enable":"false","yarn.resourcemanager.connect.retry-interval.ms":"30000","yarn.nodemanager.container.stderr.pattern":"{*stderr*,*STDERR*}","yarn.scheduler.minimum-allocation-mb":"1024","yarn.app.mapreduce.am.staging-dir":"/tmp/hadoop-yarn/staging","mapreduce.reduce.shuffle.read.timeout":"180000","hadoop.http.cross-origin.max-age":"1800","io.erasurecode.codec.xor.rawcoders":"xor_native,xor_java","fs.s3a.s3guard.consistency.retry.limit":"7","fs.s3a.connection.establish.timeout":"5000","mapreduce.job.running.map.limit":"0","yarn.minicluster.control-resource-monitoring":"false","hadoop.ssl.require.client.cert":"false","hadoop.kerberos.kinit.command":"kinit","yarn.federation.state-store.class":"org.apache.hadoop.yarn.server.federation.store.impl.MemoryFederationStateStore","mapreduce.reduce.log.level":"INFO","hadoop.security.dns.log-slow-lookups.threshold.ms":"1000","mapreduce.job.ubertask.enable":"false","adl.http.timeout":"-1","yarn.resourcemanager.placement-constraints.retry-attempts":"3","hadoop.caller.context.enabled":"false","hadoop.security.group.mapping.ldap.num.attempts":"3","yarn.nodemanager.vmem-pmem-ratio":"2.1","hadoop.rpc.protection":"authentication","ha.health-monitor.rpc-timeout.ms":"45000","yarn.nodemanager.remote-app-log-dir":"/tmp/logs","hadoop.zk.timeout-ms":"10000","fs.s3a.s3guard.cli.prune.age":"86400000","yarn.nodemanager.resource.pcores-vcores-multiplier":"1.0","yarn.nodemanager.runtime.linux.sandbox-mode":"disabled","yarn.app.mapreduce.am.containerlauncher.threadpool-initial-size":"10","fs.viewfs.overload.scheme.target.webhdfs.impl":"org.apache.hadoop.hdfs.web.WebHdfsFileSystem","fs.s3a.committer.threads":"8","hadoop.zk.retry-interval-ms":"1000","hadoop.security.crypto.buffer.size":"8192","yarn.nodemanager.node-labels.provider.fetch-interval-ms":"600000","mapreduce.jobhistory.recovery.store.leveldb.path":"${hadoop.tmp.dir}/mapred/history/recoverystore","yarn.client.failover-retries-on-socket-timeouts":"0","fs.s3a.ssl.channel.mode":"default_jsse","yarn.nodemanager.resource.memory.enabled":"false","fs.azure.authorization.caching.enable":"true","hadoop.security.instrumentation.requires.admin":"false","yarn.nodemanager.delete.thread-count":"4","mapreduce.job.finish-when-all-reducers-done":"true","hadoop.registry.jaas.context":"Client","yarn.timeline-service.leveldb-timeline-store.path":"${hadoop.tmp.dir}/yarn/timeline","io.map.index.interval":"128","yarn.resourcemanager.nm-container-queuing.max-queue-wait-time-ms":"100","fs.abfs.impl":"org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem","mapreduce.job.counters.max":"120","mapreduce.jobhistory.webapp.rest-csrf.enabled":"false","yarn.timeline-service.store-class":"org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore","mapreduce.jobhistory.move.interval-ms":"180000","fs.s3a.change.detection.version.required":"true","yarn.nodemanager.localizer.fetch.thread-count":"4","yarn.resourcemanager.scheduler.client.thread-count":"50","hadoop.ssl.hostname.verifier":"DEFAULT","yarn.timeline-service.leveldb-state-store.path":"${hadoop.tmp.dir}/yarn/timeline","mapreduce.job.classloader":"false","mapreduce.task.profile.map.params":"${mapreduce.task.profile.params}","ipc.client.connect.timeout":"20000","hadoop.security.auth_to_local.mechanism":"hadoop","yarn.timeline-service.app-collector.linger-period.ms":"60000","yarn.nm.liveness-monitor.expiry-interval-ms":"600000","yarn.resourcemanager.reservation-system.planfollower.time-step":"1000","yarn.resourcemanager.proxy.timeout.enabled":"true","yarn.resourcemanager.activities-manager.scheduler-activities.ttl-ms":"600000","yarn.nodemanager.runtime.linux.docker.enable-userremapping.allowed":"true","yarn.webapp.api-service.enable":"false","yarn.nodemanager.recovery.enabled":"false","mapreduce.job.end-notification.retry.interval":"1000","fs.du.interval":"600000","fs.ftp.impl":"org.apache.hadoop.fs.ftp.FTPFileSystem","yarn.nodemanager.container.stderr.tail.bytes":"4096","yarn.nodemanager.disk-health-checker.disk-free-space-threshold.enabled":"true","hadoop.security.group.mapping.ldap.read.timeout.ms":"60000","hadoop.security.groups.cache.warn.after.ms":"5000","file.bytes-per-checksum":"512","mapreduce.outputcommitter.factory.scheme.s3a":"org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory","hadoop.security.groups.cache.background.reload":"false","yarn.nodemanager.container-monitor.enabled":"true","yarn.nodemanager.elastic-memory-control.enabled":"false","net.topology.script.number.args":"100","mapreduce.task.merge.progress.records":"10000","yarn.nodemanager.localizer.address":"${yarn.nodemanager.hostname}:8040","yarn.timeline-service.keytab":"/etc/krb5.keytab","mapreduce.reduce.shuffle.fetch.retry.timeout-ms":"30000","yarn.resourcemanager.rm.container-allocation.expiry-interval-ms":"600000","yarn.nodemanager.container-executor.exit-code-file.timeout-ms":"2000","mapreduce.fileoutputcommitter.algorithm.version":"1","yarn.resourcemanager.work-preserving-recovery.enabled":"true","mapreduce.map.skip.maxrecords":"0","yarn.sharedcache.root-dir":"/sharedcache","fs.s3a.retry.throttle.limit":"20","hadoop.http.authentication.type":"simple","fs.viewfs.overload.scheme.target.oss.impl":"org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem","mapreduce.job.cache.limit.max-resources":"0","mapreduce.task.userlog.limit.kb":"0","ipc.[port_number].weighted-cost.handler":"1","yarn.resourcemanager.scheduler.monitor.enable":"false","ipc.client.connect.max.retries":"10","hadoop.registry.zk.retry.times":"5","yarn.nodemanager.resource-monitor.interval-ms":"3000","yarn.nodemanager.resource-plugins.gpu.allowed-gpu-devices":"auto","mapreduce.job.sharedcache.mode":"disabled","yarn.nodemanager.webapp.rest-csrf.custom-header":"X-XSRF-Header","mapreduce.shuffle.listen.queue.size":"128","yarn.scheduler.configuration.mutation.acl-policy.class":"org.apache.hadoop.yarn.server.resourcemanager.scheduler.DefaultConfigurationMutationACLPolicy","mapreduce.map.cpu.vcores":"1","yarn.log-aggregation.file-formats":"TFile","yarn.timeline-service.client.fd-retain-secs":"300","fs.s3a.select.output.csv.field.delimiter":",","yarn.nodemanager.health-checker.timeout-ms":"1200000","hadoop.user.group.static.mapping.overrides":"dr.who=;","fs.azure.sas.expiry.period":"90d","fs.s3a.select.output.csv.record.delimiter":"\\n","mapreduce.jobhistory.recovery.store.class":"org.apache.hadoop.mapreduce.v2.hs.HistoryServerFileSystemStateStoreService","fs.viewfs.overload.scheme.target.https.impl":"org.apache.hadoop.fs.http.HttpsFileSystem","fs.s3a.s3guard.ddb.table.sse.enabled":"false","yarn.resourcemanager.fail-fast":"${yarn.fail-fast}","yarn.resourcemanager.proxy-user-privileges.enabled":"false","yarn.router.webapp.interceptor-class.pipeline":"org.apache.hadoop.yarn.server.router.webapp.DefaultRequestInterceptorREST","yarn.nodemanager.resource.memory.cgroups.soft-limit-percentage":"90.0","mapreduce.job.reducer.preempt.delay.sec":"0","hadoop.util.hash.type":"murmur","yarn.nodemanager.disk-validator":"basic","yarn.app.mapreduce.client.job.max-retries":"3","fs.viewfs.overload.scheme.target.ftp.impl":"org.apache.hadoop.fs.ftp.FTPFileSystem","mapreduce.reduce.shuffle.retry-delay.max.ms":"60000","hadoop.security.group.mapping.ldap.connection.timeout.ms":"60000","mapreduce.task.profile.params":"-agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,verbose=n,file=%s","yarn.app.mapreduce.shuffle.log.backups":"0","yarn.nodemanager.container-diagnostics-maximum-size":"10000","hadoop.registry.zk.retry.interval.ms":"1000","yarn.nodemanager.linux-container-executor.cgroups.delete-timeout-ms":"1000","fs.AbstractFileSystem.file.impl":"org.apache.hadoop.fs.local.LocalFs","yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds":"-1","mapreduce.jobhistory.cleaner.interval-ms":"86400000","hadoop.registry.zk.quorum":"localhost:2181","yarn.nodemanager.runtime.linux.runc.allowed-container-runtimes":"runc","mapreduce.output.fileoutputformat.compress":"false","yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs":"*********(redacted)","fs.s3a.assumed.role.session.duration":"30m","hadoop.security.group.mapping.ldap.conversion.rule":"none","hadoop.ssl.server.conf":"ssl-server.xml","fs.s3a.retry.throttle.interval":"100ms","seq.io.sort.factor":"100","fs.viewfs.overload.scheme.target.ofs.impl":"org.apache.hadoop.fs.ozone.RootedOzoneFileSystem","yarn.sharedcache.cleaner.initial-delay-mins":"10","mapreduce.client.completion.pollinterval":"5000","hadoop.ssl.keystores.factory.class":"org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory","yarn.app.mapreduce.am.resource.cpu-vcores":"1","yarn.timeline-service.enabled":"false","yarn.nodemanager.runtime.linux.docker.capabilities":"CHOWN,DAC_OVERRIDE,FSETID,FOWNER,MKNOD,NET_RAW,SETGID,SETUID,SETFCAP,SETPCAP,NET_BIND_SERVICE,SYS_CHROOT,KILL,AUDIT_WRITE","yarn.acl.enable":"false","yarn.timeline-service.entity-group-fs-store.done-dir":"/tmp/entity-file-history/done/","hadoop.security.group.mapping.ldap.num.attempts.before.failover":"3","mapreduce.task.profile":"false","hadoop.prometheus.endpoint.enabled":"false","yarn.resourcemanager.fs.state-store.uri":"${hadoop.tmp.dir}/yarn/system/rmstore","mapreduce.jobhistory.always-scan-user-dir":"false","fs.s3a.metadatastore.metadata.ttl":"15m","yarn.nodemanager.opportunistic-containers-use-pause-for-preemption":"false","yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user":"nobody","yarn.timeline-service.reader.class":"org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl","yarn.resourcemanager.configuration.provider-class":"org.apache.hadoop.yarn.LocalConfigurationProvider","yarn.nodemanager.runtime.linux.docker.userremapping-uid-threshold":"1","yarn.resourcemanager.configuration.file-system-based-store":"/yarn/conf","mapreduce.job.cache.limit.max-single-resource-mb":"0","yarn.nodemanager.runtime.linux.docker.stop.grace-period":"10","yarn.resourcemanager.resource-profiles.source-file":"resource-profiles.json","mapreduce.job.dfs.storage.capacity.kill-limit-exceed":"false","yarn.nodemanager.resource.percentage-physical-cpu-limit":"100","mapreduce.jobhistory.client.thread-count":"10","tfile.fs.input.buffer.size":"262144","mapreduce.client.progressmonitor.pollinterval":"1000","yarn.nodemanager.log-dirs":"${yarn.log.dir}/userlogs","yarn.resourcemanager.opportunistic.max.container-allocation.per.am.heartbeat":"-1","fs.automatic.close":"true","yarn.resourcemanager.delegation-token-renewer.thread-retry-interval":"*********(redacted)","fs.s3a.select.input.csv.quote.character":"\"","yarn.nodemanager.hostname":"0.0.0.0","ipc.[port_number].cost-provider.impl":"org.apache.hadoop.ipc.DefaultCostProvider","yarn.nodemanager.runtime.linux.runc.manifest-to-resources-plugin":"org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.runc.HdfsManifestToResourcesPlugin","yarn.nodemanager.remote-app-log-dir-include-older":"true","yarn.nodemanager.resource.memory.cgroups.swappiness":"0","ftp.stream-buffer-size":"4096","yarn.fail-fast":"false","yarn.nodemanager.runtime.linux.runc.layer-mounts-to-keep":"100","yarn.timeline-service.app-aggregation-interval-secs":"15","hadoop.security.group.mapping.ldap.search.filter.user":"(&(objectClass=user)(sAMAccountName={0}))","ipc.[port_number].weighted-cost.lockshared":"10","yarn.nodemanager.container-localizer.log.level":"INFO","yarn.timeline-service.address":"${yarn.timeline-service.hostname}:10200","mapreduce.job.ubertask.maxmaps":"9","fs.s3a.threads.keepalivetime":"60","mapreduce.jobhistory.webapp.rest-csrf.methods-to-ignore":"GET,OPTIONS,HEAD","mapreduce.task.files.preserve.failedtasks":"false","yarn.app.mapreduce.client.job.retry-interval":"2000","ha.failover-controller.graceful-fence.connection.retries":"1","fs.s3a.select.output.csv.quote.escape.character":"\\\\","yarn.resourcemanager.delegation.token.max-lifetime":"*********(redacted)","hadoop.kerberos.keytab.login.autorenewal.enabled":"false","yarn.timeline-service.client.drain-entities.timeout.ms":"2000","yarn.nodemanager.resource-plugins.fpga.vendor-plugin.class":"org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.fpga.IntelFpgaOpenclPlugin","yarn.resourcemanager.nodemanagers.heartbeat-interval-min-ms":"1000","yarn.timeline-service.entity-group-fs-store.summary-store":"org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore","mapreduce.reduce.cpu.vcores":"1","mapreduce.job.encrypted-intermediate-data.buffer.kb":"128","fs.client.resolve.remote.symlinks":"true","yarn.nodemanager.webapp.https.address":"0.0.0.0:8044","hadoop.http.cross-origin.allowed-origins":"*","mapreduce.job.encrypted-intermediate-data":"false","yarn.nodemanager.disk-health-checker.disk-utilization-threshold.enabled":"true","fs.s3a.executor.capacity":"16","yarn.timeline-service.entity-group-fs-store.retain-seconds":"604800","yarn.resourcemanager.metrics.runtime.buckets":"60,300,1440","yarn.timeline-service.generic-application-history.max-applications":"10000","yarn.nodemanager.local-dirs":"${hadoop.tmp.dir}/nm-local-dir","mapreduce.shuffle.connection-keep-alive.enable":"false","yarn.node-labels.configuration-type":"centralized","fs.s3a.path.style.access":"false","yarn.nodemanager.aux-services.mapreduce_shuffle.class":"org.apache.hadoop.mapred.ShuffleHandler","yarn.sharedcache.store.in-memory.staleness-period-mins":"10080","fs.adl.impl":"org.apache.hadoop.fs.adl.AdlFileSystem","yarn.resourcemanager.application.max-tags":"10","hadoop.domainname.resolver.impl":"org.apache.hadoop.net.DNSDomainNameResolver","yarn.resourcemanager.nodemanager.minimum.version":"NONE","mapreduce.jobhistory.webapp.xfs-filter.xframe-options":"SAMEORIGIN","yarn.app.mapreduce.am.staging-dir.erasurecoding.enabled":"false","net.topology.impl":"org.apache.hadoop.net.NetworkTopology","io.map.index.skip":"0","yarn.timeline-service.reader.webapp.https.address":"${yarn.timeline-service.webapp.https.address}","fs.ftp.data.connection.mode":"ACTIVE_LOCAL_DATA_CONNECTION_MODE","mapreduce.job.local-fs.single-disk-limit.check.kill-limit-exceed":"true","fs.azure.buffer.dir":"${hadoop.tmp.dir}/abfs","yarn.scheduler.maximum-allocation-vcores":"4","hadoop.http.cross-origin.allowed-headers":"X-Requested-With,Content-Type,Accept,Origin","yarn.nodemanager.log-aggregation.compression-type":"none","yarn.timeline-service.version":"1.0f","yarn.ipc.rpc.class":"org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC","mapreduce.reduce.maxattempts":"4","yarn.resourcemanager.system-metrics-publisher.timeline-server-v1.batch-size":"1000","hadoop.security.dns.log-slow-lookups.enabled":"false","mapreduce.job.committer.setup.cleanup.needed":"true","hadoop.security.secure.random.impl":"org.apache.hadoop.crypto.random.OpensslSecureRandom","mapreduce.job.running.reduce.limit":"0","fs.s3a.select.errors.include.sql":"false","fs.s3a.connection.request.timeout":"0","ipc.maximum.response.length":"134217728","yarn.resourcemanager.webapp.rest-csrf.methods-to-ignore":"GET,OPTIONS,HEAD","mapreduce.job.token.tracking.ids.enabled":"*********(redacted)","hadoop.caller.context.max.size":"128","yarn.nodemanager.runtime.linux.docker.host-pid-namespace.allowed":"false","yarn.nodemanager.runtime.linux.docker.delayed-removal.allowed":"false","hadoop.registry.system.acls":"sasl:yarn@, sasl:mapred@, sasl:hdfs@","yarn.nodemanager.recovery.dir":"${hadoop.tmp.dir}/yarn-nm-recovery","fs.s3a.fast.upload.buffer":"disk","mapreduce.jobhistory.intermediate-done-dir":"${yarn.app.mapreduce.am.staging-dir}/history/done_intermediate","yarn.app.mapreduce.shuffle.log.separate":"true","yarn.log-aggregation.debug.filesize":"104857600","fs.s3a.max.total.tasks":"32","fs.s3a.readahead.range":"64K","hadoop.http.authentication.simple.anonymous.allowed":"true","fs.s3a.attempts.maximum":"20","hadoop.registry.zk.connection.timeout.ms":"15000","yarn.resourcemanager.delegation-token-renewer.thread-count":"*********(redacted)","yarn.resourcemanager.delegation-token-renewer.thread-timeout":"*********(redacted)","yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size":"10000","yarn.nodemanager.aux-services.manifest.reload-ms":"0","yarn.nodemanager.emit-container-events":"true","yarn.resourcemanager.resource-profiles.enabled":"false","yarn.timeline-service.hbase-schema.prefix":"prod.","fs.azure.authorization":"false","mapreduce.map.log.level":"INFO","ha.failover-controller.active-standby-elector.zk.op.retries":"3","yarn.resourcemanager.decommissioning-nodes-watcher.poll-interval-secs":"20","mapreduce.output.fileoutputformat.compress.type":"RECORD","yarn.resourcemanager.leveldb-state-store.path":"${hadoop.tmp.dir}/yarn/system/rmstore","yarn.timeline-service.webapp.rest-csrf.custom-header":"X-XSRF-Header","mapreduce.ifile.readahead.bytes":"4194304","yarn.sharedcache.app-checker.class":"org.apache.hadoop.yarn.server.sharedcachemanager.RemoteAppChecker","yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users":"true","yarn.nodemanager.resource.detect-hardware-capabilities":"false","mapreduce.cluster.acls.enabled":"false","mapreduce.job.speculative.retry-after-no-speculate":"1000","fs.viewfs.overload.scheme.target.abfs.impl":"org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem","hadoop.security.group.mapping.ldap.search.group.hierarchy.levels":"0","yarn.resourcemanager.fs.state-store.retry-interval-ms":"1000","file.stream-buffer-size":"4096","yarn.resourcemanager.application-timeouts.monitor.interval-ms":"3000","mapreduce.map.output.compress.codec":"org.apache.hadoop.io.compress.DefaultCodec","mapreduce.map.speculative":"true","yarn.nodemanager.runtime.linux.runc.image-tag-to-manifest-plugin.hdfs-hash-file":"/runc-root/image-tag-to-hash","mapreduce.job.speculative.retry-after-speculate":"15000","yarn.nodemanager.linux-container-executor.cgroups.mount":"false","yarn.app.mapreduce.am.container.log.backups":"0","yarn.app.mapreduce.am.log.level":"INFO","yarn.nodemanager.runtime.linux.runc.image-tag-to-manifest-plugin":"org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.runc.ImageTagToManifestPlugin","io.bytes.per.checksum":"512","mapreduce.job.reduce.slowstart.completedmaps":"0.05","yarn.timeline-service.http-authentication.type":"simple","hadoop.security.group.mapping.ldap.search.attr.group.name":"cn","yarn.nodemanager.resource-plugins.fpga.allowed-fpga-devices":"auto","yarn.timeline-service.client.internal-timers-ttl-secs":"420","fs.s3a.select.output.csv.quote.character":"\"","hadoop.http.logs.enabled":"true","fs.s3a.block.size":"32M","yarn.sharedcache.client-server.address":"0.0.0.0:8045","yarn.nodemanager.logaggregation.threadpool-size-max":"100","yarn.resourcemanager.hostname":"0.0.0.0","yarn.resourcemanager.delegation.key.update-interval":"86400000","mapreduce.reduce.shuffle.fetch.retry.enabled":"${yarn.nodemanager.recovery.enabled}","mapreduce.map.memory.mb":"-1","mapreduce.task.skip.start.attempts":"2","fs.AbstractFileSystem.hdfs.impl":"org.apache.hadoop.fs.Hdfs","yarn.nodemanager.disk-health-checker.enable":"true","fs.s3a.select.output.csv.quote.fields":"always","ipc.client.tcpnodelay":"true","ipc.client.rpc-timeout.ms":"0","yarn.nodemanager.webapp.rest-csrf.methods-to-ignore":"GET,OPTIONS,HEAD","yarn.resourcemanager.delegation-token-renewer.thread-retry-max-attempts":"*********(redacted)","ipc.client.low-latency":"false","mapreduce.input.lineinputformat.linespermap":"1","yarn.router.interceptor.user.threadpool-size":"5","ipc.client.connect.max.retries.on.timeouts":"45","yarn.timeline-service.leveldb-timeline-store.read-cache-size":"104857600","fs.AbstractFileSystem.har.impl":"org.apache.hadoop.fs.HarFs","mapreduce.job.split.metainfo.maxsize":"10000000","yarn.am.liveness-monitor.expiry-interval-ms":"600000","yarn.resourcemanager.container-tokens.master-key-rolling-interval-secs":"*********(redacted)","yarn.timeline-service.entity-group-fs-store.app-cache-size":"10","yarn.nodemanager.runtime.linux.runc.hdfs-manifest-to-resources-plugin.stat-cache-timeout-interval-secs":"360","fs.s3a.socket.recv.buffer":"8192","rpc.metrics.timeunit":"MILLISECONDS","yarn.resourcemanager.resource-tracker.address":"${yarn.resourcemanager.hostname}:8031","yarn.nodemanager.node-labels.provider.fetch-timeout-ms":"1200000","mapreduce.job.heap.memory-mb.ratio":"0.8","yarn.resourcemanager.leveldb-state-store.compaction-interval-secs":"3600","yarn.resourcemanager.webapp.rest-csrf.custom-header":"X-XSRF-Header","yarn.nodemanager.pluggable-device-framework.enabled":"false","yarn.scheduler.configuration.fs.path":"file://${hadoop.tmp.dir}/yarn/system/schedconf","mapreduce.client.output.filter":"FAILED","hadoop.http.filter.initializers":"org.apache.hadoop.http.lib.StaticUserWebFilter","mapreduce.reduce.memory.mb":"-1","yarn.timeline-service.hostname":"0.0.0.0","file.replication":"1","yarn.nodemanager.container-metrics.unregister-delay-ms":"10000","yarn.nodemanager.container-metrics.period-ms":"-1","mapreduce.fileoutputcommitter.task.cleanup.enabled":"false","yarn.nodemanager.log.retain-seconds":"10800","yarn.timeline-service.entity-group-fs-store.cleaner-interval-seconds":"3600","ipc.[port_number].callqueue.impl":"java.util.concurrent.LinkedBlockingQueue","yarn.resourcemanager.keytab":"/etc/krb5.keytab","hadoop.security.group.mapping.providers.combined":"true","mapreduce.reduce.merge.inmem.threshold":"1000","yarn.timeline-service.recovery.enabled":"false","fs.azure.saskey.usecontainersaskeyforallaccess":"true","yarn.sharedcache.nm.uploader.thread-count":"20","yarn.resourcemanager.nodemanager-graceful-decommission-timeout-secs":"3600","ipc.[port_number].weighted-cost.lockfree":"1","mapreduce.shuffle.ssl.enabled":"false","yarn.timeline-service.hbase.coprocessor.app-final-value-retention-milliseconds":"259200000","yarn.nodemanager.opportunistic-containers-max-queue-length":"0","yarn.resourcemanager.state-store.max-completed-applications":"${yarn.resourcemanager.max-completed-applications}","mapreduce.job.speculative.minimum-allowed-tasks":"10","fs.s3a.aws.credentials.provider":"\n    org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider,\n    org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider,\n    com.amazonaws.auth.EnvironmentVariableCredentialsProvider,\n    org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider\n  ","yarn.log-aggregation.retain-seconds":"-1","yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb":"0","mapreduce.jobhistory.max-age-ms":"604800000","hadoop.http.cross-origin.allowed-methods":"GET,POST,HEAD","yarn.resourcemanager.opportunistic-container-allocation.enabled":"false","mapreduce.jobhistory.webapp.address":"0.0.0.0:19888","hadoop.system.tags":"YARN,HDFS,NAMENODE,DATANODE,REQUIRED,SECURITY,KERBEROS,PERFORMANCE,CLIENT\n      ,SERVER,DEBUG,DEPRECATED,COMMON,OPTIONAL","yarn.log-aggregation.file-controller.TFile.class":"org.apache.hadoop.yarn.logaggregation.filecontroller.tfile.LogAggregationTFileController","yarn.client.nodemanager-connect.max-wait-ms":"180000","yarn.resourcemanager.webapp.address":"${yarn.resourcemanager.hostname}:8088","mapreduce.jobhistory.recovery.enable":"false","mapreduce.reduce.shuffle.parallelcopies":"5","fs.AbstractFileSystem.webhdfs.impl":"org.apache.hadoop.fs.WebHdfs","fs.trash.interval":"0","yarn.app.mapreduce.client.max-retries":"3","hadoop.security.authentication":"simple","mapreduce.task.profile.reduce.params":"${mapreduce.task.profile.params}","yarn.app.mapreduce.am.resource.mb":"1536","mapreduce.input.fileinputformat.list-status.num-threads":"1","yarn.nodemanager.container-executor.class":"org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor","io.mapfile.bloom.size":"1048576","yarn.timeline-service.ttl-ms":"604800000","yarn.resourcemanager.nm-container-queuing.min-queue-length":"5","yarn.nodemanager.resource.cpu-vcores":"-1","mapreduce.job.reduces":"1","fs.s3a.multipart.size":"64M","fs.s3a.select.input.csv.comment.marker":"#","yarn.scheduler.minimum-allocation-vcores":"1","mapreduce.job.speculative.speculative-cap-total-tasks":"0.01","hadoop.ssl.client.conf":"ssl-client.xml","mapreduce.job.queuename":"default","mapreduce.job.encrypted-intermediate-data-key-size-bits":"128","fs.s3a.metadatastore.authoritative":"false","ipc.[port_number].weighted-cost.response":"1","yarn.nodemanager.webapp.xfs-filter.xframe-options":"SAMEORIGIN","ha.health-monitor.sleep-after-disconnect.ms":"1000","yarn.app.mapreduce.shuffle.log.limit.kb":"0","hadoop.security.group.mapping":"org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback","yarn.client.application-client-protocol.poll-timeout-ms":"-1","mapreduce.jobhistory.jhist.format":"binary","mapreduce.task.stuck.timeout-ms":"600000","yarn.resourcemanager.application.max-tag.length":"100","yarn.resourcemanager.ha.enabled":"false","dfs.client.ignore.namenode.default.kms.uri":"false","hadoop.http.staticuser.user":"dr.who","mapreduce.task.exit.timeout.check-interval-ms":"20000","mapreduce.jobhistory.intermediate-user-done-dir.permissions":"770","mapreduce.task.exit.timeout":"60000","yarn.nodemanager.linux-container-executor.resources-handler.class":"org.apache.hadoop.yarn.server.nodemanager.util.DefaultLCEResourcesHandler","mapreduce.reduce.shuffle.memory.limit.percent":"0.25","yarn.resourcemanager.reservation-system.enable":"false","mapreduce.map.output.compress":"false","ha.zookeeper.acl":"world:anyone:rwcda","ipc.server.max.connections":"0","yarn.nodemanager.runtime.linux.docker.default-container-network":"host","yarn.router.webapp.address":"0.0.0.0:8089","yarn.scheduler.maximum-allocation-mb":"8192","yarn.resourcemanager.scheduler.monitor.policies":"org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy","yarn.sharedcache.cleaner.period-mins":"1440","yarn.nodemanager.resource-plugins.gpu.docker-plugin.nvidia-docker-v1.endpoint":"http://localhost:3476/v1.0/docker/cli","yarn.app.mapreduce.am.container.log.limit.kb":"0","ipc.client.connect.retry.interval":"1000","yarn.timeline-service.http-cross-origin.enabled":"false","fs.wasbs.impl":"org.apache.hadoop.fs.azure.NativeAzureFileSystem$Secure","yarn.resourcemanager.nodemanagers.heartbeat-interval-max-ms":"1000","yarn.federation.subcluster-resolver.class":"org.apache.hadoop.yarn.server.federation.resolver.DefaultSubClusterResolverImpl","yarn.resourcemanager.zk-state-store.parent-path":"/rmstore","fs.s3a.select.input.csv.field.delimiter":",","mapreduce.jobhistory.cleaner.enable":"true","yarn.timeline-service.client.fd-flush-interval-secs":"10","hadoop.security.kms.client.encrypted.key.cache.expiry":"43200000","yarn.client.nodemanager-client-async.thread-pool-max-size":"500","mapreduce.map.maxattempts":"4","yarn.resourcemanager.nm-container-queuing.sorting-nodes-interval-ms":"1000","fs.s3a.committer.staging.tmp.path":"tmp/staging","yarn.nodemanager.sleep-delay-before-sigkill.ms":"250","yarn.resourcemanager.nm-container-queuing.min-queue-wait-time-ms":"10","mapreduce.job.end-notification.retry.attempts":"0","yarn.nodemanager.resource.count-logical-processors-as-cores":"false","hadoop.registry.zk.root":"/registry","adl.feature.ownerandgroup.enableupn":"false","yarn.resourcemanager.zk-max-znode-size.bytes":"1048576","mapreduce.job.reduce.shuffle.consumer.plugin.class":"org.apache.hadoop.mapreduce.task.reduce.Shuffle","yarn.resourcemanager.delayed.delegation-token.removal-interval-ms":"*********(redacted)","yarn.nodemanager.localizer.cache.target-size-mb":"10240","fs.s3a.committer.staging.conflict-mode":"append","mapreduce.client.libjars.wildcard":"true","fs.s3a.committer.staging.unique-filenames":"true","yarn.nodemanager.node-attributes.provider.fetch-timeout-ms":"1200000","fs.s3a.list.version":"2","ftp.client-write-packet-size":"65536","ipc.[port_number].weighted-cost.lockexclusive":"100","fs.AbstractFileSystem.adl.impl":"org.apache.hadoop.fs.adl.Adl","yarn.nodemanager.container-log-monitor.enable":"false","hadoop.security.key.default.cipher":"AES/CTR/NoPadding","yarn.client.failover-retries":"0","fs.s3a.multipart.purge.age":"86400","mapreduce.job.local-fs.single-disk-limit.check.interval-ms":"5000","net.topology.node.switch.mapping.impl":"org.apache.hadoop.net.ScriptBasedMapping","yarn.nodemanager.amrmproxy.address":"0.0.0.0:8049","ipc.server.listen.queue.size":"256","ipc.[port_number].decay-scheduler.period-ms":"5000","yarn.nodemanager.runtime.linux.runc.image-tag-to-manifest-plugin.cache-refresh-interval-secs":"60","map.sort.class":"org.apache.hadoop.util.QuickSort","fs.viewfs.rename.strategy":"SAME_MOUNTPOINT","hadoop.security.kms.client.authentication.retry-count":"1","fs.permissions.umask-mode":"022","fs.s3a.assumed.role.credentials.provider":"org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider","yarn.nodemanager.runtime.linux.runc.privileged-containers.allowed":"false","yarn.nodemanager.vmem-check-enabled":"true","yarn.nodemanager.numa-awareness.enabled":"false","yarn.nodemanager.recovery.compaction-interval-secs":"3600","yarn.app.mapreduce.client-am.ipc.max-retries":"3","yarn.resourcemanager.system-metrics-publisher.timeline-server-v1.interval-seconds":"60","yarn.federation.registry.base-dir":"yarnfederation/","yarn.nodemanager.health-checker.run-before-startup":"false","mapreduce.job.max.map":"-1","mapreduce.job.local-fs.single-disk-limit.bytes":"-1","mapreduce.shuffle.pathcache.concurrency-level":"16","mapreduce.job.ubertask.maxreduces":"1","mapreduce.shuffle.pathcache.max-weight":"10485760","hadoop.security.kms.client.encrypted.key.cache.size":"500","hadoop.security.java.secure.random.algorithm":"SHA1PRNG","ha.failover-controller.cli-check.rpc-timeout.ms":"20000","mapreduce.jobhistory.jobname.limit":"50","fs.s3a.select.input.compression":"none","yarn.client.nodemanager-connect.retry-interval-ms":"10000","ipc.[port_number].scheduler.priority.levels":"4","yarn.timeline-service.state-store-class":"org.apache.hadoop.yarn.server.timeline.recovery.LeveldbTimelineStateStore","yarn.nodemanager.env-whitelist":"JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_HOME,PATH,LANG,TZ","yarn.sharedcache.nested-level":"3","yarn.timeline-service.webapp.rest-csrf.methods-to-ignore":"GET,OPTIONS,HEAD","fs.azure.user.agent.prefix":"unknown","yarn.resourcemanager.zk-delegation-token-node.split-index":"*********(redacted)","yarn.nodemanager.numa-awareness.read-topology":"false","yarn.nodemanager.webapp.address":"${yarn.nodemanager.hostname}:8042","rpc.metrics.quantile.enable":"false","yarn.registry.class":"org.apache.hadoop.registry.client.impl.FSRegistryOperationsService","mapreduce.jobhistory.admin.acl":"*","yarn.resourcemanager.system-metrics-publisher.dispatcher.pool-size":"10","yarn.scheduler.queue-placement-rules":"user-group","hadoop.http.authentication.kerberos.keytab":"${user.home}/hadoop.keytab","yarn.resourcemanager.recovery.enabled":"false","fs.s3a.select.input.csv.header":"none","yarn.nodemanager.runtime.linux.runc.hdfs-manifest-to-resources-plugin.stat-cache-size":"500","yarn.timeline-service.webapp.rest-csrf.enabled":"false","yarn.nodemanager.disk-health-checker.min-free-space-per-disk-watermark-high-mb":"0"},"System Properties":{"java.io.tmpdir":"/tmp","line.separator":"\n","path.separator":":","sun.management.compiler":"HotSpot 64-Bit Tiered Compilers","SPARK_SUBMIT":"true","sun.cpu.endian":"little","java.specification.maintenance.version":"5","java.specification.version":"1.8","java.vm.specification.name":"Java Virtual Machine Specification","java.vendor":"Private Build","java.vm.specification.version":"1.8","user.home":"/root","file.encoding.pkg":"sun.io","sun.arch.data.model":"64","sun.boot.library.path":"/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64","user.dir":"/root","java.library.path":"/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib","sun.cpu.isalist":"","os.arch":"amd64","java.vm.version":"25.382-b05","jetty.git.hash":"da9a0b30691a45daf90a9f17b5defa2f1434f882","java.endorsed.dirs":"/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed","java.runtime.version":"1.8.0_382-8u382-ga-1~22.04.1-b05","java.vm.info":"mixed mode","java.ext.dirs":"/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/usr/java/packages/lib/ext","java.runtime.name":"OpenJDK Runtime Environment","file.separator":"/","java.class.version":"52.0","java.specification.name":"Java Platform API Specification","sun.boot.class.path":"/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jfr.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/classes","file.encoding":"UTF-8","jdk.reflect.useDirectMethodHandle":"false","user.timezone":"Asia/Seoul","java.specification.vendor":"Oracle Corporation","sun.java.launcher":"SUN_STANDARD","os.version":"5.15.0-89-generic","sun.os.patch.level":"unknown","java.vm.specification.vendor":"Oracle Corporation","user.country":"US","sun.jnu.encoding":"UTF-8","user.language":"en","java.vendor.url":"http://java.oracle.com/","java.awt.printerjob":"sun.print.PSPrinterJob","java.awt.graphicsenv":"sun.awt.X11GraphicsEnvironment","awt.toolkit":"sun.awt.X11.XToolkit","os.name":"Linux","java.vm.vendor":"Private Build","java.vendor.url.bug":"http://bugreport.sun.com/bugreport/","user.name":"root","java.vm.name":"OpenJDK 64-Bit Server VM","sun.java.command":"org.apache.spark.deploy.SparkSubmit --master spark://30.0.0.2:7077 --conf spark.executor.extraClassPath=/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar --conf spark.driver.extraClassPath=/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar --conf spark.shuffle.manager=com.nvidia.spark.rapids.spark341.RapidsShuffleManager --packages org.apache.spark:spark-sql-kafka-0-10_2.12:3.2.0 --jars /opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar --files /root/spark-3.4.1-bin-hadoop3/examples/src/main/scripts/getGpusResources.sh /root/CNN_TensorFlow_Classify.py","java.home":"/usr/lib/jvm/java-8-openjdk-amd64/jre","java.version":"1.8.0_382","sun.io.unicode.encoding":"UnicodeLittle"},"Metrics Properties":{"*.sink.servlet.class":"org.apache.spark.metrics.sink.MetricsServlet","*.sink.servlet.path":"/metrics/json","applications.sink.servlet.path":"/metrics/applications/json","master.sink.servlet.path":"/metrics/master/json"},"Classpath Entries":{"/root/spark-3.4.1-bin-hadoop3/jars/gson-2.2.4.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-jdbc-2.3.9.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.apache.hadoop_hadoop-client-api-3.3.1.jar":"Added By User","spark://30.0.0.2:41803/files/org.spark-project.spark_unused-1.0.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/jakarta.xml.bind-api-2.3.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jaxb-runtime-2.3.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/ST4-4.0.4.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-pool-1.5.4.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.xerial.snappy_snappy-java-1.1.8.4.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/httpclient-4.5.14.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/json4s-jackson_2.12-3.7.0-M11.jar":"System Classpath","spark://30.0.0.2:41803/files/org.apache.spark_spark-token-provider-kafka-0-10_2.12-3.2.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/hive-shims-0.23-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-collections-3.2.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-core-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/janino-3.1.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-streaming_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jackson-datatype-jsr310-2.14.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-metastore-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-tags_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-buffer-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jakarta.annotation-api-1.3.5.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-serde-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jersey-server-2.36.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-shims-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hadoop-client-api-3.3.4.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/log4j-core-2.19.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-certificates-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/curator-client-2.13.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/mesos-1.4.3-shaded-protobuf.jar":"System Classpath","spark://30.0.0.2:41803/files/org.apache.kafka_kafka-clients-2.8.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/netty-common-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/datanucleus-api-jdo-4.2.4.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-rbac-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-crypto-1.1.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/osgi-resource-locator-1.0.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/chill_2.12-0.10.0.jar":"System Classpath","spark://30.0.0.2:41803/files/org.apache.spark_spark-sql-kafka-0-10_2.12-3.2.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/arrow-memory-netty-11.0.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-launcher_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/super-csv-2.2.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jersey-container-servlet-2.36.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-apiextensions-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/ivy-2.5.1.jar":"System Classpath","spark://30.0.0.2:41803/files/org.slf4j_slf4j-api-1.7.30.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-autoscaling-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/lapack-3.0.3.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.apache.kafka_kafka-clients-2.8.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/netty-transport-classes-kqueue-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/httpcore-4.4.16.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-text-1.10.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-hive-thriftserver_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-codec-http-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/transaction-api-1.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jul-to-slf4j-2.0.6.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/scala-parser-combinators_2.12-2.1.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-all-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/avro-mapred-1.11.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/slf4j-api-2.0.6.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.lz4_lz4-java-1.7.1.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/netty-transport-native-epoll-4.1.87.Final-linux-aarch_64.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-storage-api-2.8.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/compress-lzf-1.1.2.jar":"System Classpath","/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-resolver-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/audience-annotations-0.5.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jersey-client-2.36.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/libfb303-0.9.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-repl_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jdo-api-3.0.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/json4s-core_2.12-3.7.0-M11.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/xz-1.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/zjsonpatch-0.3.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jakarta.ws.rs-api-2.1.6.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-policy-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-metrics-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-scheduling-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jodd-core-3.5.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/json-1.8.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/libthrift-0.12.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/py4j-0.10.9.7.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/leveldbjni-all-1.8.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-compress-1.22.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-codec-1.15.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/zstd-jni-1.5.2-5.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/arpack-3.0.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-lang3-3.12.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/arpack_combined_all-0.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/breeze-macros_2.12-2.1.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/paranamer-2.8.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/minlog-1.3.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/log4j-slf4j2-impl-2.19.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/scala-compiler-2.12.17.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-codec-http2-4.1.87.Final.jar":"System Classpath","spark://30.0.0.2:41803/jars/commons-logging_commons-logging-1.1.3.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/snakeyaml-1.33.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-collections4-4.4.jar":"System Classpath","spark://30.0.0.2:41803/files/org.lz4_lz4-java-1.7.1.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/arrow-memory-core-11.0.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-shims-common-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/bonecp-0.8.0.RELEASE.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/oro-2.0.8.jar":"System Classpath","spark://30.0.0.2:41803/files/org.apache.hadoop_hadoop-client-api-3.3.1.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/spark-core_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/JTransforms-3.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-shims-scheduler-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jakarta.validation-api-2.0.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-llap-common-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-networking-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jakarta.inject-2.6.1.jar":"System Classpath","spark://30.0.0.2:41803/files/getGpusResources.sh":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/stax-api-1.0.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/metrics-graphite-4.2.15.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-storageclass-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jackson-annotations-2.14.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-codec-socks-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jersey-container-servlet-core-2.36.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jpam-1.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-mllib-local_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jta-1.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/JLargeArrays-1.5.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/cats-kernel_2.12-2.1.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/rocksdbjni-7.9.2.jar":"System Classpath","spark://30.0.0.2:41803/files/commons-logging_commons-logging-1.1.3.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-batch-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-network-shuffle_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/logging-interceptor-3.12.12.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jackson-module-scala_2.12-2.14.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jersey-common-2.36.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/log4j-api-2.19.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-dbcp-1.4.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/zookeeper-jute-3.6.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jsr305-3.0.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/xbean-asm9-shaded-4.22.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/javax.jdo-3.2.0-m3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/shims-0.9.38.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-client-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spire_2.12-0.17.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-node-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/RoaringBitmap-0.9.38.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/tink-1.7.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/snappy-java-1.1.10.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-client-api-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/curator-framework-2.13.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-apps-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-cli-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-extensions-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/stream-2.9.6.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jcl-over-slf4j-2.0.6.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jline-2.14.6.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/opencsv-2.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hadoop-client-runtime-3.3.4.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/avro-ipc-1.11.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/datanucleus-rdbms-4.1.19.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-codec-4.1.87.Final.jar":"System Classpath","spark://30.0.0.2:41803/files/org.xerial.snappy_snappy-java-1.1.8.4.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/hk2-utils-2.6.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/aircompressor-0.21.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/parquet-common-1.12.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jackson-databind-2.14.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/activation-1.1.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jackson-core-asl-1.9.13.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-network-common_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-sql_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/pickle-1.3.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.slf4j_slf4j-api-1.7.30.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/spire-util_2.12-0.17.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-catalyst_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/log4j-1.2-api-2.19.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-discovery-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-compiler-3.1.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-hive_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-mllib_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jakarta.servlet-api-4.0.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hadoop-yarn-server-web-proxy-3.3.4.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-coordination-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/istack-commons-runtime-3.0.8.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-admissionregistration-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jackson-core-2.14.2.jar":"System Classpath","spark://30.0.0.2:41803/jars/com.google.code.findbugs_jsr305-3.0.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/hive-common-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jackson-dataformat-yaml-2.14.2.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.apache.spark_spark-token-provider-kafka-0-10_2.12-3.2.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/javassist-3.25.0-GA.jar":"System Classpath","spark://30.0.0.2:41803/files/org.apache.htrace_htrace-core4-4.1.0-incubating.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/netty-transport-native-epoll-4.1.87.Final-linux-x86_64.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/protobuf-java-2.5.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/jersey-hk2-2.36.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/flatbuffers-java-1.12.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/conf/":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kryo-shaded-4.0.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-transport-native-unix-common-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-handler-proxy-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/scala-reflect-2.12.17.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-flowcontrol-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-logging-1.1.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-mesos_2.12-3.4.1.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.spark-project.spark_unused-1.0.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/annotations-17.0.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/parquet-column-1.12.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/derby-10.14.2.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-transport-native-kqueue-4.1.87.Final-osx-x86_64.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/algebra_2.12-2.0.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/blas-3.0.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/scala-collection-compat_2.12-2.7.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/arrow-format-11.0.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/arrow-vector-11.0.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/orc-core-1.8.4-shaded-protobuf.jar":"System Classpath","spark://30.0.0.2:41803/files/org.apache.commons_commons-pool2-2.6.2.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/orc-shims-1.8.4.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-io-2.11.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-events-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/metrics-core-4.2.15.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-math3-3.6.1.jar":"System Classpath","spark://30.0.0.2:41803/files/org.apache.hadoop_hadoop-client-runtime-3.3.1.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/parquet-hadoop-1.12.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spire-macros_2.12-0.17.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-handler-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/breeze_2.12-2.1.0.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.apache.hadoop_hadoop-client-runtime-3.3.1.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/jackson-mapper-asl-1.9.13.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.apache.spark_spark-sql-kafka-0-10_2.12-3.2.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/okio-1.15.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-transport-4.1.87.Final.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/scala-xml_2.12-2.1.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/okhttp-3.12.12.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-transport-native-kqueue-4.1.87.Final-osx-aarch_64.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-beeline-2.3.9.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/orc-mapreduce-1.8.4-shaded-protobuf.jar":"System Classpath","spark://30.0.0.2:41803/jars/rapids-4-spark_2.12-23.10.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/metrics-json-4.2.15.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-graphx_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/metrics-jvm-4.2.15.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-yarn_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/json4s-ast_2.12-3.7.0-M11.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-gatewayapi-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/joda-time-2.12.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/scala-library-2.12.17.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/zookeeper-3.6.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/guava-14.0.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/parquet-format-structures-1.12.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/antlr-runtime-3.5.2.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.apache.commons_commons-pool2-2.6.2.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/hive-exec-2.3.9-core.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/antlr4-runtime-4.9.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/json4s-scalap_2.12-3.7.0-M11.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/chill-java-0.10.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-kvstore_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/objenesis-3.2.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spire-platform_2.12-0.17.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hk2-locator-2.6.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/netty-transport-classes-epoll-4.1.87.Final.jar":"System Classpath","spark://30.0.0.2:41803/jars/org.apache.htrace_htrace-core4-4.1.0-incubating.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/HikariCP-2.5.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-httpclient-okhttp-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/parquet-jackson-1.12.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hadoop-shaded-guava-1.1.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hk2-api-2.6.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-cli-1.5.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/datanucleus-core-4.1.17.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/threeten-extra-1.7.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/parquet-encoding-1.12.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/metrics-jmx-4.2.15.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/hive-service-rpc-3.1.3.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-kubernetes_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/javolution-5.5.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/curator-recipes-2.13.0.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/aopalliance-repackaged-2.6.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-unsafe_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/spark-sketch_2.12-3.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/commons-lang-2.6.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/kubernetes-model-common-6.4.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/univocity-parsers-2.9.1.jar":"System Classpath","/root/spark-3.4.1-bin-hadoop3/jars/lz4-java-1.8.0.jar":"System Classpath","spark://30.0.0.2:41803/files/com.google.code.findbugs_jsr305-3.0.0.jar":"Added By User","/root/spark-3.4.1-bin-hadoop3/jars/avro-1.11.1.jar":"System Classpath"}}
{"Event":"SparkListenerApplicationStart","App Name":"Linear_Regression","App ID":"app-20231123131052-0002","Timestamp":1700712650869,"User":"root"}
{"Event":"SparkListenerExecutorAdded","Timestamp":1700712655103,"Executor ID":"1","Executor Info":{"Host":"30.0.0.2","Total Cores":8,"Log Urls":{"stdout":"http://30.0.0.2:8081/logPage/?appId=app-20231123131052-0002&executorId=1&logType=stdout","stderr":"http://30.0.0.2:8081/logPage/?appId=app-20231123131052-0002&executorId=1&logType=stderr"},"Attributes":{},"Resources":{"gpu":{"name":"gpu","addresses":["0"]}},"Resource Profile Id":0,"Registration Time":1700712655103}}
{"Event":"SparkListenerBlockManagerAdded","Block Manager ID":{"Executor ID":"1","Host":"30.0.0.2","Port":46221},"Maximum Memory":73112066457,"Timestamp":1700712655173,"Maximum Onheap Memory":73112066457,"Maximum Offheap Memory":0}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712655638,"Executor ID":"0","Removed Reason":"Command exited with code 1"}
{"Event":"org.apache.spark.sql.execution.ui.SparkListenerSQLExecutionStart","executionId":0,"rootExecutionId":0,"description":"showString at NativeMethodAccessorImpl.java:0","details":"org.apache.spark.sql.Dataset.showString(Dataset.scala:323)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:498)\npy4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\npy4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)\npy4j.Gateway.invoke(Gateway.java:282)\npy4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\npy4j.commands.CallCommand.execute(CallCommand.java:79)\npy4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)\npy4j.ClientServerConnection.run(ClientServerConnection.java:106)\njava.lang.Thread.run(Thread.java:750)","physicalPlanDescription":"== Physical Plan ==\nCollectLimit (7)\n+- GpuColumnarToRow (6)\n   +- GpuProject (5)\n      +- GpuRowToColumnar (4)\n         +- ArrowEvalPython (3)\n            +- * LocalLimit (2)\n               +- Scan image  (1)\n\n\n(1) Scan image \nOutput [1]: [image#0]\nBatched: false\nLocation: InMemoryFileIndex [file:/root/imagenette2/train/n02102040]\nReadSchema: struct<image:struct<origin:string,height:int,width:int,nChannels:int,mode:int,data:binary>>\n\n(2) LocalLimit [codegen id : 1]\nInput [1]: [image#0]\nArguments: 21\n\n(3) ArrowEvalPython\nInput [1]: [image#0]\nArguments: [predict(resize_image(image#0)#2)#6], [pythonUDF0#12], 204\n\n(4) GpuRowToColumnar\nInput [2]: [image#0, pythonUDF0#12]\nArguments: targetsize(1073741824)\n\n(5) GpuProject\nInput [2]: [image#0, pythonUDF0#12]\nArguments: [cast(pythonUDF0#12 as string) AS pred#10]\n\n(6) GpuColumnarToRow\nInput [1]: [pred#10]\nArguments: false\n\n(7) CollectLimit\nInput [1]: [pred#10]\nArguments: 21\n\n","sparkPlanInfo":{"nodeName":"CollectLimit","simpleString":"CollectLimit 21","children":[{"nodeName":"GpuColumnarToRow","simpleString":"GpuColumnarToRow false","children":[{"nodeName":"GpuProject","simpleString":"GpuProject [cast(pythonUDF0#12 as string) AS pred#10]","children":[{"nodeName":"GpuRowToColumnar","simpleString":"GpuRowToColumnar targetsize(1073741824)","children":[{"nodeName":"ArrowEvalPython","simpleString":"ArrowEvalPython [predict(resize_image(image#0)#2)#6], [pythonUDF0#12], 204","children":[{"nodeName":"WholeStageCodegen (1)","simpleString":"WholeStageCodegen (1)","children":[{"nodeName":"LocalLimit","simpleString":"LocalLimit 21","children":[{"nodeName":"InputAdapter","simpleString":"InputAdapter","children":[{"nodeName":"Scan image ","simpleString":"FileScan image [image#0] Batched: false, DataFilters: [], Format: org.apache.spark.ml.source.image.ImageFileFormat@58424b47, Location: InMemoryFileIndex(1 paths)[file:/root/imagenette2/train/n02102040], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<image:struct<origin:string,height:int,width:int,nChannels:int,mode:int,data:binary>>","children":[],"metadata":{"Location":"InMemoryFileIndex(1 paths)[file:/root/imagenette2/train/n02102040]","ReadSchema":"struct<image:struct<origin:string,height:int,width:int,nChannels:int,mode:int,data:binary>>","Format":"org.apache.spark.ml.source.image.ImageFileFormat@58424b47","Batched":"false","PartitionFilters":"[]","PushedFilters":"[]","DataFilters":"[]"},"metrics":[{"name":"number of output rows","accumulatorId":52,"metricType":"sum"},{"name":"number of files read","accumulatorId":53,"metricType":"sum"},{"name":"metadata time","accumulatorId":54,"metricType":"timing"},{"name":"size of files read","accumulatorId":55,"metricType":"size"}]}],"metadata":{},"metrics":[]}],"metadata":{},"metrics":[]}],"metadata":{},"metrics":[{"name":"duration","accumulatorId":51,"metricType":"timing"}]}],"metadata":{},"metrics":[{"name":"data sent to Python workers","accumulatorId":23,"metricType":"size"},{"name":"data returned from Python workers","accumulatorId":24,"metricType":"size"},{"name":"number of output rows","accumulatorId":25,"metricType":"sum"}]}],"metadata":{},"metrics":[{"name":"stream time","accumulatorId":50,"metricType":"nsTiming"},{"name":"op time","accumulatorId":49,"metricType":"nsTiming"}]}],"metadata":{},"metrics":[{"name":"op time","accumulatorId":48,"metricType":"nsTiming"}]}],"metadata":{},"metrics":[{"name":"op time","accumulatorId":46,"metricType":"nsTiming"},{"name":"stream time","accumulatorId":47,"metricType":"nsTiming"}]}],"metadata":{},"metrics":[{"name":"shuffle records written","accumulatorId":44,"metricType":"sum"},{"name":"local merged chunks fetched","accumulatorId":38,"metricType":"sum"},{"name":"shuffle write time","accumulatorId":45,"metricType":"nsTiming"},{"name":"remote merged bytes read","accumulatorId":39,"metricType":"size"},{"name":"local merged blocks fetched","accumulatorId":36,"metricType":"sum"},{"name":"corrupt merged block chunks","accumulatorId":33,"metricType":"sum"},{"name":"remote merged reqs duration","accumulatorId":42,"metricType":"timing"},{"name":"remote merged blocks fetched","accumulatorId":35,"metricType":"sum"},{"name":"records read","accumulatorId":32,"metricType":"sum"},{"name":"local bytes read","accumulatorId":30,"metricType":"size"},{"name":"fetch wait time","accumulatorId":31,"metricType":"timing"},{"name":"remote bytes read","accumulatorId":28,"metricType":"size"},{"name":"merged fetch fallback count","accumulatorId":34,"metricType":"sum"},{"name":"local blocks read","accumulatorId":27,"metricType":"sum"},{"name":"remote merged chunks fetched","accumulatorId":37,"metricType":"sum"},{"name":"remote blocks read","accumulatorId":26,"metricType":"sum"},{"name":"local merged bytes read","accumulatorId":40,"metricType":"size"},{"name":"remote reqs duration","accumulatorId":41,"metricType":"timing"},{"name":"remote bytes read to disk","accumulatorId":29,"metricType":"size"},{"name":"shuffle bytes written","accumulatorId":43,"metricType":"size"}]},"time":1700712657391,"modifiedConfigs":{}}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712657659,"Executor ID":"2","Removed Reason":"Command exited with code 1"}
{"Event":"org.apache.spark.sql.execution.ui.SparkListenerDriverAccumUpdates","executionId":0,"accumUpdates":[[53,955],[54,7],[55,111552831]]}
{"Event":"SparkListenerJobStart","Job ID":0,"Submission Time":1700712658136,"Stage Infos":[{"Stage ID":0,"Stage Attempt ID":0,"Stage Name":"showString at NativeMethodAccessorImpl.java:0","Number of Tasks":1,"RDD Info":[{"RDD ID":10,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"8\",\"name\":\"mapPartitionsInternal\"}","Callsite":"showString at NativeMethodAccessorImpl.java:0","Parent IDs":[9],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":1,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"7\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[0],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":9,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"0\",\"name\":\"GpuColumnarToRow\"}","Callsite":"mapPartitions at GpuColumnarToRowExec.scala:368","Parent IDs":[8],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":8,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"1\",\"name\":\"GpuProject\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[7],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":2,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"4\",\"name\":\"WholeStageCodegen (1)\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[1],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":6,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"2\",\"name\":\"GpuRowToColumnar\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[5],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":4,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"3\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[3],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":0,"Name":"FileScanRDD","Scope":"{\"id\":\"7\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":3,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"3\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[2],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":5,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"2\",\"name\":\"GpuRowToColumnar\"}","Callsite":"mapPartitions at GpuRowToColumnarExec.scala:913","Parent IDs":[4],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":7,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"1\",\"name\":\"GpuProject\"}","Callsite":"map at basicPhysicalOperators.scala:380","Parent IDs":[6],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0}],"Parent IDs":[],"Details":"org.apache.spark.sql.Dataset.showString(Dataset.scala:323)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:498)\npy4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\npy4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)\npy4j.Gateway.invoke(Gateway.java:282)\npy4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\npy4j.commands.CallCommand.execute(CallCommand.java:79)\npy4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)\npy4j.ClientServerConnection.run(ClientServerConnection.java:106)\njava.lang.Thread.run(Thread.java:750)","Accumulables":[],"Resource Profile Id":0,"Shuffle Push Enabled":false,"Shuffle Push Mergers Count":0}],"Stage IDs":[0],"Properties":{"spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.concurrentGpuTasks":"4","spark.shuffle.manager":"com.nvidia.spark.rapids.spark341.RapidsShuffleManager","spark.driver.port":"41803","spark.sql.files.maxPartitionBytes":"512m","spark.executor.cores":"8","spark.sql.execution.root.id":"0","spark.rapids.sql.python.gpu.enabled":"true","spark.repl.local.jars":"*********(redacted)","spark.plugins":"com.nvidia.spark.SQLPlugin","spark.files":"*********(redacted)","spark.task.resource.gpu.amount":"0.125","spark.submit.pyFiles":"*********(redacted)","spark.executor.memory":"128G","spark.executor.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.executor.id":"driver","spark.app.startTime":"1700712650869","spark.rdd.compress":"True","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.driver.user.timezone":"Asia/Seoul","spark.executor.resource.gpu.amount":"1","spark.sql.extensions":"com.nvidia.spark.rapids.SQLExecPlugin,com.nvidia.spark.udf.Plugin,com.nvidia.spark.rapids.optimizer.SQLOptimizerPlugin","spark.rapids.sql.explain":"NOT_ON_GPU","spark.master":"spark://30.0.0.2:7077","spark.serializer.objectStreamReset":"100","spark.app.initial.jar.urls":"*********(redacted)","spark.app.initial.file.urls":"*********(redacted)","spark.driver.host":"30.0.0.2","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.python.gpu.enabled":"true","spark.submit.deployMode":"client","spark.executor.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.rapids.memory.pinnedPool.size":"16G","spark.sql.execution.id":"0","spark.history.fs.logDirectory":"file:///root/eventLog","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.gpu.reserve":"1500000000","spark.app.submitTime":"1700712647352","spark.app.name":"Linear_Regression","spark.eventLog.enabled":"true","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.explain":"NOT_ON_GPU","spark.rapids.sql.concurrentGpuTasks":"4","spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.rapids.memory.gpu.reserve":"1500000000","spark.app.id":"app-20231123131052-0002","spark.sql.warehouse.dir":"file:/root/spark-warehouse","spark.eventLog.dir":"file:///root/eventLog","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.pinnedPool.size":"16G","spark.rapids.driver.user.timezone":"Asia/Seoul","spark.driver.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.driver.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.task.cpus":"1","spark.executor.instances":"2","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.jars":"*********(redacted)"}}
{"Event":"SparkListenerStageSubmitted","Stage Info":{"Stage ID":0,"Stage Attempt ID":0,"Stage Name":"showString at NativeMethodAccessorImpl.java:0","Number of Tasks":1,"RDD Info":[{"RDD ID":10,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"8\",\"name\":\"mapPartitionsInternal\"}","Callsite":"showString at NativeMethodAccessorImpl.java:0","Parent IDs":[9],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":1,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"7\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[0],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":9,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"0\",\"name\":\"GpuColumnarToRow\"}","Callsite":"mapPartitions at GpuColumnarToRowExec.scala:368","Parent IDs":[8],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":8,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"1\",\"name\":\"GpuProject\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[7],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":2,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"4\",\"name\":\"WholeStageCodegen (1)\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[1],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":6,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"2\",\"name\":\"GpuRowToColumnar\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[5],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":4,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"3\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[3],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":0,"Name":"FileScanRDD","Scope":"{\"id\":\"7\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":3,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"3\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[2],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":5,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"2\",\"name\":\"GpuRowToColumnar\"}","Callsite":"mapPartitions at GpuRowToColumnarExec.scala:913","Parent IDs":[4],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":7,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"1\",\"name\":\"GpuProject\"}","Callsite":"map at basicPhysicalOperators.scala:380","Parent IDs":[6],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0}],"Parent IDs":[],"Details":"org.apache.spark.sql.Dataset.showString(Dataset.scala:323)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:498)\npy4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\npy4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)\npy4j.Gateway.invoke(Gateway.java:282)\npy4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\npy4j.commands.CallCommand.execute(CallCommand.java:79)\npy4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)\npy4j.ClientServerConnection.run(ClientServerConnection.java:106)\njava.lang.Thread.run(Thread.java:750)","Submission Time":1700712658154,"Accumulables":[],"Resource Profile Id":0,"Shuffle Push Enabled":false,"Shuffle Push Mergers Count":0},"Properties":{"resource.executor.cores":"8","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.concurrentGpuTasks":"4","spark.shuffle.manager":"com.nvidia.spark.rapids.spark341.RapidsShuffleManager","spark.driver.port":"41803","spark.sql.files.maxPartitionBytes":"512m","spark.executor.cores":"8","spark.sql.execution.root.id":"0","spark.rapids.sql.python.gpu.enabled":"true","spark.repl.local.jars":"*********(redacted)","spark.plugins":"com.nvidia.spark.SQLPlugin","spark.files":"*********(redacted)","spark.task.resource.gpu.amount":"0.125","spark.submit.pyFiles":"*********(redacted)","spark.executor.memory":"128G","spark.executor.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.executor.id":"driver","spark.app.startTime":"1700712650869","spark.rdd.compress":"True","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.driver.user.timezone":"Asia/Seoul","spark.executor.resource.gpu.amount":"1","spark.sql.extensions":"com.nvidia.spark.rapids.SQLExecPlugin,com.nvidia.spark.udf.Plugin,com.nvidia.spark.rapids.optimizer.SQLOptimizerPlugin","spark.rapids.sql.explain":"NOT_ON_GPU","spark.master":"spark://30.0.0.2:7077","spark.serializer.objectStreamReset":"100","spark.app.initial.jar.urls":"*********(redacted)","spark.app.initial.file.urls":"*********(redacted)","spark.driver.host":"30.0.0.2","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.python.gpu.enabled":"true","spark.submit.deployMode":"client","spark.executor.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.rapids.memory.pinnedPool.size":"16G","spark.sql.execution.id":"0","spark.history.fs.logDirectory":"file:///root/eventLog","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.gpu.reserve":"1500000000","spark.app.submitTime":"1700712647352","spark.app.name":"Linear_Regression","spark.eventLog.enabled":"true","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.explain":"NOT_ON_GPU","spark.rapids.sql.concurrentGpuTasks":"4","spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.rapids.memory.gpu.reserve":"1500000000","spark.app.id":"app-20231123131052-0002","spark.sql.warehouse.dir":"file:/root/spark-warehouse","spark.eventLog.dir":"file:///root/eventLog","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.pinnedPool.size":"16G","spark.rapids.driver.user.timezone":"Asia/Seoul","spark.driver.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.driver.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.task.cpus":"1","spark.executor.instances":"2","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.jars":"*********(redacted)"}}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712664995,"Executor ID":"3","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712667574,"Executor ID":"4","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712670725,"Executor ID":"5","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712672802,"Executor ID":"6","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerTaskStart","Stage ID":0,"Stage Attempt ID":0,"Task Info":{"Task ID":0,"Index":0,"Attempt":0,"Partition ID":0,"Launch Time":1700712673011,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":0,"Failed":false,"Killed":false,"Accumulables":[]}}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712675233,"Executor ID":"7","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712677655,"Executor ID":"8","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712680030,"Executor ID":"9","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712682452,"Executor ID":"10","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerTaskEnd","Stage ID":0,"Stage Attempt ID":0,"Task Type":"ResultTask","Task End Reason":{"Reason":"Success"},"Task Info":{"Task ID":0,"Index":0,"Attempt":0,"Partition ID":0,"Launch Time":1700712673011,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1700712684126,"Failed":false,"Killed":false,"Accumulables":[{"ID":16,"Name":"gpuSemaphoreWait","Update":"00:00:00.005","Value":"00:00:00.005","Internal":false,"Count Failed Values":true},{"ID":23,"Name":"data sent to Python workers","Update":"61170520","Value":"61170520","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":24,"Name":"data returned from Python workers","Update":"1120","Value":"1120","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":25,"Name":"number of output rows","Update":"21","Value":"21","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":46,"Name":"op time","Update":"2284942","Value":"2284942","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":47,"Name":"stream time","Update":"8601172414","Value":"8601172414","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":48,"Name":"op time","Update":"69381420","Value":"69381420","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":49,"Name":"op time","Update":"30671498","Value":"30671498","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":50,"Name":"stream time","Update":"158147464","Value":"158147464","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":51,"Name":"duration","Update":"13709","Value":"13709","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":52,"Name":"number of output rows","Update":"21","Value":"21","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":56,"Name":"internal.metrics.executorDeserializeTime","Update":1166,"Value":1166,"Internal":true,"Count Failed Values":true},{"ID":57,"Name":"internal.metrics.executorDeserializeCpuTime","Update":909512162,"Value":909512162,"Internal":true,"Count Failed Values":true},{"ID":58,"Name":"internal.metrics.executorRunTime","Update":9828,"Value":9828,"Internal":true,"Count Failed Values":true},{"ID":59,"Name":"internal.metrics.executorCpuTime","Update":982347376,"Value":982347376,"Internal":true,"Count Failed Values":true},{"ID":60,"Name":"internal.metrics.resultSize","Update":6186,"Value":6186,"Internal":true,"Count Failed Values":true},{"ID":61,"Name":"internal.metrics.jvmGCTime","Update":355,"Value":355,"Internal":true,"Count Failed Values":true},{"ID":62,"Name":"internal.metrics.resultSerializationTime","Update":4,"Value":4,"Internal":true,"Count Failed Values":true},{"ID":63,"Name":"internal.metrics.memoryBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":64,"Name":"internal.metrics.diskBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":87,"Name":"internal.metrics.input.bytesRead","Update":2141944,"Value":2141944,"Internal":true,"Count Failed Values":true},{"ID":88,"Name":"internal.metrics.input.recordsRead","Update":21,"Value":21,"Internal":true,"Count Failed Values":true}]},"Task Executor Metrics":{"JVMHeapMemory":543433456,"JVMOffHeapMemory":113338360,"OnHeapExecutionMemory":67108864,"OffHeapExecutionMemory":0,"OnHeapStorageMemory":810618,"OffHeapStorageMemory":0,"OnHeapUnifiedMemory":67919482,"OffHeapUnifiedMemory":0,"DirectPoolMemory":18434,"MappedPoolMemory":0,"ProcessTreeJVMVMemory":0,"ProcessTreeJVMRSSMemory":0,"ProcessTreePythonVMemory":0,"ProcessTreePythonRSSMemory":0,"ProcessTreeOtherVMemory":0,"ProcessTreeOtherRSSMemory":0,"MinorGCCount":5,"MinorGCTime":267,"MajorGCCount":3,"MajorGCTime":243,"TotalGCTime":510},"Task Metrics":{"Executor Deserialize Time":1166,"Executor Deserialize CPU Time":909512162,"Executor Run Time":9828,"Executor CPU Time":982347376,"Peak Execution Memory":0,"Result Size":6186,"JVM GC Time":355,"Result Serialization Time":4,"Memory Bytes Spilled":0,"Disk Bytes Spilled":0,"Shuffle Read Metrics":{"Remote Blocks Fetched":0,"Local Blocks Fetched":0,"Fetch Wait Time":0,"Remote Bytes Read":0,"Remote Bytes Read To Disk":0,"Local Bytes Read":0,"Total Records Read":0,"Remote Requests Duration":0,"Push Based Shuffle":{"Corrupt Merged Block Chunks":0,"Merged Fetch Fallback Count":0,"Merged Remote Blocks Fetched":0,"Merged Local Blocks Fetched":0,"Merged Remote Chunks Fetched":0,"Merged Local Chunks Fetched":0,"Merged Remote Bytes Read":0,"Merged Local Bytes Read":0,"Merged Remote Requests Duration":0}},"Shuffle Write Metrics":{"Shuffle Bytes Written":0,"Shuffle Write Time":0,"Shuffle Records Written":0},"Input Metrics":{"Bytes Read":2141944,"Records Read":21},"Output Metrics":{"Bytes Written":0,"Records Written":0},"Updated Blocks":[]}}
{"Event":"SparkListenerStageCompleted","Stage Info":{"Stage ID":0,"Stage Attempt ID":0,"Stage Name":"showString at NativeMethodAccessorImpl.java:0","Number of Tasks":1,"RDD Info":[{"RDD ID":10,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"8\",\"name\":\"mapPartitionsInternal\"}","Callsite":"showString at NativeMethodAccessorImpl.java:0","Parent IDs":[9],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":1,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"7\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[0],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":9,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"0\",\"name\":\"GpuColumnarToRow\"}","Callsite":"mapPartitions at GpuColumnarToRowExec.scala:368","Parent IDs":[8],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":8,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"1\",\"name\":\"GpuProject\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[7],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":2,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"4\",\"name\":\"WholeStageCodegen (1)\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[1],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":6,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"2\",\"name\":\"GpuRowToColumnar\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[5],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":4,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"3\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[3],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":0,"Name":"FileScanRDD","Scope":"{\"id\":\"7\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":3,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"3\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[2],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":5,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"2\",\"name\":\"GpuRowToColumnar\"}","Callsite":"mapPartitions at GpuRowToColumnarExec.scala:913","Parent IDs":[4],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":7,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"1\",\"name\":\"GpuProject\"}","Callsite":"map at basicPhysicalOperators.scala:380","Parent IDs":[6],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0}],"Parent IDs":[],"Details":"org.apache.spark.sql.Dataset.showString(Dataset.scala:323)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:498)\npy4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\npy4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)\npy4j.Gateway.invoke(Gateway.java:282)\npy4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\npy4j.commands.CallCommand.execute(CallCommand.java:79)\npy4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)\npy4j.ClientServerConnection.run(ClientServerConnection.java:106)\njava.lang.Thread.run(Thread.java:750)","Submission Time":1700712658154,"Completion Time":1700712684140,"Accumulables":[{"ID":16,"Name":"gpuSemaphoreWait","Value":"00:00:00.005","Internal":false,"Count Failed Values":true},{"ID":23,"Name":"data sent to Python workers","Value":"61170520","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":24,"Name":"data returned from Python workers","Value":"1120","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":25,"Name":"number of output rows","Value":"21","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":46,"Name":"op time","Value":"2284942","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":47,"Name":"stream time","Value":"8601172414","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":48,"Name":"op time","Value":"69381420","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":49,"Name":"op time","Value":"30671498","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":50,"Name":"stream time","Value":"158147464","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":51,"Name":"duration","Value":"13709","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":52,"Name":"number of output rows","Value":"21","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":56,"Name":"internal.metrics.executorDeserializeTime","Value":1166,"Internal":true,"Count Failed Values":true},{"ID":57,"Name":"internal.metrics.executorDeserializeCpuTime","Value":909512162,"Internal":true,"Count Failed Values":true},{"ID":58,"Name":"internal.metrics.executorRunTime","Value":9828,"Internal":true,"Count Failed Values":true},{"ID":59,"Name":"internal.metrics.executorCpuTime","Value":982347376,"Internal":true,"Count Failed Values":true},{"ID":60,"Name":"internal.metrics.resultSize","Value":6186,"Internal":true,"Count Failed Values":true},{"ID":61,"Name":"internal.metrics.jvmGCTime","Value":355,"Internal":true,"Count Failed Values":true},{"ID":62,"Name":"internal.metrics.resultSerializationTime","Value":4,"Internal":true,"Count Failed Values":true},{"ID":63,"Name":"internal.metrics.memoryBytesSpilled","Value":0,"Internal":true,"Count Failed Values":true},{"ID":64,"Name":"internal.metrics.diskBytesSpilled","Value":0,"Internal":true,"Count Failed Values":true},{"ID":87,"Name":"internal.metrics.input.bytesRead","Value":2141944,"Internal":true,"Count Failed Values":true},{"ID":88,"Name":"internal.metrics.input.recordsRead","Value":21,"Internal":true,"Count Failed Values":true}],"Resource Profile Id":0,"Shuffle Push Enabled":false,"Shuffle Push Mergers Count":0}}
{"Event":"SparkListenerJobEnd","Job ID":0,"Completion Time":1700712684146,"Job Result":{"Result":"JobSucceeded"}}
{"Event":"org.apache.spark.sql.execution.ui.SparkListenerSQLExecutionEnd","executionId":0,"time":1700712684234,"errorMessage":""}
{"Event":"org.apache.spark.sql.execution.ui.SparkListenerSQLExecutionStart","executionId":1,"rootExecutionId":1,"description":"save at NativeMethodAccessorImpl.java:0","details":"org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:498)\npy4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\npy4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)\npy4j.Gateway.invoke(Gateway.java:282)\npy4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\npy4j.commands.CallCommand.execute(CallCommand.java:79)\npy4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)\npy4j.ClientServerConnection.run(ClientServerConnection.java:106)\njava.lang.Thread.run(Thread.java:750)","physicalPlanDescription":"== Physical Plan ==\nExecute InsertIntoHadoopFsRelationCommand (11)\n+- WriteFiles (10)\n   +- GpuColumnarToRow (9)\n      +- GpuProject (8)\n         +- GpuRowToColumnar (7)\n            +- BatchEvalPython (6)\n               +- GpuColumnarToRow (5)\n                  +- GpuProject (4)\n                     +- GpuRowToColumnar (3)\n                        +- ArrowEvalPython (2)\n                           +- Scan image  (1)\n\n\n(1) Scan image \nOutput [1]: [image#0]\nBatched: false\nLocation: InMemoryFileIndex [file:/root/imagenette2/train/n02102040]\nReadSchema: struct<image:struct<origin:string,height:int,width:int,nChannels:int,mode:int,data:binary>>\n\n(2) ArrowEvalPython\nInput [1]: [image#0]\nArguments: [predict(resize_image(image#0)#2)#6], [pythonUDF0#22], 204\n\n(3) GpuRowToColumnar\nInput [2]: [image#0, pythonUDF0#22]\nArguments: targetsize(1073741824)\n\n(4) GpuProject\nInput [2]: [image#0, pythonUDF0#22]\nArguments: [pythonUDF0#22]\n\n(5) GpuColumnarToRow\nInput [1]: [pythonUDF0#22]\nArguments: false\n\n(6) BatchEvalPython\nInput [1]: [pythonUDF0#22]\nArguments: [<lambda>(pythonUDF0#22)#16], [pythonUDF0#23]\n\n(7) GpuRowToColumnar\nInput [2]: [pythonUDF0#22, pythonUDF0#23]\nArguments: targetsize(1073741824)\n\n(8) GpuProject\nInput [2]: [pythonUDF0#22, pythonUDF0#23]\nArguments: [pythonUDF0#23 AS pred_index#17]\n\n(9) GpuColumnarToRow\nInput [1]: [pred_index#17]\nArguments: false\n\n(10) WriteFiles\nInput [1]: [pred_index#17]\n\n(11) Execute InsertIntoHadoopFsRelationCommand\nInput: []\nArguments: hdfs://30.0.0.2:9000/res, false, CSV, [header=true, path=hdfs://30.0.0.2:9000/res], Overwrite, [pred_index]\n\n","sparkPlanInfo":{"nodeName":"Execute InsertIntoHadoopFsRelationCommand","simpleString":"Execute InsertIntoHadoopFsRelationCommand hdfs://30.0.0.2:9000/res, false, CSV, [header=true, path=hdfs://30.0.0.2:9000/res], Overwrite, [pred_index]","children":[{"nodeName":"WriteFiles","simpleString":"WriteFiles","children":[{"nodeName":"GpuColumnarToRow","simpleString":"GpuColumnarToRow false","children":[{"nodeName":"GpuProject","simpleString":"GpuProject [pythonUDF0#23 AS pred_index#17]","children":[{"nodeName":"GpuRowToColumnar","simpleString":"GpuRowToColumnar targetsize(1073741824)","children":[{"nodeName":"BatchEvalPython","simpleString":"BatchEvalPython [<lambda>(pythonUDF0#22)#16], [pythonUDF0#23]","children":[{"nodeName":"GpuColumnarToRow","simpleString":"GpuColumnarToRow false","children":[{"nodeName":"GpuProject","simpleString":"GpuProject [pythonUDF0#22]","children":[{"nodeName":"GpuRowToColumnar","simpleString":"GpuRowToColumnar targetsize(1073741824)","children":[{"nodeName":"ArrowEvalPython","simpleString":"ArrowEvalPython [predict(resize_image(image#0)#2)#6], [pythonUDF0#22], 204","children":[{"nodeName":"Scan image ","simpleString":"FileScan image [image#0] Batched: false, DataFilters: [], Format: org.apache.spark.ml.source.image.ImageFileFormat@58424b47, Location: InMemoryFileIndex(1 paths)[file:/root/imagenette2/train/n02102040], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<image:struct<origin:string,height:int,width:int,nChannels:int,mode:int,data:binary>>","children":[],"metadata":{"Location":"InMemoryFileIndex(1 paths)[file:/root/imagenette2/train/n02102040]","ReadSchema":"struct<image:struct<origin:string,height:int,width:int,nChannels:int,mode:int,data:binary>>","Format":"org.apache.spark.ml.source.image.ImageFileFormat@58424b47","Batched":"false","PartitionFilters":"[]","PushedFilters":"[]","DataFilters":"[]"},"metrics":[{"name":"number of output rows","accumulatorId":156,"metricType":"sum"},{"name":"number of files read","accumulatorId":157,"metricType":"sum"},{"name":"metadata time","accumulatorId":158,"metricType":"timing"},{"name":"size of files read","accumulatorId":159,"metricType":"size"}]}],"metadata":{},"metrics":[{"name":"data sent to Python workers","accumulatorId":103,"metricType":"size"},{"name":"data returned from Python workers","accumulatorId":104,"metricType":"size"},{"name":"number of output rows","accumulatorId":105,"metricType":"sum"}]}],"metadata":{},"metrics":[{"name":"stream time","accumulatorId":155,"metricType":"nsTiming"},{"name":"op time","accumulatorId":154,"metricType":"nsTiming"}]}],"metadata":{},"metrics":[{"name":"op time","accumulatorId":153,"metricType":"nsTiming"}]}],"metadata":{},"metrics":[{"name":"op time","accumulatorId":151,"metricType":"nsTiming"},{"name":"stream time","accumulatorId":152,"metricType":"nsTiming"}]}],"metadata":{},"metrics":[{"name":"data sent to Python workers","accumulatorId":130,"metricType":"size"},{"name":"data returned from Python workers","accumulatorId":131,"metricType":"size"},{"name":"number of output rows","accumulatorId":132,"metricType":"sum"}]}],"metadata":{},"metrics":[{"name":"stream time","accumulatorId":150,"metricType":"nsTiming"},{"name":"op time","accumulatorId":149,"metricType":"nsTiming"}]}],"metadata":{},"metrics":[{"name":"op time","accumulatorId":148,"metricType":"nsTiming"}]}],"metadata":{},"metrics":[{"name":"op time","accumulatorId":146,"metricType":"nsTiming"},{"name":"stream time","accumulatorId":147,"metricType":"nsTiming"}]}],"metadata":{},"metrics":[]}],"metadata":{},"metrics":[{"name":"task commit time","accumulatorId":144,"metricType":"timing"},{"name":"number of written files","accumulatorId":140,"metricType":"sum"},{"name":"job commit time","accumulatorId":145,"metricType":"timing"},{"name":"number of output rows","accumulatorId":142,"metricType":"sum"},{"name":"number of dynamic part","accumulatorId":143,"metricType":"sum"},{"name":"written output","accumulatorId":141,"metricType":"size"}]},"time":1700712684832,"modifiedConfigs":{}}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712684893,"Executor ID":"11","Removed Reason":"Command exited with code 1"}
{"Event":"org.apache.spark.sql.execution.ui.SparkListenerDriverAccumUpdates","executionId":1,"accumUpdates":[[157,955],[158,1],[159,111552831]]}
{"Event":"SparkListenerJobStart","Job ID":1,"Submission Time":1700712685899,"Stage Infos":[{"Stage ID":1,"Stage Attempt ID":0,"Stage Name":"save at NativeMethodAccessorImpl.java:0","Number of Tasks":8,"RDD Info":[{"RDD ID":27,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"20\",\"name\":\"WriteFiles\"}","Callsite":"save at NativeMethodAccessorImpl.java:0","Parent IDs":[26],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":11,"Name":"FileScanRDD","Scope":"{\"id\":\"29\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":24,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"22\",\"name\":\"GpuProject\"}","Callsite":"map at basicPhysicalOperators.scala:380","Parent IDs":[23],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":13,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"28\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[12],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":14,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"28\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[13],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":18,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"26\",\"name\":\"GpuProject\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[17],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":15,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"27\",\"name\":\"GpuRowToColumnar\"}","Callsite":"mapPartitions at GpuRowToColumnarExec.scala:913","Parent IDs":[14],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":21,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"24\",\"name\":\"BatchEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[20],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":12,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"29\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[11],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":19,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"25\",\"name\":\"GpuColumnarToRow\"}","Callsite":"mapPartitions at GpuColumnarToRowExec.scala:368","Parent IDs":[18],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":22,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"23\",\"name\":\"GpuRowToColumnar\"}","Callsite":"mapPartitions at GpuRowToColumnarExec.scala:913","Parent IDs":[21],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":17,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"26\",\"name\":\"GpuProject\"}","Callsite":"map at basicPhysicalOperators.scala:380","Parent IDs":[16],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":23,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"23\",\"name\":\"GpuRowToColumnar\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[22],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":26,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"21\",\"name\":\"GpuColumnarToRow\"}","Callsite":"mapPartitions at GpuColumnarToRowExec.scala:368","Parent IDs":[25],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":20,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"24\",\"name\":\"BatchEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[19],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":16,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"27\",\"name\":\"GpuRowToColumnar\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[15],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":25,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"22\",\"name\":\"GpuProject\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[24],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0}],"Parent IDs":[],"Details":"org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:498)\npy4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\npy4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)\npy4j.Gateway.invoke(Gateway.java:282)\npy4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\npy4j.commands.CallCommand.execute(CallCommand.java:79)\npy4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)\npy4j.ClientServerConnection.run(ClientServerConnection.java:106)\njava.lang.Thread.run(Thread.java:750)","Accumulables":[],"Resource Profile Id":0,"Shuffle Push Enabled":false,"Shuffle Push Mergers Count":0}],"Stage IDs":[1],"Properties":{"spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.concurrentGpuTasks":"4","spark.shuffle.manager":"com.nvidia.spark.rapids.spark341.RapidsShuffleManager","spark.driver.port":"41803","spark.sql.files.maxPartitionBytes":"512m","spark.executor.cores":"8","spark.sql.execution.root.id":"1","spark.rapids.sql.python.gpu.enabled":"true","spark.repl.local.jars":"*********(redacted)","spark.plugins":"com.nvidia.spark.SQLPlugin","spark.files":"*********(redacted)","spark.task.resource.gpu.amount":"0.125","spark.submit.pyFiles":"*********(redacted)","spark.executor.memory":"128G","spark.executor.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.executor.id":"driver","spark.app.startTime":"1700712650869","spark.rdd.compress":"True","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.driver.user.timezone":"Asia/Seoul","spark.executor.resource.gpu.amount":"1","spark.sql.extensions":"com.nvidia.spark.rapids.SQLExecPlugin,com.nvidia.spark.udf.Plugin,com.nvidia.spark.rapids.optimizer.SQLOptimizerPlugin","spark.rapids.sql.explain":"NOT_ON_GPU","spark.master":"spark://30.0.0.2:7077","spark.serializer.objectStreamReset":"100","spark.app.initial.jar.urls":"*********(redacted)","spark.app.initial.file.urls":"*********(redacted)","spark.driver.host":"30.0.0.2","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.python.gpu.enabled":"true","spark.submit.deployMode":"client","spark.executor.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.rapids.memory.pinnedPool.size":"16G","spark.sql.execution.id":"1","spark.history.fs.logDirectory":"file:///root/eventLog","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.gpu.reserve":"1500000000","spark.app.submitTime":"1700712647352","spark.app.name":"Linear_Regression","spark.eventLog.enabled":"true","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.explain":"NOT_ON_GPU","spark.rapids.sql.concurrentGpuTasks":"4","spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.rapids.memory.gpu.reserve":"1500000000","spark.app.id":"app-20231123131052-0002","spark.sql.warehouse.dir":"file:/root/spark-warehouse","spark.eventLog.dir":"file:///root/eventLog","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.pinnedPool.size":"16G","spark.rapids.driver.user.timezone":"Asia/Seoul","spark.driver.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.driver.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.task.cpus":"1","spark.executor.instances":"2","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.jars":"*********(redacted)"}}
{"Event":"SparkListenerStageSubmitted","Stage Info":{"Stage ID":1,"Stage Attempt ID":0,"Stage Name":"save at NativeMethodAccessorImpl.java:0","Number of Tasks":8,"RDD Info":[{"RDD ID":27,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"20\",\"name\":\"WriteFiles\"}","Callsite":"save at NativeMethodAccessorImpl.java:0","Parent IDs":[26],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":11,"Name":"FileScanRDD","Scope":"{\"id\":\"29\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":24,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"22\",\"name\":\"GpuProject\"}","Callsite":"map at basicPhysicalOperators.scala:380","Parent IDs":[23],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":13,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"28\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[12],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":14,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"28\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[13],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":18,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"26\",\"name\":\"GpuProject\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[17],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":15,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"27\",\"name\":\"GpuRowToColumnar\"}","Callsite":"mapPartitions at GpuRowToColumnarExec.scala:913","Parent IDs":[14],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":21,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"24\",\"name\":\"BatchEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[20],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":12,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"29\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[11],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":19,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"25\",\"name\":\"GpuColumnarToRow\"}","Callsite":"mapPartitions at GpuColumnarToRowExec.scala:368","Parent IDs":[18],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":22,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"23\",\"name\":\"GpuRowToColumnar\"}","Callsite":"mapPartitions at GpuRowToColumnarExec.scala:913","Parent IDs":[21],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":17,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"26\",\"name\":\"GpuProject\"}","Callsite":"map at basicPhysicalOperators.scala:380","Parent IDs":[16],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":23,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"23\",\"name\":\"GpuRowToColumnar\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[22],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":26,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"21\",\"name\":\"GpuColumnarToRow\"}","Callsite":"mapPartitions at GpuColumnarToRowExec.scala:368","Parent IDs":[25],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":20,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"24\",\"name\":\"BatchEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[19],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":16,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"27\",\"name\":\"GpuRowToColumnar\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[15],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":25,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"22\",\"name\":\"GpuProject\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[24],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0}],"Parent IDs":[],"Details":"org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:498)\npy4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\npy4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)\npy4j.Gateway.invoke(Gateway.java:282)\npy4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\npy4j.commands.CallCommand.execute(CallCommand.java:79)\npy4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)\npy4j.ClientServerConnection.run(ClientServerConnection.java:106)\njava.lang.Thread.run(Thread.java:750)","Submission Time":1700712685910,"Accumulables":[],"Resource Profile Id":0,"Shuffle Push Enabled":false,"Shuffle Push Mergers Count":0},"Properties":{"resource.executor.cores":"8","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.concurrentGpuTasks":"4","spark.shuffle.manager":"com.nvidia.spark.rapids.spark341.RapidsShuffleManager","spark.driver.port":"41803","spark.sql.files.maxPartitionBytes":"512m","spark.executor.cores":"8","spark.sql.execution.root.id":"1","spark.rapids.sql.python.gpu.enabled":"true","spark.repl.local.jars":"*********(redacted)","spark.plugins":"com.nvidia.spark.SQLPlugin","spark.files":"*********(redacted)","spark.task.resource.gpu.amount":"0.125","spark.submit.pyFiles":"*********(redacted)","spark.executor.memory":"128G","spark.executor.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.executor.id":"driver","spark.app.startTime":"1700712650869","spark.rdd.compress":"True","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.driver.user.timezone":"Asia/Seoul","spark.executor.resource.gpu.amount":"1","spark.sql.extensions":"com.nvidia.spark.rapids.SQLExecPlugin,com.nvidia.spark.udf.Plugin,com.nvidia.spark.rapids.optimizer.SQLOptimizerPlugin","spark.rapids.sql.explain":"NOT_ON_GPU","spark.master":"spark://30.0.0.2:7077","spark.serializer.objectStreamReset":"100","spark.app.initial.jar.urls":"*********(redacted)","spark.app.initial.file.urls":"*********(redacted)","spark.driver.host":"30.0.0.2","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.python.gpu.enabled":"true","spark.submit.deployMode":"client","spark.executor.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.rapids.memory.pinnedPool.size":"16G","spark.sql.execution.id":"1","spark.history.fs.logDirectory":"file:///root/eventLog","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.gpu.reserve":"1500000000","spark.app.submitTime":"1700712647352","spark.app.name":"Linear_Regression","spark.eventLog.enabled":"true","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.explain":"NOT_ON_GPU","spark.rapids.sql.concurrentGpuTasks":"4","spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.rapids.memory.gpu.reserve":"1500000000","spark.app.id":"app-20231123131052-0002","spark.sql.warehouse.dir":"file:/root/spark-warehouse","spark.eventLog.dir":"file:///root/eventLog","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.memory.pinnedPool.size":"16G","spark.rapids.driver.user.timezone":"Asia/Seoul","spark.driver.extraClassPath":"/opt/sparkRapidsPlugin/rapids-4-spark_2.12-23.10.0.jar","spark.driver.extraJavaOptions":"-Djava.net.preferIPv6Addresses=false -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false","spark.task.cpus":"1","spark.executor.instances":"2","spark.plugins.internal.conf.com.nvidia.spark.SQLPlugin.spark.rapids.sql.multiThreadedRead.numThreads":"1","spark.jars":"*********(redacted)"}}
{"Event":"SparkListenerTaskStart","Stage ID":1,"Stage Attempt ID":0,"Task Info":{"Task ID":1,"Index":0,"Attempt":0,"Partition ID":0,"Launch Time":1700712686014,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":0,"Failed":false,"Killed":false,"Accumulables":[]}}
{"Event":"SparkListenerTaskStart","Stage ID":1,"Stage Attempt ID":0,"Task Info":{"Task ID":2,"Index":1,"Attempt":0,"Partition ID":1,"Launch Time":1700712686016,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":0,"Failed":false,"Killed":false,"Accumulables":[]}}
{"Event":"SparkListenerTaskStart","Stage ID":1,"Stage Attempt ID":0,"Task Info":{"Task ID":3,"Index":2,"Attempt":0,"Partition ID":2,"Launch Time":1700712686018,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":0,"Failed":false,"Killed":false,"Accumulables":[]}}
{"Event":"SparkListenerTaskStart","Stage ID":1,"Stage Attempt ID":0,"Task Info":{"Task ID":4,"Index":3,"Attempt":0,"Partition ID":3,"Launch Time":1700712686020,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":0,"Failed":false,"Killed":false,"Accumulables":[]}}
{"Event":"SparkListenerTaskStart","Stage ID":1,"Stage Attempt ID":0,"Task Info":{"Task ID":5,"Index":4,"Attempt":0,"Partition ID":4,"Launch Time":1700712686021,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":0,"Failed":false,"Killed":false,"Accumulables":[]}}
{"Event":"SparkListenerTaskStart","Stage ID":1,"Stage Attempt ID":0,"Task Info":{"Task ID":6,"Index":5,"Attempt":0,"Partition ID":5,"Launch Time":1700712686023,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":0,"Failed":false,"Killed":false,"Accumulables":[]}}
{"Event":"SparkListenerTaskStart","Stage ID":1,"Stage Attempt ID":0,"Task Info":{"Task ID":7,"Index":6,"Attempt":0,"Partition ID":6,"Launch Time":1700712686024,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":0,"Failed":false,"Killed":false,"Accumulables":[]}}
{"Event":"SparkListenerTaskStart","Stage ID":1,"Stage Attempt ID":0,"Task Info":{"Task ID":8,"Index":7,"Attempt":0,"Partition ID":7,"Launch Time":1700712686025,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":0,"Failed":false,"Killed":false,"Accumulables":[]}}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712686957,"Executor ID":"12","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712690201,"Executor ID":"13","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerTaskEnd","Stage ID":1,"Stage Attempt ID":0,"Task Type":"ResultTask","Task End Reason":{"Reason":"Success"},"Task Info":{"Task ID":6,"Index":5,"Attempt":0,"Partition ID":5,"Launch Time":1700712686023,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1700712692979,"Failed":false,"Killed":false,"Accumulables":[{"ID":103,"Name":"data sent to Python workers","Update":"70259584","Value":"70259584","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":104,"Name":"data returned from Python workers","Update":"5520","Value":"5520","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":105,"Name":"number of output rows","Update":"121","Value":"121","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":130,"Name":"data sent to Python workers","Update":"11396","Value":"11396","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":131,"Name":"data returned from Python workers","Update":"274","Value":"274","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":132,"Name":"number of output rows","Update":"121","Value":"121","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":133,"Name":"gpuSemaphoreWait","Update":"00:00:00.002","Value":"00:00:00.002","Internal":false,"Count Failed Values":true},{"ID":144,"Name":"task commit time","Update":"34","Value":"34","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":146,"Name":"op time","Update":"268266","Value":"268266","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":147,"Name":"stream time","Update":"5124956123","Value":"5124956123","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":148,"Name":"op time","Update":"1600454","Value":"1600454","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":149,"Name":"op time","Update":"651605","Value":"651605","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":150,"Name":"stream time","Update":"8445976","Value":"8445976","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":151,"Name":"op time","Update":"430721","Value":"430721","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":152,"Name":"stream time","Update":"6002212735","Value":"6002212735","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":153,"Name":"op time","Update":"2858290","Value":"2858290","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":154,"Name":"op time","Update":"6567851","Value":"6567851","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":155,"Name":"stream time","Update":"103246395","Value":"103246395","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":156,"Name":"number of output rows","Update":"121","Value":"121","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":160,"Name":"internal.metrics.executorDeserializeTime","Update":503,"Value":503,"Internal":true,"Count Failed Values":true},{"ID":161,"Name":"internal.metrics.executorDeserializeCpuTime","Update":170575225,"Value":170575225,"Internal":true,"Count Failed Values":true},{"ID":162,"Name":"internal.metrics.executorRunTime","Update":6403,"Value":6403,"Internal":true,"Count Failed Values":true},{"ID":163,"Name":"internal.metrics.executorCpuTime","Update":490705163,"Value":490705163,"Internal":true,"Count Failed Values":true},{"ID":164,"Name":"internal.metrics.resultSize","Update":4691,"Value":4691,"Internal":true,"Count Failed Values":true},{"ID":165,"Name":"internal.metrics.jvmGCTime","Update":1001,"Value":1001,"Internal":true,"Count Failed Values":true},{"ID":166,"Name":"internal.metrics.resultSerializationTime","Update":7,"Value":7,"Internal":true,"Count Failed Values":true},{"ID":167,"Name":"internal.metrics.memoryBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":168,"Name":"internal.metrics.diskBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":191,"Name":"internal.metrics.input.bytesRead","Update":95328,"Value":95328,"Internal":true,"Count Failed Values":true},{"ID":192,"Name":"internal.metrics.input.recordsRead","Update":121,"Value":121,"Internal":true,"Count Failed Values":true},{"ID":193,"Name":"internal.metrics.output.bytesWritten","Update":253,"Value":253,"Internal":true,"Count Failed Values":true},{"ID":194,"Name":"internal.metrics.output.recordsWritten","Update":121,"Value":121,"Internal":true,"Count Failed Values":true}]},"Task Executor Metrics":{"JVMHeapMemory":2291698536,"JVMOffHeapMemory":136323232,"OnHeapExecutionMemory":872415232,"OffHeapExecutionMemory":0,"OnHeapStorageMemory":1113288,"OffHeapStorageMemory":0,"OnHeapUnifiedMemory":873528520,"OffHeapUnifiedMemory":0,"DirectPoolMemory":33282,"MappedPoolMemory":0,"ProcessTreeJVMVMemory":0,"ProcessTreeJVMRSSMemory":0,"ProcessTreePythonVMemory":0,"ProcessTreePythonRSSMemory":0,"ProcessTreeOtherVMemory":0,"ProcessTreeOtherRSSMemory":0,"MinorGCCount":13,"MinorGCTime":1093,"MajorGCCount":4,"MajorGCTime":418,"TotalGCTime":1511},"Task Metrics":{"Executor Deserialize Time":503,"Executor Deserialize CPU Time":170575225,"Executor Run Time":6403,"Executor CPU Time":490705163,"Peak Execution Memory":0,"Result Size":4691,"JVM GC Time":1001,"Result Serialization Time":7,"Memory Bytes Spilled":0,"Disk Bytes Spilled":0,"Shuffle Read Metrics":{"Remote Blocks Fetched":0,"Local Blocks Fetched":0,"Fetch Wait Time":0,"Remote Bytes Read":0,"Remote Bytes Read To Disk":0,"Local Bytes Read":0,"Total Records Read":0,"Remote Requests Duration":0,"Push Based Shuffle":{"Corrupt Merged Block Chunks":0,"Merged Fetch Fallback Count":0,"Merged Remote Blocks Fetched":0,"Merged Local Blocks Fetched":0,"Merged Remote Chunks Fetched":0,"Merged Local Chunks Fetched":0,"Merged Remote Bytes Read":0,"Merged Local Bytes Read":0,"Merged Remote Requests Duration":0}},"Shuffle Write Metrics":{"Shuffle Bytes Written":0,"Shuffle Write Time":0,"Shuffle Records Written":0},"Input Metrics":{"Bytes Read":95328,"Records Read":121},"Output Metrics":{"Bytes Written":253,"Records Written":121},"Updated Blocks":[]}}
{"Event":"SparkListenerTaskEnd","Stage ID":1,"Stage Attempt ID":0,"Task Type":"ResultTask","Task End Reason":{"Reason":"Success"},"Task Info":{"Task ID":8,"Index":7,"Attempt":0,"Partition ID":7,"Launch Time":1700712686025,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1700712695864,"Failed":false,"Killed":false,"Accumulables":[{"ID":103,"Name":"data sent to Python workers","Update":"55888664","Value":"126148248","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":104,"Name":"data returned from Python workers","Update":"5392","Value":"10912","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":105,"Name":"number of output rows","Update":"118","Value":"239","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":130,"Name":"data sent to Python workers","Update":"11114","Value":"22510","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":131,"Name":"data returned from Python workers","Update":"268","Value":"542","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":132,"Name":"number of output rows","Update":"118","Value":"239","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":133,"Name":"gpuSemaphoreWait","Update":"00:00:00.001","Value":"00:00:00.004","Internal":false,"Count Failed Values":true},{"ID":144,"Name":"task commit time","Update":"62","Value":"96","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":146,"Name":"op time","Update":"108980","Value":"377246","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":147,"Name":"stream time","Update":"8101166229","Value":"13226122352","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":148,"Name":"op time","Update":"668359","Value":"2268813","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":149,"Name":"op time","Update":"268618","Value":"920223","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":150,"Name":"stream time","Update":"3808157","Value":"12254133","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":151,"Name":"op time","Update":"228059","Value":"658780","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":152,"Name":"stream time","Update":"9002718990","Value":"15004931725","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":153,"Name":"op time","Update":"1264496","Value":"4122786","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":154,"Name":"op time","Update":"5352566","Value":"11920417","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":155,"Name":"stream time","Update":"71546185","Value":"174792580","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":156,"Name":"number of output rows","Update":"118","Value":"239","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":160,"Name":"internal.metrics.executorDeserializeTime","Update":501,"Value":1004,"Internal":true,"Count Failed Values":true},{"ID":161,"Name":"internal.metrics.executorDeserializeCpuTime","Update":130205346,"Value":300780571,"Internal":true,"Count Failed Values":true},{"ID":162,"Name":"internal.metrics.executorRunTime","Update":9273,"Value":15676,"Internal":true,"Count Failed Values":true},{"ID":163,"Name":"internal.metrics.executorCpuTime","Update":123332547,"Value":614037710,"Internal":true,"Count Failed Values":true},{"ID":164,"Name":"internal.metrics.resultSize","Update":4648,"Value":9339,"Internal":true,"Count Failed Values":true},{"ID":165,"Name":"internal.metrics.jvmGCTime","Update":1001,"Value":2002,"Internal":true,"Count Failed Values":true},{"ID":167,"Name":"internal.metrics.memoryBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":168,"Name":"internal.metrics.diskBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":191,"Name":"internal.metrics.input.bytesRead","Update":37497,"Value":132825,"Internal":true,"Count Failed Values":true},{"ID":192,"Name":"internal.metrics.input.recordsRead","Update":118,"Value":239,"Internal":true,"Count Failed Values":true},{"ID":193,"Name":"internal.metrics.output.bytesWritten","Update":247,"Value":500,"Internal":true,"Count Failed Values":true},{"ID":194,"Name":"internal.metrics.output.recordsWritten","Update":118,"Value":239,"Internal":true,"Count Failed Values":true}]},"Task Executor Metrics":{"JVMHeapMemory":2291698536,"JVMOffHeapMemory":136323232,"OnHeapExecutionMemory":872415232,"OffHeapExecutionMemory":0,"OnHeapStorageMemory":1113288,"OffHeapStorageMemory":0,"OnHeapUnifiedMemory":873528520,"OffHeapUnifiedMemory":0,"DirectPoolMemory":33282,"MappedPoolMemory":0,"ProcessTreeJVMVMemory":0,"ProcessTreeJVMRSSMemory":0,"ProcessTreePythonVMemory":0,"ProcessTreePythonRSSMemory":0,"ProcessTreeOtherVMemory":0,"ProcessTreeOtherRSSMemory":0,"MinorGCCount":13,"MinorGCTime":1093,"MajorGCCount":4,"MajorGCTime":418,"TotalGCTime":1511},"Task Metrics":{"Executor Deserialize Time":501,"Executor Deserialize CPU Time":130205346,"Executor Run Time":9273,"Executor CPU Time":123332547,"Peak Execution Memory":0,"Result Size":4648,"JVM GC Time":1001,"Result Serialization Time":0,"Memory Bytes Spilled":0,"Disk Bytes Spilled":0,"Shuffle Read Metrics":{"Remote Blocks Fetched":0,"Local Blocks Fetched":0,"Fetch Wait Time":0,"Remote Bytes Read":0,"Remote Bytes Read To Disk":0,"Local Bytes Read":0,"Total Records Read":0,"Remote Requests Duration":0,"Push Based Shuffle":{"Corrupt Merged Block Chunks":0,"Merged Fetch Fallback Count":0,"Merged Remote Blocks Fetched":0,"Merged Local Blocks Fetched":0,"Merged Remote Chunks Fetched":0,"Merged Local Chunks Fetched":0,"Merged Remote Bytes Read":0,"Merged Local Bytes Read":0,"Merged Remote Requests Duration":0}},"Shuffle Write Metrics":{"Shuffle Bytes Written":0,"Shuffle Write Time":0,"Shuffle Records Written":0},"Input Metrics":{"Bytes Read":37497,"Records Read":118},"Output Metrics":{"Bytes Written":247,"Records Written":118},"Updated Blocks":[]}}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712695875,"Executor ID":"14","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerTaskEnd","Stage ID":1,"Stage Attempt ID":0,"Task Type":"ResultTask","Task End Reason":{"Reason":"Success"},"Task Info":{"Task ID":7,"Index":6,"Attempt":0,"Partition ID":6,"Launch Time":1700712686024,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1700712696416,"Failed":false,"Killed":false,"Accumulables":[{"ID":103,"Name":"data sent to Python workers","Update":"74932408","Value":"201080656","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":104,"Name":"data returned from Python workers","Update":"5568","Value":"16480","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":105,"Name":"number of output rows","Update":"122","Value":"361","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":130,"Name":"data sent to Python workers","Update":"11490","Value":"34000","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":131,"Name":"data returned from Python workers","Update":"276","Value":"818","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":132,"Name":"number of output rows","Update":"122","Value":"361","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":133,"Name":"gpuSemaphoreWait","Update":"00:00:00.001","Value":"00:00:00.005","Internal":false,"Count Failed Values":true},{"ID":144,"Name":"task commit time","Update":"26","Value":"122","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":146,"Name":"op time","Update":"152409","Value":"529655","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":147,"Name":"stream time","Update":"8688038174","Value":"21914160526","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":148,"Name":"op time","Update":"827412","Value":"3096225","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":149,"Name":"op time","Update":"361382","Value":"1281605","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":150,"Name":"stream time","Update":"2127213","Value":"14381346","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":151,"Name":"op time","Update":"251034","Value":"909814","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":152,"Name":"stream time","Update":"9573836393","Value":"24578768118","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":153,"Name":"op time","Update":"1495595","Value":"5618381","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":154,"Name":"op time","Update":"7526897","Value":"19447314","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":155,"Name":"stream time","Update":"111234040","Value":"286026620","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":156,"Name":"number of output rows","Update":"122","Value":"361","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":160,"Name":"internal.metrics.executorDeserializeTime","Update":502,"Value":1506,"Internal":true,"Count Failed Values":true},{"ID":161,"Name":"internal.metrics.executorDeserializeCpuTime","Update":129554609,"Value":430335180,"Internal":true,"Count Failed Values":true},{"ID":162,"Name":"internal.metrics.executorRunTime","Update":9826,"Value":25502,"Internal":true,"Count Failed Values":true},{"ID":163,"Name":"internal.metrics.executorCpuTime","Update":61729470,"Value":675767180,"Internal":true,"Count Failed Values":true},{"ID":164,"Name":"internal.metrics.resultSize","Update":4648,"Value":13987,"Internal":true,"Count Failed Values":true},{"ID":165,"Name":"internal.metrics.jvmGCTime","Update":1001,"Value":3003,"Internal":true,"Count Failed Values":true},{"ID":167,"Name":"internal.metrics.memoryBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":168,"Name":"internal.metrics.diskBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":191,"Name":"internal.metrics.input.bytesRead","Update":76836,"Value":209661,"Internal":true,"Count Failed Values":true},{"ID":192,"Name":"internal.metrics.input.recordsRead","Update":122,"Value":361,"Internal":true,"Count Failed Values":true},{"ID":193,"Name":"internal.metrics.output.bytesWritten","Update":255,"Value":755,"Internal":true,"Count Failed Values":true},{"ID":194,"Name":"internal.metrics.output.recordsWritten","Update":122,"Value":361,"Internal":true,"Count Failed Values":true}]},"Task Executor Metrics":{"JVMHeapMemory":2291698536,"JVMOffHeapMemory":136323232,"OnHeapExecutionMemory":872415232,"OffHeapExecutionMemory":0,"OnHeapStorageMemory":1113288,"OffHeapStorageMemory":0,"OnHeapUnifiedMemory":873528520,"OffHeapUnifiedMemory":0,"DirectPoolMemory":33282,"MappedPoolMemory":0,"ProcessTreeJVMVMemory":0,"ProcessTreeJVMRSSMemory":0,"ProcessTreePythonVMemory":0,"ProcessTreePythonRSSMemory":0,"ProcessTreeOtherVMemory":0,"ProcessTreeOtherRSSMemory":0,"MinorGCCount":13,"MinorGCTime":1093,"MajorGCCount":4,"MajorGCTime":418,"TotalGCTime":1511},"Task Metrics":{"Executor Deserialize Time":502,"Executor Deserialize CPU Time":129554609,"Executor Run Time":9826,"Executor CPU Time":61729470,"Peak Execution Memory":0,"Result Size":4648,"JVM GC Time":1001,"Result Serialization Time":0,"Memory Bytes Spilled":0,"Disk Bytes Spilled":0,"Shuffle Read Metrics":{"Remote Blocks Fetched":0,"Local Blocks Fetched":0,"Fetch Wait Time":0,"Remote Bytes Read":0,"Remote Bytes Read To Disk":0,"Local Bytes Read":0,"Total Records Read":0,"Remote Requests Duration":0,"Push Based Shuffle":{"Corrupt Merged Block Chunks":0,"Merged Fetch Fallback Count":0,"Merged Remote Blocks Fetched":0,"Merged Local Blocks Fetched":0,"Merged Remote Chunks Fetched":0,"Merged Local Chunks Fetched":0,"Merged Remote Bytes Read":0,"Merged Local Bytes Read":0,"Merged Remote Requests Duration":0}},"Shuffle Write Metrics":{"Shuffle Bytes Written":0,"Shuffle Write Time":0,"Shuffle Records Written":0},"Input Metrics":{"Bytes Read":76836,"Records Read":122},"Output Metrics":{"Bytes Written":255,"Records Written":122},"Updated Blocks":[]}}
{"Event":"SparkListenerTaskEnd","Stage ID":1,"Stage Attempt ID":0,"Task Type":"ResultTask","Task End Reason":{"Reason":"Success"},"Task Info":{"Task ID":3,"Index":2,"Attempt":0,"Partition ID":2,"Launch Time":1700712686018,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1700712696850,"Failed":false,"Killed":false,"Accumulables":[{"ID":103,"Name":"data sent to Python workers","Update":"66068728","Value":"267149384","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":104,"Name":"data returned from Python workers","Update":"5432","Value":"21912","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":105,"Name":"number of output rows","Update":"119","Value":"480","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":130,"Name":"data sent to Python workers","Update":"11208","Value":"45208","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":131,"Name":"data returned from Python workers","Update":"270","Value":"1088","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":132,"Name":"number of output rows","Update":"119","Value":"480","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":133,"Name":"gpuSemaphoreWait","Update":"00:00:00.001","Value":"00:00:00.007","Internal":false,"Count Failed Values":true},{"ID":144,"Name":"task commit time","Update":"17","Value":"139","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":146,"Name":"op time","Update":"195564","Value":"725219","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":147,"Name":"stream time","Update":"9146897085","Value":"31061057611","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":148,"Name":"op time","Update":"1106589","Value":"4202814","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":149,"Name":"op time","Update":"454896","Value":"1736501","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":150,"Name":"stream time","Update":"8373593","Value":"22754939","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":151,"Name":"op time","Update":"337204","Value":"1247018","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":152,"Name":"stream time","Update":"10007907278","Value":"34586675396","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":153,"Name":"op time","Update":"2484115","Value":"8102496","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":154,"Name":"op time","Update":"7051721","Value":"26499035","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":155,"Name":"stream time","Update":"167971737","Value":"453998357","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":156,"Name":"number of output rows","Update":"119","Value":"480","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":160,"Name":"internal.metrics.executorDeserializeTime","Update":507,"Value":2013,"Internal":true,"Count Failed Values":true},{"ID":161,"Name":"internal.metrics.executorDeserializeCpuTime","Update":134462565,"Value":564797745,"Internal":true,"Count Failed Values":true},{"ID":162,"Name":"internal.metrics.executorRunTime","Update":10290,"Value":35792,"Internal":true,"Count Failed Values":true},{"ID":163,"Name":"internal.metrics.executorCpuTime","Update":64806744,"Value":740573924,"Internal":true,"Count Failed Values":true},{"ID":164,"Name":"internal.metrics.resultSize","Update":4648,"Value":18635,"Internal":true,"Count Failed Values":true},{"ID":165,"Name":"internal.metrics.jvmGCTime","Update":1001,"Value":4004,"Internal":true,"Count Failed Values":true},{"ID":167,"Name":"internal.metrics.memoryBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":168,"Name":"internal.metrics.diskBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":191,"Name":"internal.metrics.input.bytesRead","Update":149418,"Value":359079,"Internal":true,"Count Failed Values":true},{"ID":192,"Name":"internal.metrics.input.recordsRead","Update":119,"Value":480,"Internal":true,"Count Failed Values":true},{"ID":193,"Name":"internal.metrics.output.bytesWritten","Update":249,"Value":1004,"Internal":true,"Count Failed Values":true},{"ID":194,"Name":"internal.metrics.output.recordsWritten","Update":119,"Value":480,"Internal":true,"Count Failed Values":true}]},"Task Executor Metrics":{"JVMHeapMemory":2291698536,"JVMOffHeapMemory":136323232,"OnHeapExecutionMemory":872415232,"OffHeapExecutionMemory":0,"OnHeapStorageMemory":1113288,"OffHeapStorageMemory":0,"OnHeapUnifiedMemory":873528520,"OffHeapUnifiedMemory":0,"DirectPoolMemory":33282,"MappedPoolMemory":0,"ProcessTreeJVMVMemory":0,"ProcessTreeJVMRSSMemory":0,"ProcessTreePythonVMemory":0,"ProcessTreePythonRSSMemory":0,"ProcessTreeOtherVMemory":0,"ProcessTreeOtherRSSMemory":0,"MinorGCCount":13,"MinorGCTime":1093,"MajorGCCount":4,"MajorGCTime":418,"TotalGCTime":1511},"Task Metrics":{"Executor Deserialize Time":507,"Executor Deserialize CPU Time":134462565,"Executor Run Time":10290,"Executor CPU Time":64806744,"Peak Execution Memory":0,"Result Size":4648,"JVM GC Time":1001,"Result Serialization Time":0,"Memory Bytes Spilled":0,"Disk Bytes Spilled":0,"Shuffle Read Metrics":{"Remote Blocks Fetched":0,"Local Blocks Fetched":0,"Fetch Wait Time":0,"Remote Bytes Read":0,"Remote Bytes Read To Disk":0,"Local Bytes Read":0,"Total Records Read":0,"Remote Requests Duration":0,"Push Based Shuffle":{"Corrupt Merged Block Chunks":0,"Merged Fetch Fallback Count":0,"Merged Remote Blocks Fetched":0,"Merged Local Blocks Fetched":0,"Merged Remote Chunks Fetched":0,"Merged Local Chunks Fetched":0,"Merged Remote Bytes Read":0,"Merged Local Bytes Read":0,"Merged Remote Requests Duration":0}},"Shuffle Write Metrics":{"Shuffle Bytes Written":0,"Shuffle Write Time":0,"Shuffle Records Written":0},"Input Metrics":{"Bytes Read":149418,"Records Read":119},"Output Metrics":{"Bytes Written":249,"Records Written":119},"Updated Blocks":[]}}
{"Event":"SparkListenerTaskEnd","Stage ID":1,"Stage Attempt ID":0,"Task Type":"ResultTask","Task End Reason":{"Reason":"Success"},"Task Info":{"Task ID":5,"Index":4,"Attempt":0,"Partition ID":4,"Launch Time":1700712686021,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1700712697192,"Failed":false,"Killed":false,"Accumulables":[{"ID":103,"Name":"data sent to Python workers","Update":"64109488","Value":"331258872","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":104,"Name":"data returned from Python workers","Update":"5480","Value":"27392","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":105,"Name":"number of output rows","Update":"120","Value":"600","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":130,"Name":"data sent to Python workers","Update":"11302","Value":"56510","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":131,"Name":"data returned from Python workers","Update":"272","Value":"1360","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":132,"Name":"number of output rows","Update":"120","Value":"600","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":133,"Name":"gpuSemaphoreWait","Update":"00:00:00.003","Value":"00:00:00.009","Internal":false,"Count Failed Values":true},{"ID":144,"Name":"task commit time","Update":"8","Value":"147","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":146,"Name":"op time","Update":"727822","Value":"1453041","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":147,"Name":"stream time","Update":"9533423393","Value":"40594481004","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":148,"Name":"op time","Update":"3056096","Value":"7258910","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":149,"Name":"op time","Update":"1174010","Value":"2910511","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":150,"Name":"stream time","Update":"2056695","Value":"24811634","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":151,"Name":"op time","Update":"192222","Value":"1439240","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":152,"Name":"stream time","Update":"10385834054","Value":"44972509450","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":153,"Name":"op time","Update":"1852026","Value":"9954522","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":154,"Name":"op time","Update":"5927669","Value":"32426704","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":155,"Name":"stream time","Update":"151317766","Value":"605316123","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":156,"Name":"number of output rows","Update":"120","Value":"600","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":160,"Name":"internal.metrics.executorDeserializeTime","Update":505,"Value":2518,"Internal":true,"Count Failed Values":true},{"ID":161,"Name":"internal.metrics.executorDeserializeCpuTime","Update":138247290,"Value":703045035,"Internal":true,"Count Failed Values":true},{"ID":162,"Name":"internal.metrics.executorRunTime","Update":10633,"Value":46425,"Internal":true,"Count Failed Values":true},{"ID":163,"Name":"internal.metrics.executorCpuTime","Update":121598441,"Value":862172365,"Internal":true,"Count Failed Values":true},{"ID":164,"Name":"internal.metrics.resultSize","Update":4648,"Value":23283,"Internal":true,"Count Failed Values":true},{"ID":165,"Name":"internal.metrics.jvmGCTime","Update":1001,"Value":5005,"Internal":true,"Count Failed Values":true},{"ID":167,"Name":"internal.metrics.memoryBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":168,"Name":"internal.metrics.diskBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":191,"Name":"internal.metrics.input.bytesRead","Update":114302,"Value":473381,"Internal":true,"Count Failed Values":true},{"ID":192,"Name":"internal.metrics.input.recordsRead","Update":120,"Value":600,"Internal":true,"Count Failed Values":true},{"ID":193,"Name":"internal.metrics.output.bytesWritten","Update":251,"Value":1255,"Internal":true,"Count Failed Values":true},{"ID":194,"Name":"internal.metrics.output.recordsWritten","Update":120,"Value":600,"Internal":true,"Count Failed Values":true}]},"Task Executor Metrics":{"JVMHeapMemory":2291698536,"JVMOffHeapMemory":136323232,"OnHeapExecutionMemory":872415232,"OffHeapExecutionMemory":0,"OnHeapStorageMemory":1113288,"OffHeapStorageMemory":0,"OnHeapUnifiedMemory":873528520,"OffHeapUnifiedMemory":0,"DirectPoolMemory":33282,"MappedPoolMemory":0,"ProcessTreeJVMVMemory":0,"ProcessTreeJVMRSSMemory":0,"ProcessTreePythonVMemory":0,"ProcessTreePythonRSSMemory":0,"ProcessTreeOtherVMemory":0,"ProcessTreeOtherRSSMemory":0,"MinorGCCount":13,"MinorGCTime":1093,"MajorGCCount":4,"MajorGCTime":418,"TotalGCTime":1511},"Task Metrics":{"Executor Deserialize Time":505,"Executor Deserialize CPU Time":138247290,"Executor Run Time":10633,"Executor CPU Time":121598441,"Peak Execution Memory":0,"Result Size":4648,"JVM GC Time":1001,"Result Serialization Time":0,"Memory Bytes Spilled":0,"Disk Bytes Spilled":0,"Shuffle Read Metrics":{"Remote Blocks Fetched":0,"Local Blocks Fetched":0,"Fetch Wait Time":0,"Remote Bytes Read":0,"Remote Bytes Read To Disk":0,"Local Bytes Read":0,"Total Records Read":0,"Remote Requests Duration":0,"Push Based Shuffle":{"Corrupt Merged Block Chunks":0,"Merged Fetch Fallback Count":0,"Merged Remote Blocks Fetched":0,"Merged Local Blocks Fetched":0,"Merged Remote Chunks Fetched":0,"Merged Local Chunks Fetched":0,"Merged Remote Bytes Read":0,"Merged Local Bytes Read":0,"Merged Remote Requests Duration":0}},"Shuffle Write Metrics":{"Shuffle Bytes Written":0,"Shuffle Write Time":0,"Shuffle Records Written":0},"Input Metrics":{"Bytes Read":114302,"Records Read":120},"Output Metrics":{"Bytes Written":251,"Records Written":120},"Updated Blocks":[]}}
{"Event":"SparkListenerTaskEnd","Stage ID":1,"Stage Attempt ID":0,"Task Type":"ResultTask","Task End Reason":{"Reason":"Success"},"Task Info":{"Task ID":2,"Index":1,"Attempt":0,"Partition ID":1,"Launch Time":1700712686016,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1700712697242,"Failed":false,"Killed":false,"Accumulables":[{"ID":103,"Name":"data sent to Python workers","Update":"68768192","Value":"400027064","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":104,"Name":"data returned from Python workers","Update":"5432","Value":"32824","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":105,"Name":"number of output rows","Update":"119","Value":"719","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":130,"Name":"data sent to Python workers","Update":"11208","Value":"67718","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":131,"Name":"data returned from Python workers","Update":"270","Value":"1630","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":132,"Name":"number of output rows","Update":"119","Value":"719","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":133,"Name":"gpuSemaphoreWait","Update":"00:00:00.001","Value":"00:00:00.011","Internal":false,"Count Failed Values":true},{"ID":144,"Name":"task commit time","Update":"6","Value":"153","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":146,"Name":"op time","Update":"102525","Value":"1555566","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":147,"Name":"stream time","Update":"9606226922","Value":"50200707926","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":148,"Name":"op time","Update":"860826","Value":"8119736","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":149,"Name":"op time","Update":"317469","Value":"3227980","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":150,"Name":"stream time","Update":"1236820","Value":"26048454","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":151,"Name":"op time","Update":"144080","Value":"1583320","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":152,"Name":"stream time","Update":"10485036370","Value":"55457545820","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":153,"Name":"op time","Update":"1378935","Value":"11333457","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":154,"Name":"op time","Update":"6585268","Value":"39011972","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":155,"Name":"stream time","Update":"124020923","Value":"729337046","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":156,"Name":"number of output rows","Update":"119","Value":"719","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":160,"Name":"internal.metrics.executorDeserializeTime","Update":503,"Value":3021,"Internal":true,"Count Failed Values":true},{"ID":161,"Name":"internal.metrics.executorDeserializeCpuTime","Update":121330412,"Value":824375447,"Internal":true,"Count Failed Values":true},{"ID":162,"Name":"internal.metrics.executorRunTime","Update":10696,"Value":57121,"Internal":true,"Count Failed Values":true},{"ID":163,"Name":"internal.metrics.executorCpuTime","Update":60878129,"Value":923050494,"Internal":true,"Count Failed Values":true},{"ID":164,"Name":"internal.metrics.resultSize","Update":4648,"Value":27931,"Internal":true,"Count Failed Values":true},{"ID":165,"Name":"internal.metrics.jvmGCTime","Update":1001,"Value":6006,"Internal":true,"Count Failed Values":true},{"ID":167,"Name":"internal.metrics.memoryBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":168,"Name":"internal.metrics.diskBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":191,"Name":"internal.metrics.input.bytesRead","Update":175411,"Value":648792,"Internal":true,"Count Failed Values":true},{"ID":192,"Name":"internal.metrics.input.recordsRead","Update":119,"Value":719,"Internal":true,"Count Failed Values":true},{"ID":193,"Name":"internal.metrics.output.bytesWritten","Update":249,"Value":1504,"Internal":true,"Count Failed Values":true},{"ID":194,"Name":"internal.metrics.output.recordsWritten","Update":119,"Value":719,"Internal":true,"Count Failed Values":true}]},"Task Executor Metrics":{"JVMHeapMemory":2291698536,"JVMOffHeapMemory":136323232,"OnHeapExecutionMemory":872415232,"OffHeapExecutionMemory":0,"OnHeapStorageMemory":1113288,"OffHeapStorageMemory":0,"OnHeapUnifiedMemory":873528520,"OffHeapUnifiedMemory":0,"DirectPoolMemory":33282,"MappedPoolMemory":0,"ProcessTreeJVMVMemory":0,"ProcessTreeJVMRSSMemory":0,"ProcessTreePythonVMemory":0,"ProcessTreePythonRSSMemory":0,"ProcessTreeOtherVMemory":0,"ProcessTreeOtherRSSMemory":0,"MinorGCCount":13,"MinorGCTime":1093,"MajorGCCount":4,"MajorGCTime":418,"TotalGCTime":1511},"Task Metrics":{"Executor Deserialize Time":503,"Executor Deserialize CPU Time":121330412,"Executor Run Time":10696,"Executor CPU Time":60878129,"Peak Execution Memory":0,"Result Size":4648,"JVM GC Time":1001,"Result Serialization Time":0,"Memory Bytes Spilled":0,"Disk Bytes Spilled":0,"Shuffle Read Metrics":{"Remote Blocks Fetched":0,"Local Blocks Fetched":0,"Fetch Wait Time":0,"Remote Bytes Read":0,"Remote Bytes Read To Disk":0,"Local Bytes Read":0,"Total Records Read":0,"Remote Requests Duration":0,"Push Based Shuffle":{"Corrupt Merged Block Chunks":0,"Merged Fetch Fallback Count":0,"Merged Remote Blocks Fetched":0,"Merged Local Blocks Fetched":0,"Merged Remote Chunks Fetched":0,"Merged Local Chunks Fetched":0,"Merged Remote Bytes Read":0,"Merged Local Bytes Read":0,"Merged Remote Requests Duration":0}},"Shuffle Write Metrics":{"Shuffle Bytes Written":0,"Shuffle Write Time":0,"Shuffle Records Written":0},"Input Metrics":{"Bytes Read":175411,"Records Read":119},"Output Metrics":{"Bytes Written":249,"Records Written":119},"Updated Blocks":[]}}
{"Event":"SparkListenerTaskEnd","Stage ID":1,"Stage Attempt ID":0,"Task Type":"ResultTask","Task End Reason":{"Reason":"Success"},"Task Info":{"Task ID":4,"Index":3,"Attempt":0,"Partition ID":3,"Launch Time":1700712686020,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1700712697328,"Failed":false,"Killed":false,"Accumulables":[{"ID":103,"Name":"data sent to Python workers","Update":"68121120","Value":"468148184","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":104,"Name":"data returned from Python workers","Update":"5480","Value":"38304","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":105,"Name":"number of output rows","Update":"120","Value":"839","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":130,"Name":"data sent to Python workers","Update":"11302","Value":"79020","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":131,"Name":"data returned from Python workers","Update":"272","Value":"1902","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":132,"Name":"number of output rows","Update":"120","Value":"839","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":133,"Name":"gpuSemaphoreWait","Update":"00:00:00.001","Value":"00:00:00.012","Internal":false,"Count Failed Values":true},{"ID":144,"Name":"task commit time","Update":"7","Value":"160","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":146,"Name":"op time","Update":"189706","Value":"1745272","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":147,"Name":"stream time","Update":"9694290741","Value":"59894998667","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":148,"Name":"op time","Update":"1256094","Value":"9375830","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":149,"Name":"op time","Update":"489471","Value":"3717451","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":150,"Name":"stream time","Update":"1723629","Value":"27772083","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":151,"Name":"op time","Update":"188773","Value":"1772093","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":152,"Name":"stream time","Update":"10572846600","Value":"66030392420","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":153,"Name":"op time","Update":"1054832","Value":"12388289","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":154,"Name":"op time","Update":"7459365","Value":"46471337","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":155,"Name":"stream time","Update":"88236861","Value":"817573907","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":156,"Name":"number of output rows","Update":"120","Value":"839","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":160,"Name":"internal.metrics.executorDeserializeTime","Update":507,"Value":3528,"Internal":true,"Count Failed Values":true},{"ID":161,"Name":"internal.metrics.executorDeserializeCpuTime","Update":128136434,"Value":952511881,"Internal":true,"Count Failed Values":true},{"ID":162,"Name":"internal.metrics.executorRunTime","Update":10775,"Value":67896,"Internal":true,"Count Failed Values":true},{"ID":163,"Name":"internal.metrics.executorCpuTime","Update":74480871,"Value":997531365,"Internal":true,"Count Failed Values":true},{"ID":164,"Name":"internal.metrics.resultSize","Update":4648,"Value":32579,"Internal":true,"Count Failed Values":true},{"ID":165,"Name":"internal.metrics.jvmGCTime","Update":1001,"Value":7007,"Internal":true,"Count Failed Values":true},{"ID":167,"Name":"internal.metrics.memoryBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":168,"Name":"internal.metrics.diskBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":191,"Name":"internal.metrics.input.bytesRead","Update":130411,"Value":779203,"Internal":true,"Count Failed Values":true},{"ID":192,"Name":"internal.metrics.input.recordsRead","Update":120,"Value":839,"Internal":true,"Count Failed Values":true},{"ID":193,"Name":"internal.metrics.output.bytesWritten","Update":251,"Value":1755,"Internal":true,"Count Failed Values":true},{"ID":194,"Name":"internal.metrics.output.recordsWritten","Update":120,"Value":839,"Internal":true,"Count Failed Values":true}]},"Task Executor Metrics":{"JVMHeapMemory":2291698536,"JVMOffHeapMemory":136323232,"OnHeapExecutionMemory":872415232,"OffHeapExecutionMemory":0,"OnHeapStorageMemory":1113288,"OffHeapStorageMemory":0,"OnHeapUnifiedMemory":873528520,"OffHeapUnifiedMemory":0,"DirectPoolMemory":33282,"MappedPoolMemory":0,"ProcessTreeJVMVMemory":0,"ProcessTreeJVMRSSMemory":0,"ProcessTreePythonVMemory":0,"ProcessTreePythonRSSMemory":0,"ProcessTreeOtherVMemory":0,"ProcessTreeOtherRSSMemory":0,"MinorGCCount":13,"MinorGCTime":1093,"MajorGCCount":4,"MajorGCTime":418,"TotalGCTime":1511},"Task Metrics":{"Executor Deserialize Time":507,"Executor Deserialize CPU Time":128136434,"Executor Run Time":10775,"Executor CPU Time":74480871,"Peak Execution Memory":0,"Result Size":4648,"JVM GC Time":1001,"Result Serialization Time":0,"Memory Bytes Spilled":0,"Disk Bytes Spilled":0,"Shuffle Read Metrics":{"Remote Blocks Fetched":0,"Local Blocks Fetched":0,"Fetch Wait Time":0,"Remote Bytes Read":0,"Remote Bytes Read To Disk":0,"Local Bytes Read":0,"Total Records Read":0,"Remote Requests Duration":0,"Push Based Shuffle":{"Corrupt Merged Block Chunks":0,"Merged Fetch Fallback Count":0,"Merged Remote Blocks Fetched":0,"Merged Local Blocks Fetched":0,"Merged Remote Chunks Fetched":0,"Merged Local Chunks Fetched":0,"Merged Remote Bytes Read":0,"Merged Local Bytes Read":0,"Merged Remote Requests Duration":0}},"Shuffle Write Metrics":{"Shuffle Bytes Written":0,"Shuffle Write Time":0,"Shuffle Records Written":0},"Input Metrics":{"Bytes Read":130411,"Records Read":120},"Output Metrics":{"Bytes Written":251,"Records Written":120},"Updated Blocks":[]}}
{"Event":"SparkListenerExecutorRemoved","Timestamp":1700712698948,"Executor ID":"15","Removed Reason":"Command exited with code 1"}
{"Event":"SparkListenerTaskEnd","Stage ID":1,"Stage Attempt ID":0,"Task Type":"ResultTask","Task End Reason":{"Reason":"Success"},"Task Info":{"Task ID":1,"Index":0,"Attempt":0,"Partition ID":0,"Launch Time":1700712686014,"Executor ID":"1","Host":"30.0.0.2","Locality":"PROCESS_LOCAL","Speculative":false,"Getting Result Time":0,"Finish Time":1700712699001,"Failed":false,"Killed":false,"Accumulables":[{"ID":103,"Name":"data sent to Python workers","Update":"125642664","Value":"593790848","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":104,"Name":"data returned from Python workers","Update":"5304","Value":"43608","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":105,"Name":"number of output rows","Update":"116","Value":"955","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":130,"Name":"data sent to Python workers","Update":"10926","Value":"89946","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":131,"Name":"data returned from Python workers","Update":"264","Value":"2166","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":132,"Name":"number of output rows","Update":"116","Value":"955","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":133,"Name":"gpuSemaphoreWait","Update":"00:00:00.001","Value":"00:00:00.013","Internal":false,"Count Failed Values":true},{"ID":144,"Name":"task commit time","Update":"10","Value":"170","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":146,"Name":"op time","Update":"173589","Value":"1918861","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":147,"Name":"stream time","Update":"10442264090","Value":"70337262757","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":148,"Name":"op time","Update":"1171449","Value":"10547279","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":149,"Name":"op time","Update":"516169","Value":"4233620","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":150,"Name":"stream time","Update":"1885358","Value":"29657441","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":151,"Name":"op time","Update":"163398","Value":"1935491","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":152,"Name":"stream time","Update":"12245798765","Value":"78276191185","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":153,"Name":"op time","Update":"1158232","Value":"13546521","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":154,"Name":"op time","Update":"11929706","Value":"58401043","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":155,"Name":"stream time","Update":"182972447","Value":"1000546354","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":156,"Name":"number of output rows","Update":"116","Value":"955","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":160,"Name":"internal.metrics.executorDeserializeTime","Update":502,"Value":4030,"Internal":true,"Count Failed Values":true},{"ID":161,"Name":"internal.metrics.executorDeserializeCpuTime","Update":139299373,"Value":1091811254,"Internal":true,"Count Failed Values":true},{"ID":162,"Name":"internal.metrics.executorRunTime","Update":12460,"Value":80356,"Internal":true,"Count Failed Values":true},{"ID":163,"Name":"internal.metrics.executorCpuTime","Update":353839576,"Value":1351370941,"Internal":true,"Count Failed Values":true},{"ID":164,"Name":"internal.metrics.resultSize","Update":4648,"Value":37227,"Internal":true,"Count Failed Values":true},{"ID":165,"Name":"internal.metrics.jvmGCTime","Update":1001,"Value":8008,"Internal":true,"Count Failed Values":true},{"ID":167,"Name":"internal.metrics.memoryBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":168,"Name":"internal.metrics.diskBytesSpilled","Update":0,"Value":0,"Internal":true,"Count Failed Values":true},{"ID":191,"Name":"internal.metrics.input.bytesRead","Update":2141944,"Value":2921147,"Internal":true,"Count Failed Values":true},{"ID":192,"Name":"internal.metrics.input.recordsRead","Update":116,"Value":955,"Internal":true,"Count Failed Values":true},{"ID":193,"Name":"internal.metrics.output.bytesWritten","Update":243,"Value":1998,"Internal":true,"Count Failed Values":true},{"ID":194,"Name":"internal.metrics.output.recordsWritten","Update":116,"Value":955,"Internal":true,"Count Failed Values":true}]},"Task Executor Metrics":{"JVMHeapMemory":2291698536,"JVMOffHeapMemory":136323232,"OnHeapExecutionMemory":872415232,"OffHeapExecutionMemory":0,"OnHeapStorageMemory":1113288,"OffHeapStorageMemory":0,"OnHeapUnifiedMemory":873528520,"OffHeapUnifiedMemory":0,"DirectPoolMemory":33282,"MappedPoolMemory":0,"ProcessTreeJVMVMemory":0,"ProcessTreeJVMRSSMemory":0,"ProcessTreePythonVMemory":0,"ProcessTreePythonRSSMemory":0,"ProcessTreeOtherVMemory":0,"ProcessTreeOtherRSSMemory":0,"MinorGCCount":13,"MinorGCTime":1093,"MajorGCCount":4,"MajorGCTime":418,"TotalGCTime":1511},"Task Metrics":{"Executor Deserialize Time":502,"Executor Deserialize CPU Time":139299373,"Executor Run Time":12460,"Executor CPU Time":353839576,"Peak Execution Memory":0,"Result Size":4648,"JVM GC Time":1001,"Result Serialization Time":0,"Memory Bytes Spilled":0,"Disk Bytes Spilled":0,"Shuffle Read Metrics":{"Remote Blocks Fetched":0,"Local Blocks Fetched":0,"Fetch Wait Time":0,"Remote Bytes Read":0,"Remote Bytes Read To Disk":0,"Local Bytes Read":0,"Total Records Read":0,"Remote Requests Duration":0,"Push Based Shuffle":{"Corrupt Merged Block Chunks":0,"Merged Fetch Fallback Count":0,"Merged Remote Blocks Fetched":0,"Merged Local Blocks Fetched":0,"Merged Remote Chunks Fetched":0,"Merged Local Chunks Fetched":0,"Merged Remote Bytes Read":0,"Merged Local Bytes Read":0,"Merged Remote Requests Duration":0}},"Shuffle Write Metrics":{"Shuffle Bytes Written":0,"Shuffle Write Time":0,"Shuffle Records Written":0},"Input Metrics":{"Bytes Read":2141944,"Records Read":116},"Output Metrics":{"Bytes Written":243,"Records Written":116},"Updated Blocks":[]}}
{"Event":"SparkListenerStageCompleted","Stage Info":{"Stage ID":1,"Stage Attempt ID":0,"Stage Name":"save at NativeMethodAccessorImpl.java:0","Number of Tasks":8,"RDD Info":[{"RDD ID":27,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"20\",\"name\":\"WriteFiles\"}","Callsite":"save at NativeMethodAccessorImpl.java:0","Parent IDs":[26],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":11,"Name":"FileScanRDD","Scope":"{\"id\":\"29\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":24,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"22\",\"name\":\"GpuProject\"}","Callsite":"map at basicPhysicalOperators.scala:380","Parent IDs":[23],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":13,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"28\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[12],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":14,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"28\",\"name\":\"ArrowEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[13],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":18,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"26\",\"name\":\"GpuProject\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[17],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":15,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"27\",\"name\":\"GpuRowToColumnar\"}","Callsite":"mapPartitions at GpuRowToColumnarExec.scala:913","Parent IDs":[14],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":21,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"24\",\"name\":\"BatchEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[20],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":12,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"29\",\"name\":\"Scan image \"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[11],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":19,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"25\",\"name\":\"GpuColumnarToRow\"}","Callsite":"mapPartitions at GpuColumnarToRowExec.scala:368","Parent IDs":[18],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":22,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"23\",\"name\":\"GpuRowToColumnar\"}","Callsite":"mapPartitions at GpuRowToColumnarExec.scala:913","Parent IDs":[21],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":17,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"26\",\"name\":\"GpuProject\"}","Callsite":"map at basicPhysicalOperators.scala:380","Parent IDs":[16],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":23,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"23\",\"name\":\"GpuRowToColumnar\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[22],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":26,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"21\",\"name\":\"GpuColumnarToRow\"}","Callsite":"mapPartitions at GpuColumnarToRowExec.scala:368","Parent IDs":[25],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":20,"Name":"MapPartitionsRDD","Scope":"{\"id\":\"24\",\"name\":\"BatchEvalPython\"}","Callsite":"execute at GpuRowToColumnarExec.scala:895","Parent IDs":[19],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":16,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"27\",\"name\":\"GpuRowToColumnar\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[15],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0},{"RDD ID":25,"Name":"LocationPreservingMapPartitionsRDD","Scope":"{\"id\":\"22\",\"name\":\"GpuProject\"}","Callsite":"MapPartitionsRDD at LocationPreservingMapPartitionsRDD.scala:44","Parent IDs":[24],"Storage Level":{"Use Disk":false,"Use Memory":false,"Use Off Heap":false,"Deserialized":false,"Replication":1},"Barrier":false,"DeterministicLevel":"DETERMINATE","Number of Partitions":8,"Number of Cached Partitions":0,"Memory Size":0,"Disk Size":0}],"Parent IDs":[],"Details":"org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:498)\npy4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)\npy4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)\npy4j.Gateway.invoke(Gateway.java:282)\npy4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\npy4j.commands.CallCommand.execute(CallCommand.java:79)\npy4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)\npy4j.ClientServerConnection.run(ClientServerConnection.java:106)\njava.lang.Thread.run(Thread.java:750)","Submission Time":1700712685910,"Completion Time":1700712699004,"Accumulables":[{"ID":103,"Name":"data sent to Python workers","Value":"593790848","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":104,"Name":"data returned from Python workers","Value":"43608","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":105,"Name":"number of output rows","Value":"955","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":130,"Name":"data sent to Python workers","Value":"89946","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":131,"Name":"data returned from Python workers","Value":"2166","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":132,"Name":"number of output rows","Value":"955","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":133,"Name":"gpuSemaphoreWait","Value":"00:00:00.013","Internal":false,"Count Failed Values":true},{"ID":144,"Name":"task commit time","Value":"170","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":146,"Name":"op time","Value":"1918861","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":147,"Name":"stream time","Value":"70337262757","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":148,"Name":"op time","Value":"10547279","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":149,"Name":"op time","Value":"4233620","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":150,"Name":"stream time","Value":"29657441","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":151,"Name":"op time","Value":"1935491","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":152,"Name":"stream time","Value":"78276191185","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":153,"Name":"op time","Value":"13546521","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":154,"Name":"op time","Value":"58401043","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":155,"Name":"stream time","Value":"1000546354","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":156,"Name":"number of output rows","Value":"955","Internal":true,"Count Failed Values":true,"Metadata":"sql"},{"ID":160,"Name":"internal.metrics.executorDeserializeTime","Value":4030,"Internal":true,"Count Failed Values":true},{"ID":161,"Name":"internal.metrics.executorDeserializeCpuTime","Value":1091811254,"Internal":true,"Count Failed Values":true},{"ID":162,"Name":"internal.metrics.executorRunTime","Value":80356,"Internal":true,"Count Failed Values":true},{"ID":163,"Name":"internal.metrics.executorCpuTime","Value":1351370941,"Internal":true,"Count Failed Values":true},{"ID":164,"Name":"internal.metrics.resultSize","Value":37227,"Internal":true,"Count Failed Values":true},{"ID":165,"Name":"internal.metrics.jvmGCTime","Value":8008,"Internal":true,"Count Failed Values":true},{"ID":166,"Name":"internal.metrics.resultSerializationTime","Value":7,"Internal":true,"Count Failed Values":true},{"ID":167,"Name":"internal.metrics.memoryBytesSpilled","Value":0,"Internal":true,"Count Failed Values":true},{"ID":168,"Name":"internal.metrics.diskBytesSpilled","Value":0,"Internal":true,"Count Failed Values":true},{"ID":191,"Name":"internal.metrics.input.bytesRead","Value":2921147,"Internal":true,"Count Failed Values":true},{"ID":192,"Name":"internal.metrics.input.recordsRead","Value":955,"Internal":true,"Count Failed Values":true},{"ID":193,"Name":"internal.metrics.output.bytesWritten","Value":1998,"Internal":true,"Count Failed Values":true},{"ID":194,"Name":"internal.metrics.output.recordsWritten","Value":955,"Internal":true,"Count Failed Values":true}],"Resource Profile Id":0,"Shuffle Push Enabled":false,"Shuffle Push Mergers Count":0}}
{"Event":"SparkListenerJobEnd","Job ID":1,"Completion Time":1700712699004,"Job Result":{"Result":"JobSucceeded"}}
{"Event":"org.apache.spark.sql.execution.ui.SparkListenerDriverAccumUpdates","executionId":1,"accumUpdates":[[140,8],[145,1223],[142,955],[143,0],[141,1998]]}
{"Event":"org.apache.spark.sql.execution.ui.SparkListenerSQLExecutionEnd","executionId":1,"time":1700712700247,"errorMessage":""}
{"Event":"SparkListenerApplicationEnd","Timestamp":1700712700959}

@leehaoun
Copy link
Author

I find this log in the remote worker's /root/spark-3.4.1-bin-hadoop3/work/app-20231124084536-0000/2/stderr

Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
23/11/24 08:45:39 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 30258@ati
23/11/24 08:45:39 INFO SignalUtils: Registering signal handler for TERM
23/11/24 08:45:39 INFO SignalUtils: Registering signal handler for HUP
23/11/24 08:45:39 INFO SignalUtils: Registering signal handler for INT
23/11/24 08:45:40 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
23/11/24 08:45:40 INFO SecurityManager: Changing view acls to: root
23/11/24 08:45:40 INFO SecurityManager: Changing modify acls to: root
23/11/24 08:45:40 INFO SecurityManager: Changing view acls groups to:
23/11/24 08:45:40 INFO SecurityManager: Changing modify acls groups to:
23/11/24 08:45:40 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: root; groups with view permissions: EMPTY; users with modify permissions: root; groups with modify permissions: EMPTY
23/11/24 08:45:40 INFO TransportClientFactory: Successfully created connection to /30.0.0.2:42187 after 56 ms (0 ms spent in bootstraps)
23/11/24 08:45:40 INFO SecurityManager: Changing view acls to: root
23/11/24 08:45:40 INFO SecurityManager: Changing modify acls to: root
23/11/24 08:45:40 INFO SecurityManager: Changing view acls groups to:
23/11/24 08:45:40 INFO SecurityManager: Changing modify acls groups to:
23/11/24 08:45:40 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: root; groups with view permissions: EMPTY; users with modify permissions: root; groups with modify permissions: EMPTY
23/11/24 08:45:40 INFO TransportClientFactory: Successfully created connection to /30.0.0.2:42187 after 2 ms (0 ms spent in bootstraps)
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1894)
at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:62)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:428)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:417)
at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
Caused by: java.lang.ClassNotFoundException: com.nvidia.spark.rapids.spark341.RapidsShuffleManager
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:225)
at org.apache.spark.util.Utils$.instantiateSerializerOrShuffleManager(Utils.scala:2715)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:323)
at org.apache.spark.SparkEnv$.createExecutorEnv(SparkEnv.scala:212)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$.$anonfun$run$7(CoarseGrainedExecutorBackend.scala:477)
at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:63)
at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:62)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
... 4 more

@leehaoun
Copy link
Author

I'm sorry, it was my fault.

The issue was caused by the difference in the paths of SPARK_RAPIDS_JAR between the HOST and RemoteWorker.

After aligning the RAPIDS_JAR paths on both PCs, RAPIDS_Shuffle worked correctly.

I will close the issue.

@abellina
Copy link
Collaborator

Glad you got it working! Let us know if we can help!

@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Dec 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@abellina @sameerz @leehaoun