Skip to content

Commit

Permalink
[CELEBORN-1187][FOLLOWUP] Unify the size and file count of active shu…
Browse files Browse the repository at this point in the history
…ffle metrics for master and worker

### What changes were proposed in this pull request?

Unify the size and file count of active shuffle metrics for `MasterSource` and `WorkerSource`.

Follow up #2171.

### Why are the changes needed?

`MasterSource` uses `PartitionWritten` and `PartitionFileCount` metrics as the size and file count of active shuffle for all workers. Meanwhile, `WorkerSource` uses `ActiveShuffleSize` and `ActiveShuffleFileCount` metrics as the size and file count of active shuffle for a worker including master replica and slave replica. It's recommended to unify the size and file count of active shuffle metrics between `MasterSource` and `WorkerSource`.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Internal tests.

Closes #2186 from SteNicholas/CELEBORN-1187.

Authored-by: SteNicholas <[email protected]>
Signed-off-by: mingji <[email protected]>
  • Loading branch information
SteNicholas authored and FMX committed Dec 22, 2023
1 parent 277f7ce commit 276ab97
Showing 1 changed file with 2 additions and 52 deletions.
54 changes: 2 additions & 52 deletions assets/grafana/celeborn-dashboard.json
Original file line number Diff line number Diff line change
Expand Up @@ -1792,32 +1792,7 @@
},
"unit": "decbytes"
},
"overrides": [
{
"__systemRef": "hideSeriesFrom",
"matcher": {
"id": "byNames",
"options": {
"mode": "exclude",
"names": [
"metrics_ActiveShuffleSize_Value{instance=\"core-1-1:9096\", job=\"RSS\", role=\"Worker\"}"
],
"prefix": "All except:",
"readOnly": true
}
},
"properties": [
{
"id": "custom.hideFrom",
"value": {
"legend": false,
"tooltip": false,
"viz": true
}
}
]
}
]
"overrides": []
},
"gridPos": {
"h": 8,
Expand Down Expand Up @@ -1911,32 +1886,7 @@
]
}
},
"overrides": [
{
"__systemRef": "hideSeriesFrom",
"matcher": {
"id": "byNames",
"options": {
"mode": "exclude",
"names": [
"metrics_ActiveShuffleFileCount_Value{instance=\"core-1-1:9096\", job=\"RSS\", role=\"Worker\"}"
],
"prefix": "All except:",
"readOnly": true
}
},
"properties": [
{
"id": "custom.hideFrom",
"value": {
"legend": false,
"tooltip": false,
"viz": true
}
}
]
}
]
"overrides": []
},
"gridPos": {
"h": 8,
Expand Down

0 comments on commit 276ab97

Please sign in to comment.