Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

session: add indexes for mysql.analyze_jobs #58134

Merged

Conversation

Rustin170506
Copy link
Member

@Rustin170506 Rustin170506 commented Dec 10, 2024

What problem does this PR solve?

Issue Number: close #57996

Problem Summary:

What changed and how does it work?

This PR added two indexes for the mysql.analyze_jobs to speed up queries.

I also bumped the bootstrap version from 239 to 140.

Check List

Tests

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

为 `mysql.analyze_jobs` 添加索引加速查询
Add some indexes to `mysql.analyze_jobs` to speed up queries

@ti-chi-bot ti-chi-bot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 10, 2024
Copy link

codecov bot commented Dec 10, 2024

Codecov Report

Attention: Patch coverage is 57.14286% with 3 lines in your changes missing coverage. Please review.

Project coverage is 75.2632%. Comparing base (cc83417) to head (e2c0dd9).
Report is 36 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #58134        +/-   ##
================================================
+ Coverage   73.2296%   75.2632%   +2.0335%     
================================================
  Files          1675       1724        +49     
  Lines        462254     472345     +10091     
================================================
+ Hits         338507     355502     +16995     
+ Misses       102970      94676      -8294     
- Partials      20777      22167      +1390     
Flag Coverage Δ
integration 49.3607% <0.0000%> (?)
unit 72.6462% <57.1428%> (+0.2854%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.6910% <ø> (ø)
parser ∅ <ø> (∅)
br 61.7115% <ø> (+15.7245%) ⬆️

@Rustin170506 Rustin170506 force-pushed the rustin-patch-analyze-jobs-index branch from 2b64e04 to d543cf2 Compare December 16, 2024 08:41
@ti-chi-bot ti-chi-bot bot added sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 16, 2024
@Rustin170506
Copy link
Member Author

Tested locally:
Before:

explain SELECT
		MIN(TIMESTAMPDIFF(SECOND, aj.start_time, CURRENT_TIMESTAMP)) AS min_duration
	FROM (
		SELECT
			MAX(id) AS max_id
		FROM
			mysql.analyze_jobs
		WHERE
			table_schema = '1'
			AND table_name = '1'
			AND state = 'failed'
			AND partition_name IN ('1')
		GROUP BY
			partition_name
	) AS latest_failures
	JOIN mysql.analyze_jobs aj ON aj.id = latest_failures.max_id;

+------------------------------------+--------+---------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|id                                  |estRows |task     |access object     |operator info                                                                                                                                                       |
+------------------------------------+--------+---------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|StreamAgg_18                        |1.00    |root     |                  |funcs:min(Column#31)->Column#28                                                                                                                                     |
|└─Projection_52                     |1.00    |root     |                  |timestampdiff(SECOND, mysql.analyze_jobs.start_time, 2024-12-16 17:25:42)->Column#31                                                                                |
|  └─Projection_50                   |1.00    |root     |                  |mysql.analyze_jobs.start_time                                                                                                                                       |
|    └─TopN_21                       |1.00    |root     |                  |Column#30, offset:0, count:1                                                                                                                                        |
|      └─Projection_51               |1.00    |root     |                  |mysql.analyze_jobs.start_time, timestampdiff(SECOND, mysql.analyze_jobs.start_time, 2024-12-16 17:25:42)->Column#30                                                 |
|        └─IndexJoin_29              |1.00    |root     |                  |inner join, inner:Selection_25, outer key:Column#14, inner key:mysql.analyze_jobs.id, equal cond:eq(Column#14, mysql.analyze_jobs.id)                               |
|          ├─Selection_36(Build)     |0.80    |root     |                  |not(isnull(Column#14))                                                                                                                                              |
|          │ └─HashAgg_39            |1.00    |root     |                  |group by:mysql.analyze_jobs.partition_name, funcs:max(mysql.analyze_jobs.id)->Column#14                                                                             |
|          │   └─TableReader_46      |0.00    |root     |                  |data:Selection_45                                                                                                                                                   |
|          │     └─Selection_45      |0.00    |cop[tikv]|                  |eq(mysql.analyze_jobs.partition_name, "1"), eq(mysql.analyze_jobs.state, "failed"), eq(mysql.analyze_jobs.table_name, "1"), eq(mysql.analyze_jobs.table_schema, "1")|
|          │       └─TableFullScan_44|10000.00|cop[tikv]|table:analyze_jobs|keep order:false, stats:pseudo                                                                                                                                      |
|          └─Selection_25(Probe)     |0.64    |root     |                  |not(isnull(timestampdiff("SECOND", mysql.analyze_jobs.start_time, 2024-12-16 17:25:42)))                                                                            |
|            └─TableReader_24        |0.80    |root     |                  |data:TableRangeScan_23                                                                                                                                              |
|              └─TableRangeScan_23   |0.80    |cop[tikv]|table:aj          |range: decided by [Column#14], keep order:false, stats:pseudo                                                                                                       |
+------------------------------------+--------+---------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
explain SELECT
		MIN(TIMESTAMPDIFF(SECOND, aj.start_time, CURRENT_TIMESTAMP)) AS min_duration
	FROM (
		SELECT
			MAX(id) AS max_id
		FROM
			mysql.analyze_jobs
		WHERE
			table_schema = '1'
			AND table_name = '1'
			AND state = 'failed'
			AND partition_name IN ('1')
		GROUP BY
			partition_name
	) AS latest_failures
	JOIN mysql.analyze_jobs aj ON aj.id = latest_failures.max_id;

+----------------------------+--------+---------+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|id                          |estRows |task     |access object     |operator info                                                                                                                                                                                                |
+----------------------------+--------+---------+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|StreamAgg_12                |1.00    |root     |                  |funcs:avg(Column#17)->Column#16                                                                                                                                                                              |
|└─Projection_26             |0.00    |root     |                  |cast(timestampdiff(SECOND, mysql.analyze_jobs.start_time, mysql.analyze_jobs.end_time), decimal(20,0) BINARY)->Column#17                                                                                     |
|  └─TopN_13                 |0.00    |root     |                  |mysql.analyze_jobs.id:desc, offset:0, count:5                                                                                                                                                                |
|    └─TableReader_21        |0.00    |root     |                  |data:TopN_20                                                                                                                                                                                                 |
|      └─TopN_20             |0.00    |cop[tikv]|                  |mysql.analyze_jobs.id:desc, offset:0, count:5                                                                                                                                                                |
|        └─Selection_19      |0.00    |cop[tikv]|                  |eq(mysql.analyze_jobs.partition_name, ""), eq(mysql.analyze_jobs.state, "finished"), eq(mysql.analyze_jobs.table_name, "1"), eq(mysql.analyze_jobs.table_schema, "1"), isnull(mysql.analyze_jobs.fail_reason)|
|          └─TableFullScan_18|10000.00|cop[tikv]|table:analyze_jobs|keep order:false, stats:pseudo                                                                                                                                                                               |
+----------------------------+--------+---------+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

After:

explain SELECT
		MIN(TIMESTAMPDIFF(SECOND, aj.start_time, CURRENT_TIMESTAMP)) AS min_duration
	FROM (
		SELECT
			MAX(id) AS max_id
		FROM
			mysql.analyze_jobs
		WHERE
			table_schema = '1'
			AND table_name = '1'
			AND state = 'failed'
			AND partition_name IN ('1')
		GROUP BY
			partition_name
	) AS latest_failures
	JOIN mysql.analyze_jobs aj ON aj.id = latest_failures.max_id;


StreamAgg_18,1.00,root,"",funcs:min(Column#32)->Column#28
└─Projection_57,1.00,root,"","timestampdiff(SECOND, mysql.analyze_jobs.start_time, 2024-12-16 17:36:19)->Column#32"
  └─Projection_55,1.00,root,"",mysql.analyze_jobs.start_time
    └─TopN_21,1.00,root,"","Column#31, offset:0, count:1"
      └─Projection_56,1.00,root,"","mysql.analyze_jobs.start_time, timestampdiff(SECOND, mysql.analyze_jobs.start_time, 2024-12-16 17:36:19)->Column#31"
        └─IndexJoin_29,1.00,root,"","inner join, inner:Selection_25, outer key:Column#14, inner key:mysql.analyze_jobs.id, equal cond:eq(Column#14, mysql.analyze_jobs.id)"
          ├─Selection_36(Build),0.80,root,"",not(isnull(Column#14))
          │ └─StreamAgg_41,1.00,root,"","group by:mysql.analyze_jobs.partition_name, funcs:max(mysql.analyze_jobs.id)->Column#14"
          │   └─IndexReader_51,0.00,root,"",index:IndexRangeScan_50
          │     └─IndexRangeScan_50,0.00,cop[tikv],"table:analyze_jobs, index:idx_schema_table_partition_state(table_schema, table_name, partition_name, state)","range:[""1"" ""1"" ""1"" ""failed"",""1"" ""1"" ""1"" ""failed""], keep order:true, stats:pseudo"
          └─Selection_25(Probe),0.64,root,"","not(isnull(timestampdiff(""SECOND"", mysql.analyze_jobs.start_time, 2024-12-16 17:36:19)))"
            └─TableReader_24,0.80,root,"",data:TableRangeScan_23
              └─TableRangeScan_23,0.80,cop[tikv],table:aj,"range: decided by [Column#14], keep order:false, stats:pseudo"
explain SELECT
		MIN(TIMESTAMPDIFF(SECOND, aj.start_time, CURRENT_TIMESTAMP)) AS min_duration
	FROM (
		SELECT
			MAX(id) AS max_id
		FROM
			mysql.analyze_jobs
		WHERE
			table_schema = '1'
			AND table_name = '1'
			AND state = 'failed'
			AND partition_name IN ('1')
		GROUP BY
			partition_name
	) AS latest_failures
	JOIN mysql.analyze_jobs aj ON aj.id = latest_failures.max_id;

StreamAgg_12,1.00,root,"",funcs:avg(Column#17)->Column#16
└─Projection_27,0.00,root,"","cast(timestampdiff(SECOND, mysql.analyze_jobs.start_time, mysql.analyze_jobs.end_time), decimal(20,0) BINARY)->Column#17"
  └─TopN_14,0.00,root,"","mysql.analyze_jobs.id:desc, offset:0, count:5"
    └─IndexLookUp_22,0.00,root,"",""
      ├─IndexRangeScan_18(Build),0.00,cop[tikv],"table:analyze_jobs, index:idx_schema_table_partition_state(table_schema, table_name, partition_name, state)","range:[""1"" ""1"" """" ""finished"",""1"" ""1"" """" ""finished""], keep order:false, stats:pseudo"
      └─TopN_21(Probe),0.00,cop[tikv],"","mysql.analyze_jobs.id:desc, offset:0, count:5"
        └─Selection_20,0.00,cop[tikv],"",isnull(mysql.analyze_jobs.fail_reason)
          └─TableRowIDScan_19,0.00,cop[tikv],table:analyze_jobs,"keep order:false, stats:pseudo"

Signed-off-by: Rustin170506 <[email protected]>
Signed-off-by: Rustin170506 <[email protected]>
Copy link
Member Author

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check (PR reviewed by myself and ready for feedback.)

@Rustin170506
Copy link
Member Author

/test all

@D3Hunter
Copy link
Contributor

/approve

for bootstrap

@Rustin170506 Rustin170506 requested review from qw4990 and Leavrth and removed request for D3Hunter December 16, 2024 09:52
@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Dec 16, 2024
Signed-off-by: Rustin170506 <[email protected]>
Copy link
Member Author

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check (PR reviewed by myself and ready for feedback.)

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Dec 16, 2024
Copy link

ti-chi-bot bot commented Dec 16, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-12-16 09:58:55.084030569 +0000 UTC m=+864525.172833112: ☑️ agreed by Leavrth.
  • 2024-12-16 10:47:54.632476366 +0000 UTC m=+867464.721278908: ☑️ agreed by qw4990.

@Rustin170506
Copy link
Member Author

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 16, 2024
@Rustin170506
Copy link
Member Author

Tested again:

  1. For new created cluster:
tiup playground v8.4.0 --db.binpath /Users/rustin/code/tidb/bin/tidb-server
mysql> show create table mysql.analyze_jobs;
ERROR 2013 (HY000): Lost connection to MySQL server during query
No connection. Trying to reconnect...
Connection id:    3338665990
Current database: *** NONE ***

+--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table        | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| analyze_jobs | CREATE TABLE `analyze_jobs` (
  `id` bigint unsigned NOT NULL AUTO_INCREMENT,
  `update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `table_schema` char(64) NOT NULL DEFAULT '',
  `table_name` char(64) NOT NULL DEFAULT '',
  `partition_name` char(64) NOT NULL DEFAULT '',
  `job_info` text NOT NULL,
  `processed_rows` bigint unsigned NOT NULL DEFAULT '0',
  `start_time` timestamp NULL DEFAULT NULL,
  `end_time` timestamp NULL DEFAULT NULL,
  `state` enum('pending','running','finished','failed') NOT NULL,
  `fail_reason` text DEFAULT NULL,
  `instance` varchar(512) NOT NULL COMMENT 'address of the TiDB instance executing the analyze job',
  `process_id` bigint unsigned DEFAULT NULL COMMENT 'ID of the process executing the analyze job',
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */,
  KEY `update_time` (`update_time`),
  KEY `idx_schema_table_state` (`table_schema`,`table_name`,`state`),
  KEY `idx_schema_table_partition_state` (`table_schema`,`table_name`,`partition_name`,`state`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.03 sec)
  1. For upgraded cluster:
tiup playground v8.4.0 --tag 20240329-test5
mysql> show create table mysql.analyze_jobs;
ERROR 2013 (HY000): Lost connection to MySQL server during query
No connection. Trying to reconnect...
Connection id:    4078960646
Current database: *** NONE ***

+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table        | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| analyze_jobs | CREATE TABLE `analyze_jobs` (
  `id` bigint(64) unsigned NOT NULL AUTO_INCREMENT,
  `update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `table_schema` char(64) NOT NULL DEFAULT '',
  `table_name` char(64) NOT NULL DEFAULT '',
  `partition_name` char(64) NOT NULL DEFAULT '',
  `job_info` text NOT NULL,
  `processed_rows` bigint(64) unsigned NOT NULL DEFAULT '0',
  `start_time` timestamp NULL DEFAULT NULL,
  `end_time` timestamp NULL DEFAULT NULL,
  `state` enum('pending','running','finished','failed') NOT NULL,
  `fail_reason` text DEFAULT NULL,
  `instance` varchar(512) NOT NULL COMMENT 'address of the TiDB instance executing the analyze job',
  `process_id` bigint(64) unsigned DEFAULT NULL COMMENT 'ID of the process executing the analyze job',
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */,
  KEY `update_time` (`update_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)
tiup playground v8.4.0 --db.binpath /Users/rustin/code/tidb/bin/tidb-server --tag 20240329-test5
mysql> show create table mysql.analyze_jobs;
ERROR 2013 (HY000): Lost connection to MySQL server during query
No connection. Trying to reconnect...
Connection id:    786432006
Current database: *** NONE ***

+--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table        | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| analyze_jobs | CREATE TABLE `analyze_jobs` (
  `id` bigint unsigned NOT NULL AUTO_INCREMENT,
  `update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `table_schema` char(64) NOT NULL DEFAULT '',
  `table_name` char(64) NOT NULL DEFAULT '',
  `partition_name` char(64) NOT NULL DEFAULT '',
  `job_info` text NOT NULL,
  `processed_rows` bigint unsigned NOT NULL DEFAULT '0',
  `start_time` timestamp NULL DEFAULT NULL,
  `end_time` timestamp NULL DEFAULT NULL,
  `state` enum('pending','running','finished','failed') NOT NULL,
  `fail_reason` text DEFAULT NULL,
  `instance` varchar(512) NOT NULL COMMENT 'address of the TiDB instance executing the analyze job',
  `process_id` bigint unsigned DEFAULT NULL COMMENT 'ID of the process executing the analyze job',
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */,
  KEY `update_time` (`update_time`),
  KEY `idx_schema_table_state` (`table_schema`,`table_name`,`state`),
  KEY `idx_schema_table_partition_state` (`table_schema`,`table_name`,`partition_name`,`state`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.02 sec)

@Rustin170506
Copy link
Member Author

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 16, 2024
Copy link
Member Author

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check (PR reviewed by myself and ready for feedback.)

Signed-off-by: Rustin170506 <[email protected]>
Signed-off-by: Rustin170506 <[email protected]>
@Rustin170506
Copy link
Member Author

/retest

@Rustin170506 Rustin170506 added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Dec 17, 2024
Copy link

ti-chi-bot bot commented Dec 17, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: benmeadowcroft, D3Hunter, Leavrth, qw4990, tiancaiamao

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Dec 17, 2024
@Rustin170506
Copy link
Member Author

/retest

@ti-chi-bot ti-chi-bot bot merged commit dde1dcd into pingcap:master Dec 17, 2024
30 checks passed
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Dec 17, 2024
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.5: #58355.

Copy link

ti-chi-bot bot commented Dec 19, 2024

@Rustin170506: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-lightning-integration-test e2c0dd9 link true /test pull-lightning-integration-test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@Rustin170506
Copy link
Member Author

I test this PR again locally to evaluate the performance of creating indexes. For 100k rows, it takes 16 seconds to create the indexes, although it is not that slow, but it still takes some time. So I decided to undo part of this change. We will only create the new indexes for the new cluster. And we do not create the index for the old clusters during the upgrade process.

Normally, for the smaller cluster, this should not be a problem. But for some huge clusters, we can ask users to manually create it instead of blocking the upgrade process.

> ALTER TABLE mysql.analyze_jobs ADD INDEX `idx_schema_table_state` (`table_schema`, `table_name`, `state`)
[2024-12-30 14:17:40] completed in 5 s 755 ms
> ALTER TABLE mysql.analyze_jobs ADD INDEX `idx_schema_table_partition_state` (`table_schema`, `table_name`, `partition_name`, `state`)
[2024-12-30 14:17:52] completed in 11 s 860 ms

@Rustin170506 Rustin170506 deleted the rustin-patch-analyze-jobs-index branch December 30, 2024 06:24
ti-chi-bot bot pushed a commit that referenced this pull request Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

mysql.analyze_jobs missed indexes
7 participants