Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Load the new partition 10 minutes #38166

Open
1 task done
yuan8180 opened this issue Dec 3, 2024 · 4 comments
Open
1 task done

[Bug]: Load the new partition 10 minutes #38166

yuan8180 opened this issue Dec 3, 2024 · 4 comments
Assignees
Labels
kind/bug Issues or changes related a bug stale indicates no udpates for 30 days triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@yuan8180
Copy link

yuan8180 commented Dec 3, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version:2.3.1
- Deployment mode(standalone or cluster):cluster
- MQ type(rocksmq, pulsar or kafka):    kafka
- SDK version(e.g. pymilvus v2.0.0rc2):milvus-sdk-go v2.3.1
- OS(Ubuntu or CentOS): Ubuntu 
- CPU/Memory: 32/31G
- GPU: 
- Others:

Current Behavior

现象:(2号8点9分)创建新的partition([collectionID=453265542206718289] [partitionIDs="[454319225442612106])后,卡在加载10分钟,10分钟后恢复
对比:同一分钟创建另一个新的partion加载立刻生效([collectionID=453265542206718774] [partitionIDs="[454319225442411772])
初步定位:看了下日志是09分创建分区成功后root节点发消息给query节点创建分区的segment分段,消息在kafka里没送到query节点,root节点在9分-19分期间重试了6次失败后关闭context,然后19分的时候kafka重新初始化通道之后,消息传递成功,然后segment创建成功之后分区才加载成功

Expected Behavior

创建分区后加载在1分钟内成功

Steps To Reproduce

No response

Milvus Log

milvus-log.zip

Anything else?

No response

@yuan8180 yuan8180 added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Dec 3, 2024
@yanliang567
Copy link
Contributor

/assign @bigsheeper
/unassign

@yanliang567 yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Dec 3, 2024
@bigsheeper
Copy link
Contributor

Hi @yuan8180 ,

It looks like you're using version 2.3.1, which is quite old. Many of the issues present in older versions have been addressed and resolved in the 2.4 series. We highly recommend performing a rolling upgrade to 2.4 to take advantage of these fixes, as well as the improved stability and performance.

If you need any assistance or have questions about the upgrade process, please feel free to reach out—we're here to help!

@yuan8180
Copy link
Author

ok, We will keep checking to see if it reappears after the upgrade

Copy link

stale bot commented Jan 11, 2025

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

@stale stale bot added the stale indicates no udpates for 30 days label Jan 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug stale indicates no udpates for 30 days triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

No branches or pull requests

3 participants