We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
oss 基于多线程逻辑分片,提升大文件逻辑分片的效率
The text was updated successfully, but these errors were encountered:
感觉这个没必要做,多大文件算提高呢,比如1T文件,单机再怎么多线程其实提高并不多,还是得集群处理;比如先按照10G拆分,生成分片任务,然后再次递归,对每个10G按照1G,生成分片任务,以此类推,到每片10M终止
Sorry, something went wrong.
逻辑分片的目的就是为了集群处理分片文件, 大文件一分为十只能一个线程做, 十分分一百可以多个线程并发处理,
明白了,类似reactor模型,boss单线程获得一个请求,先预处理下【大文件一分为十】; 然后交给work Threadpool【十分分一百】👍
No branches or pull requests
oss 基于多线程逻辑分片,提升大文件逻辑分片的效率
The text was updated successfully, but these errors were encountered: