Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intelligent avoid of files re-download #405

Open
wants to merge 42 commits into
base: master
Choose a base branch
from

Conversation

Lucioric2000
Copy link

Hi. I introduced some changes in the Java code to avoid re-download of files.

@imedina imedina added this to the v5.0.0 milestone Oct 3, 2019
@julie-sullivan
Copy link
Collaborator

https://www.gnu.org/software/wget/manual/wget.html#Time_002dStamping

The time-stamping in GNU Wget is turned on using ‘--timestamping’ (‘-N’) option, 
or through timestamping = on directive in .wgetrc. With this option, for each file it 
intends to download, Wget will check whether a local file of the same name exists. 
If it does, and the remote file is not newer, Wget will not download it.

@Lucioric2000
Copy link
Author

https://www.gnu.org/software/wget/manual/wget.html#Time_002dStamping

The time-stamping in GNU Wget is turned on using ‘--timestamping’ (‘-N’) option, 
or through timestamping = on directive in .wgetrc. With this option, for each file it 
intends to download, Wget will check whether a local file of the same name exists. 
If it does, and the remote file is not newer, Wget will not download it.

Yeah. What I added is that after filenane and timestamp comparison, it compares file size, and if the file is smaller in the destination, it assumes that it is a broken download, and downloads the remainder of the bytes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants