Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Advanced: Allowing User to Specify Compression Program #3

Open
Proplex opened this issue Mar 16, 2017 · 1 comment
Open

Advanced: Allowing User to Specify Compression Program #3

Proplex opened this issue Mar 16, 2017 · 1 comment
Labels

Comments

@Proplex
Copy link
Member

Proplex commented Mar 16, 2017

This ability would be nested away in aegis.json and not changeable via Aegis runtime. Perhaps built-in support for pigz and 7zip?

config value something like:
"compression-program": "tar czf {1} {0}",

where {0} is the directory to be compressed and {1} is the output so that a user could change it like so:
"compression-program": "tar cf - {0} | pigz > {1}",

@Proplex
Copy link
Member Author

Proplex commented Mar 16, 2017

Actually thinking about this further, this should be a core feature, but implemented such that a user doesn't have to manually edit config. Current thoughts:

pixz is an interesting solution. While I want backups to perform quickly to minimize server overhead during a backup, the panic of "OH GOD I NEED TO RESTORE, NOW" really hits when you're unzipping a .tar.gz for a single file because gzip doesn't provide indexing and you're sweating for twenty minutes with angry users. So I'm at this weird crossroad.

The common case is compression, and pigz excels at that. Compressing a MC server folder @ 5.33GB goes down to 3.5GB in both pixz and pigz, but pixz took 3m28s while pigz took 0m40s. That's a LOT shorter, but the decompression time for a single file (most restore cases) is instantaneous with pixz, while you have to decompress the whole tar with pigz. (which takes some time, and in the case of my 50GB+ server folders, each second feels like an hour off my life worth of stress)

Since the amount of times a person compresses a server folder vs. restoring from backup is heavily lopsided towards compression, I believe pigz is the better default option. As such, the current compression setup will look as follows:

pigz
gzip
zip
pixz
xz

in that order. Aegis will check the system for those installed programs and operate in that order. A user can switch between the various algos with a reconfig suiting their purpose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant