[ale] Dealing with really big log files....

Kenneth Ratliff lists at noctum.net
Sun Mar 22 16:12:18 EDT 2009


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> A 114GiB log file certainly will compress like mad, either via gzip or
> bzip2 - the former is faster to compute; the latter generally gives
> smaller output.  Once you've done that and pulled over the compressed
> copy for local use, use rsync -z to keep your local copy synced to the
> server's.

Thought about that too, but even compressed, it's still 15gigs, and  
it's a slow link, it wouldn't transfer before the end of my shift.

The tail method worked pretty well though, knocked it down to 3.2 gigs  
in size. I should be able to use my normal tools to extract the info I  
need out of it. 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (Darwin)

iEYEARECAAYFAknGm6UACgkQXzanDlV0VY5CUgCffdyyFEPnMtbDZgjaqifrnRSw
NksAn3rkACpG2TDzDkcIMfLKWWTtVVbU
=bO1c
-----END PGP SIGNATURE-----



More information about the Ale mailing list