So looking up a set of values (say IP address) from somefilesource in a bunch of logs (300 files with ~ 500k lines)
$ wc -l Some*
3840 SomeFilesource
fgrep:
$ time fgrep -hf SomeFilesource *log* > Wantedlogs
real 0m0.952s
user 0m0.890s
sys 0m0.030s
grep:
time grep -f SomeFilesource *log* > Wantedlogs
real 33m45.601s
user 33m39.150s
sys 0m0.350s
That is quite a speed up. From now on it's fgrep by default... Now if I need to see whether the --mmap optimization speeds up subsequent [f]greps.
No comments:
Post a Comment