From: Mark Hubbart <discord@...>
Date: 2004-02-28T17:15:10+09:00
Subject: Re: (noob) need help parsing Apache log file

On Feb 27, 2004, at 10:19 PM, Koncept wrote:

> Being very new to Ruby, I want to be able to write a small script to
> open my Apache access_log file and remove any lines that contain (\.exe
> | \.ida) from the source file. I managed to figure out how to get this
> information and log it to a new file, but I would really appreciate an
> example of how to read and remove from one file. I am the kind of
> person who learns best watching others.
>
>
> This is what I wrote to parse the file. Any revisions, hints, hacks,
> etc would also be very much appreciated.

One small change will write out the file. Instead of:

> arr.each do |line|
>    msEvil.push line if line  =~ /(\.exe| \.ida)/
> end

try:

# reject! deletes matched lines
arr.reject! do |line|
	msEvil.push line if line  =~ /(\.exe| \.ida)/
end
# truncate the file and write out
File.open(src, "w+"){|f| f.write arr.join }


Array#reject! does the same thing as Array#sort, except it pays 
attention to the return value of the passed block. If the block returns 
a true value, it deletes the item from the array. There is also the 
non-destructive version, Array#reject (no bang), but reject! makes more 
sense here.

File.open opens the file (surprise!). In this case, it truncates it and 
opens it write only. If you give it a block, it passes the opened file 
handle to the block, and automagically closes the file when it's done 
:) You might want to use the same trick at the bottom:

> # log to text file
> f = File.new("codeRed.txt", "w+")
>
> msEvil.each do |line|
>    f.puts line
> end
>
> f.close

could become:

# log to text file
File.open("codeRed.txt", "w+"){|f| f.write msEvil.join }


That's a handy Ruby idiom. Shortens code considerably :)

And finally, here's the results of running this script on my logs:

mark@imac% sudo mslogstripminer
Password:
                 Report for: Sat Feb 28 00:06:42 PST 2004
                 ----------------------------------------

Total lines:                                                        1670
Total Window exploits:                                               104
mark@imac%

And that's my box at home!!! absolutely disgusting. :)

--Mark