From: Panagiotis Atmatzidis <ml@...>
Date: 2011-08-15T02:31:38+09:00
Subject: Unwanted scan() method behavior

Greeting to everyone from Greece!

I'm writing a string which will make a list of 'fail2ban.log' captured IP addresses dump them into SQLite and them display some statistics via Sinatra. I'm using scan() method to grab the needed lines. The lines I'd like to grab are like this:
--
2011-07-23 02:04:51,107 fail2ban.actions: WARNING [ssh-ipfw] Ban 78.xxx.xxx.17x 
--
Although these are ssh brute-force login attempts I'd switch numbers with 'x' letters in the above sample.

A typical fail2ban.log file is like this:

--
2011-07-23 02:03:50,741 fail2ban.server : INFO   Changed logging target to /var/log/fail2ban.log for Fail2ban v0.8.4
2011-07-23 02:03:50,743 fail2ban.jail   : INFO   Creating new jail 'ssh-ipfw'
2011-07-23 02:03:50,745 fail2ban.jail   : INFO   Jail 'ssh-ipfw' uses poller
2011-07-23 02:03:50,853 fail2ban.filter : INFO   Added logfile = /var/log/secure.log
2011-07-23 02:03:50,856 fail2ban.filter : INFO   Set maxRetry = 3
2011-07-23 02:03:50,859 fail2ban.filter : INFO   Set findtime = 600
2011-07-23 02:03:50,861 fail2ban.actions: INFO   Set banTime = 600
2011-07-23 02:03:51,030 fail2ban.jail   : INFO   Jail 'ssh-ipfw' started
2011-07-23 02:04:51,107 fail2ban.actions: WARNING [ssh-ipfw] Ban 78.xxx.xxx.17x 
2011-07-23 02:14:51,441 fail2ban.actions: WARNING [ssh-ipfw] Unban 78.xxx.xxx.17x
2011-07-23 02:04:51,107 fail2ban.actions: WARNING [ssh-ipfw] Ban <ip>
2011-07-23 02:04:51,107 fail2ban.actions: WARNING [ssh-ipfw] Ban <ip>
2011-07-23 02:04:51,107 fail2ban.actions: WARNING [ssh-ipfw] Ban <ip>
2011-07-23 02:04:51,107 fail2ban.actions: WARNING [ssh-ipfw] Unban  <ip>
2011-07-23 02:04:51,107 fail2ban.actions: WARNING [ssh-ipfw] Unban  <ip>
2011-07-23 02:04:51,107 fail2ban.actions: WARNING [ssh-ipfw] Unban  <ip> 
--

Here is my script so far:

class Myzonereport
 attr_reader :logfile

 def initialize(logfile)
   raise "No fail2ban log file found!" if (logfile.empty?)
   @logfile = logfile
 end

 def readlog
   puts "I can't read the log file" unless (File.readable?(@logfile) || File.empty?(@logfile))
   log = File.read(@logfile)
   log.scan(/^(\d{4}-\d\d-\d\d).*?(\d{2}:\d{2}:\d{2},\d{3}).*?(Ban).*?(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/).each do |date, time, string, ip|
       puts "id: #{time} | date: #{date} | IP: #{ip}"
   end
 end

end

x = Myzonereport.new('fail2ban.log')
puts x.readlog


My problem though is that the output is printed 2 times. The first time in the form I want using puts and a second time in 'raw mode'. Like this:
--
id: 23:37:50,235 | date: 2011-08-09 | IP: <ip>
id: 02:09:32,868 | date: 2011-08-10 | IP: <ip>

2011-07-23
02:04:51,107
Ban
<ip>
2011-07-23
05:22:45,963
Ban
<ip>
2011-07-23
12:07:25,377
Ban
<ip>
[…]

I can't tell why this happens. Should I use another method in order to grab the pattern I want? Is this scan's default behavior? I'm getting same results if I don't use any (puts or other) method in the loop.

Best Regards & thanks in advance for your time

--
Panagiotis Atmatzidis

personal: atma@convalesco.org
lists: ml@convalesco.org
blog: http://www.convalesco.org

The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."