Re: awk to ruby 1.9

Good evening (it's 8:48 pm here)!

First I would like to thank you all for the responses. I've got really =
much more than I hoped too when I sent the post.

On 06 =CE=94=CE=B5=CE=BA 2009, at 2:30 =CE=BC.=CE=BC., Robert Klemme =
wrote:

> On 05.12.2009 10:07, Panagiotis Atmatzidis wrote:
>> On 04 2009, at 10:27, David Masover wrote:
>>> On Friday 04 December 2009 02:09:35 pm Panagiotis Atmatzidis wrote:
>=20
>>>> The output though parses lines the following:
>>>>=20
>>>> 2009-11-19 00:31:29,928 fail2ban.actions: WARNING [ssh-ipfw] Ban
>>>> 203.169.139.171
>=20
>>>> I would like to use the output from the cli: "$ grep Ban =
fail2ban.log|awk
>>>> -F "Ban" '{print $2}'"
>=20
>> I have found this code snippet, with a slight change runs under =
ruby1.9
>> def ip(filename)     ips =3D []
>>    File.read(filename).lines.to_a.each do |place|
>>      sf =3D 0
>>      while sfn =3D =
place.index(/(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|=
[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4]=
[0-9]|[01]?[0-9][0-9]?)/,sf)
>>        sf =3D sfn + 3
>>        ips << $&
>>      end
>>    end
>>      return ips
>>  end
>=20
> It seems that is making things unnecessary complicated (there is a =
superfluous to_a in there etc.).  That should be sufficient for your =
case:
>=20
> require 'set'
>=20
> def ips(file_name)
>  ips =3D Set.new # [] if you do want dups
>=20
>  File.foreach file_name do |line|
>    ip =3D line[/Ban\s+(\d{1,3}(?:\.\d{1,3}){3})/, 1] and
>      ips << ip
>  end
>=20
>  ips
> end
>=20
> Note: I am using a Set here in order to avoid reporting duplicates.  =
If you want duplicates just use an array as indicated above.

Yes. I did not write this function myself. I found it on the internet, =
and adjust it just a little bit to make it work with 1.9 version. I have =
no idea at this point what place.index does neither "set" which you =
used. Today I reached the "hashes" chapter and still did not finnish it =
all! But I'm eager to write actual programs that's why I started working =
on small project in parallel with the book. It's a good practice helps =
you understand things although it may turn a bit confusing at times.

>=20
> Other than that this is what the code does: File.foreach iterates a =
file line by line.  The expression line[/.../, 1] extracts capturing =
group 1 of the regexp if it matches.  Otherwise you get nil.  Then I use =
that information immediately with "and" to decide whether to add to the =
set or not.  The regular expression is not as selective as the one you =
have found in terms of the IP but it does match the "Ban" so it should =
be safer (after all, you know the format of the file).  Also, the piece =
you found seems to iterate while in your file there is at most one IP =
per line (if your example covers all options).

True. There is 1 ip per line and some duplicates but there's a catch =
also. There are some IP's that are captured more than 1 time with the =
"Ban" flag. Which means that they were captured in a different time. =
Fail2ban blocks the IP for a couple of minutes in order to avoid the =
password brute-force which is taking place. After 3 minutes Unbans the =
ip. So the usual kind of log is this:

2009-11-15 15:19:35,222 fail2ban.actions: WARNING [ssh-ipfw] Ban =
195.66.191.75
2009-11-15 15:29:35,643 fail2ban.actions: WARNING [ssh-ipfw] Unban =
195.66.191.75
2009-11-16 07:46:59,854 fail2ban.actions: WARNING [ssh-ipfw] Ban =
203.172.184.130
2009-11-16 07:57:00,085 fail2ban.actions: WARNING [ssh-ipfw] Unban =
203.172.184.130

[*I leave the IP's intact because these are actual SSH attacks and... if =
the admin don't care for his host, neither do I.]

So at this point, I need really to display duplicates and maybe issue a =
bold warning when an IP appears more than 5 times. It means that your =
host is probably *targeted*.

>=20
> Plus, the iteration with index is awkward and inefficient.  In Ruby =
you would rather do
>=20
> place.scan /.../ do |ip|
>  ips << ip
> end
>=20
> But as I said, scanning is not necessary if there is at most a single =
IP per line.

I'll keep that in mind. I never used the .scan function anyway.

>=20
> And, last but not least a more modular solution in which you do not =
need an Array or Set but rather use Ruby's block feature to deal with =
individual IPs:
>=20
> def ban_ips(file_name)
>  File.foreach file_name do |line|
>    ip =3D line[/Ban\s+(\d{1,3}(?:\.\d{1,3}){3})/, 1] and
>      yield ip
>  end
> end
>=20
> With that you can do
>=20
> ban_ips "fail2ban.log" do |ip|
>  printf "found IP %-15s\n", ip
> end

Although I dislike the printf use here, this is a code snippet that I =
understand entirely, which is a good thing :-)

>=20
> or
>=20
> ips =3D []
>=20
> ban_ips "fail2ban.log" do |ip|
>  ips << ip
> end
>=20
> puts "found #{ips.size} IPs"
>=20
> Welcome to the wonderful world of Ruby!

Thanks!!! THe learning process is for sure much easier than Objective-C, =
the syntax much more straight forward, but some concepts I'm still =
struggling to understand them! Thanks for your reply though, it was very =
enlightening.

>=20
> Kind regards
>=20
> 	robert
>=20
> --=20
> remember.guy do |as, often| as.you_can - without end
> http://blog.rubybestpractices.com/
>=20


PS. yes I know I can install SNORT and get over it, but it's much =
funnier creating your own programs! Oh, this mailing list is *really* =
good :-)

Panagiotis (atmosx) Atmatzidis

email:	[e-mail]
URL:	http://www.convalesco.org
GnuPG ID: 0xFC4E8BB4=20
gpg --keyserver x-hkp://pgp.mit.edu --recv-keys 0xFC4E8BB4
--
The wise man said: "Never argue with an idiot. They bring you down to =
their level and beat you with experience."