From: schuerig@... (Michael Schuerig)
Date: 2001-02-03T11:10:12+09:00
Subject: [ruby-talk:10298] Local directory search "server" script


Now that my X server is working again I've put together the first
version of my first half-way useful script. It is a "one shot" server
that's started by inetd for each request on a specific port set in
/etc/inetd.conf. That is at least how I use it on Linux; I can't say
what to do on Windows.

*** If you use this script, or even only try it, make sure that your
firewall blocks the port where it is listening! ***

The script uses two variants of grep: agrep and sgrep. The former
because it's a lot faster than what I could come up with for searching
files; the later because I've been lazy. Incidentally, sgrep is pretty
slow for this task, but it is incredibly neat for ad hoc extracting of
parts from structured files -- are there similarly comfortable ways to
do it natively in Ruby?

Todo
- Browser integration through a JavaScript triggered from the toolbar.
- Resolving of symbolic links.
- Searching of compressed files.
- Nice output formatting.
- Replacement of sgrep with something faster.
- "Transclusion" of CGI.unescape


Thanks for your help in writing this piece.

Michael



#! /usr/bin/ruby -w

require 'cgi'


def main 
  @myname = File.basename($0)
  @searchprog = "agrep -clr"
  @hits = {}
  @totalocc = 0

  args = parseArgs(getRequest())

  search(args['pattern'], args['dir'])
  output(args['pattern'], args['dir'], @hits.sort { |a, b| b[1] <=> a[1]
})

  exit 0
end


def getRequest()
  rawRequest = $stdin.sysread(1024)
  request = rawRequest.split('\r\n').first
  request =~ /searchdir\?(.*) HTTP/
  $1
end


def parseArgs(rawArgs)
  args = {}
  rawArgs.split('+').each do |kv|
    key, value = kv.split('=')
    args[key.strip().downcase()] = CGI.unescape(value.strip())
  end
  args
end


def search(pat, dir)
  matches = `#{@searchprog} #{pat} #{dir}`
  raise "Search for \"#{pat}\" in #{dir} failed with #{$?}!" if $? != 0
  matches.each() do |m| 
    file, occ = m.split(':')
    occ = occ.to_i
    if occ > 0
      @hits[file] = occ
      @totalocc += occ
    end
  end
end


def output(pat, dir, hits)
  print "HTTP/1.1 200/OK\r\nContent-type: text/html\r\n\r\n"

  print "<html>\r\n"
  print "<head><title>Search result for \"#{pat}\"</title></head>\r\n"
  print "<body>\r\n"
  print "<p><h1>Found #{@totalocc} occurrences of \"#{pat}\" in
#{dir}</h1></p>\r\n"
    
  hits.each do |hit|
    formatHit(*hit)
  end

  print "</body></html>\r\n"
end


def formatHit(file, occ)
  linktext = file
  if html?(file)
    title = extractTitle(file)
    if !title.empty?
      linktext = title
    end
  end
  print "<p><a href=\"file://#{file}\">#{linktext}</a></p>\r\n"
end


def html?(file)
  name = File.basename(file)
  ext = file.split(".").last()
  ext.downcase!()
  ext.include?("htm")
  #false
end


def extractTitle(file)
  # This is slow, but excessively practical
  `sgrep -g html 'first(1, stag("TITLE") __ etag("TITLE") in
(stag("HEAD") .. etag("HEAD")))' #{file}`.strip()
end


main()


-- 
Michael Schuerig
mailto:schuerig@acm.org
http://www.schuerig.de/michael/