From: Thorsten Hater <th@...1.rub.de>
Date: 2009-06-09T23:34:08+09:00
Subject: Re: [QUIZ][Solution] Twitter Personalities (#208)

Hi,

here my implementation of a Markov chain based text generator for Twitter.

class MarkovText
 
  def initialize file
    @word_map = {}
    do_preprocessing file
  end

  def do_preprocessing file
    text = File.readlines(file).collect{|l| l.chomp}.join
    text.gsub! /([a-zA-Z0-9])([,:.!?;])/, '\1 \2'
    text.squeeze! ' '
    words = text.split ' '
    for i in (0..words.size-2)
      # if we encounter this word first initialize with empty Hash
      @word_map[words[i]] ||= {}
      @word_map[words[i]][words[i+1]] ||= 0.0
      @word_map[words[i]][words[i+1]]  += 1.0
    end

    # generate prob distribution for text
    @word_map.keys.each do |k|
      sum = @word_map[k].values.inject(&:+)
      @word_map[k].keys.each{|l| @word_map[k][l] /= sum}
    end
  end

  def generate start_word = nil
    text = [start_word || @word_map.keys[rand(@word_map.keys.size-1)] ]
    while !(/[.!?:;]/ === text.last) do
      w = choose_next(text.last)
      break if !w
      text << w
    end
    text
  end

  def choose_next word
    roll = rand
    cand = @word_map[word]
    return nil if !cand
    i = 0
    while roll > cand.values[i]
      roll -= cand.values[i]
      i += 1
    end
    cand.keys[i]
  end
end

And the inteface code which maintains a collection of text generators and
does interfacing with the output (stdout at the moment) and restriction to
140 chars etc:

# load Markov chain text generator
require 'markov_text'

class TwPerson

  def initialize min, max, *files
    @max  = max
    @min  = min

    @name = "Mark V. Shaney"
    @pwd  = "markovianSplitPersonality"

    @gens = []
    files.each do |f|
      @gens << MarkovText.new(f)
    end

    @pool = []
    generate_text
  end

  def run
    loop do
      make_post
      time = rand(@max-@min) + @min
      puts "> Made my post, sleeping for #{time}s"
      sleep(time)
    end
  end

  def make_post
    puts "> Trying to post"
    text = ""
    while  (text.length + @pool.last.length) < 139
      text += " " + get_next
    end
    text.squeeze! ' '
    text.gsub!(/( )([.,;:?!])/) do |m|
      m[1]
    end
    do_post text
  end

  def generate_text
    puts "> Generating new text pool"
    m = @gens.shift
    @pool += m.generate
    @gens << m
  end

  def do_post txt
    puts txt
  end

  def get_next
    generate_text if @pool.size < 2
    @pool.shift
  end

end

This solution is quick and dirty and does not claim to be either elegant
or free from errors ;)

Thorsten

Daniel Moore wrote:
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>
> The three rules of Ruby Quiz:
>
> 1.  Please do not post any solutions or spoiler discussion for this
> quiz until 48 hours have elapsed from the time this message was
> sent.
>
> 2.  Support Ruby Quiz by submitting ideas and responses
> as often as you can!
> Visit: http://rubyquiz.strd6.com/suggestions
>
> 3.  Enjoy!
>
> Suggestion:  A [QUIZ] in the subject of emails about the problem
> helps everyone on Ruby Talk follow the discussion.  Please reply to
> the original quiz message, if you can.
>
> RSS Feed: http://rubyquiz.strd6.com/quizzes.rss
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>
> ## Twitter Personalities (#208)
>
> Merhaba Rubyists,
>
> This week's quiz is to create a program that will generate messages
> 140 characters in length. There primary use will be to create a
> Twitter "personality". At the end of the quiz period these
> "personalities" will be unleashed on the internet and we'll see how
> they do in the wild.
>
> The programs will consist of two parts: a component for interacting
> with Twitter, and a top secret "personality" module.
>
> For the Twitter interface component there will be no no-spoiler
> period. Please feel encouraged to discuss different libraries or
> methods on the mailing list. Let's all work together to find the best
> interface.
>
> The "personality" component can take any inputs and will produce a 140
> character message when called. The "personality" may remember state.
> The no-spoiler period applies for the "personality" component; please
> save them until everyone has had a chance to consider their own
> implementations.
>
>
> Have Fun!
>