From: "Allan A." <lists@...>
Date: 2012-09-01T22:11:18+09:00
Subject: Re: Beginner Help - Encoding error exporting/writing to CSV

Thank you for your response.

Bartosz Dziewoński wrote in post #1074162:
> All I can recomment now is to add .encode('utf-8') (or
> .force_encoding('utf-8')) everywhere where you accept external input.
I've tried that and I must be doing something wrong because I keep
getting undefined method errors when I do

> Can you show us the code? It would be easier to help.
I'll warn you and say that it's not very elegant but it seems to get the 
data correctly:

# encoding: utf-8
require 'nokogiri'
require 'open-uri'
require 'csv'

#define arrays
@thread = Array.new
@filename = Array.new
@postid = Array.new
@title = Array.new
@date = Array.new
@filedate = Array.new
@member = Array.new
@memberurl = Array.new
@content = Array.new

#pull in file/URL list
files = CSV.read("files1.csv")
(0..files.length - 1).each do |index|
#print out current URL or File
  puts files[index][0]
#save it to array
  @filename << files[index][0]
#load HTML to Nokogiri
  doc = Nokogiri::HTML(open(files[index][0]))

#find the first post and ID
 threadurl = doc.css('a[name="1"]').map { |link| link['href'] }
 test = threadurl.to_s
 test = test[2..-3]
#isolate the main/first post ID
  post_id = test.split("#") [1]
  post_id = post_id[4..-1]

#make other versions of post ID references if needed
  postmessageid = "post_message_" + post_id
  postmenu = "postmenu_" + post_id
  pstid = "post" + post_id

#find the Date of first post
  fdate = doc.css('td[class="thead"]')[2].content
  fdate = fdate.strip

#get member name
  membername = doc.css('a[class="bigusername"]')

#get post content
  contentid = "div#post_message_" + post_id
  postcontent = doc.css(contentid)

#write all other arrays
  @title << doc.at_css("title").text[0..-28]
  @filedate << fdate
  @member << membername[0].text
  @memberurl << membername[0]['href']
  @content << postcontent
end

CSV.open("output.csv", "wb:UTF-8") do |row|
  row << ["Thread Title", "Filename - Thread URL", "Date", "Member",
"Member URL", "Content"]
  #(0..urls.length - 1).each do |index|
  (0..files.length - 1).each do |index|
    row << [
      @title[index],
      @filename[index],
      @filedate[index],
      @member[index],
      @memberurl[index],
      @content[index]]
  end
end

-- 
Posted via http://www.ruby-forum.com/.