Re: Why is I/O slow?

On Thu, Jun 20, 2002 at 05:07:53PM +0900, Clifford Heath wrote:
> Note that Linux "cat" doesn't move the data twice. Instead it mmap's
> the file and writes that, which apparently in this case *does* actually
> transfer the data at least once - which it shouldn't need to... I would
> have thought it would mmap pages set to fault on read, so that untouched
> pages never get read.
>
> Either way, ruby can't do this,

It is possible to use mmap from Ruby (this requires Ruby 1.7):

  require 'dl/import'
  require 'dl/struct'

  module Mmap
    extend DL::Importable

    dlload 'libc.so.6'

    typealias 'size_t', 'unsigned long'
    typealias 'ssize_t', 'long'
    typealias 'off_t', 'unsigned long'
    extern 'void * mmap(void *, size_t, int, int, int, off_t)'
    extern 'ssize_t write(int, void *, size_t)'

    PROT_READ = 0x1; PROT_WRITE = 0x2; PROT_EXEC = 0x3; PROT_NONE = 0x0
    MAP_SHARED = 0x1; MAP_PRIVATE = 0x2; MAP_FIXED = 0x10
    MAP_FAILED = DL::PtrData.new(-1)
  end

  infile = File.open("test.dat", File::RDONLY)
  size = infile.stat.size
  inmem = Mmap.mmap(nil, size, Mmap::PROT_READ, Mmap::MAP_PRIVATE, infile.to_i, 0)
  raise "unable to mmap test.dat" if inmem == Mmap::MAP_FAILED

  outfile = File.open("out.dat", File::WRONLY | File::CREAT | File::TRUNC)
  Mmap.write(outfile.to_i, inmem, size)

Some systems have "copy file" system call that could be used this way,
too.

> but it can use fread! What about it Matz?
> Since fread is almost as fast as read, the restriction on not mixing
> sysread and read could perhaps be relaxed too?

Sounds like a good idea to me.  I'm sure there's a good reason for it
being written the way it is.

Paul