From: see@... (Douglas Wells) Date: 2007-04-13T08:10:06+09:00 Subject: Re: Missing packets in multicast receiver. In article <461D210E.4020609@awayfar.org>, "James W. Mills" writes: > Hello list. > > I need to use multicast in a project I am working on. I would like to > have a multicast sender send a large amount of multicast data, and a > multicast receiver receive it. Hi, You seem to have a number of the networking ideas properly in your head, but you also seem to be some erroreous fundamental concepts. So, I'm going to go over some basics. (Please note that I'm also going to rearrange your post for purposes of exposition.) First, I should note that I didn't notice any useful clues about your execution environment (other than your use of /dev/zero), including OS and whether you were communicating between two different networks hosts or looping back on one host. These differences would introduce various effects on your symptoms. Second, I should note that I ran your programs (all three) on my system after making a few fundamental changes (more about that later) and found that the ruby and perl receivers behave just about the same. (Just for the record, I was running on FreeBSD using a loop-around configuration) So, let me start with your concluding analysis: > If you run the perl receiver against the Ruby sender, the programs print > out the same messages, as expected. I am very curious to know what I am > doing wrong with the Ruby receiver that is causing it to miss so much > data. I suspect the line: > > sock.setsockopt(Socket::IPPROTO_IP, Socket::IP_ADD_MEMBERSHIP, ip) > > in the receiver might hold the answer, but so far nothing I have tried > has made any difference. I find it interesting that it is using TCP > instead of UDP/DGRAM, but I was unable to get it to work with IPPROTO_UDP... Here you show a fundamental understanding about the IP stack. First, there's nothing here that has anything to do with TCP. If there were, it would have used Socket::IPPROTO_TCP. IP (Internet Protocol) underlies *both* TCP and UDP. In addition, multicast, whose official name is "IP Multicast," has many differences from UDP. > What I want initially is to be able to blast packets to a multicast > group. I wrote a simple multicast sender that prints a message every > (almost) MB of data it sends. I also wrote a simple receiver that does > the same thing for every (almost) MB of data it receives. Here they are. Your use of the term "blast" here is telling. You need to understand that IP Multicast (and UDP) are "unreliable" protocols. When you transmit something using TCP, which is a "reliable" protocol, the operating system transmits that the data and then holds onto a copy until the receiver acknowledges successful receipt. At some point, the OS will decide that you have consumed your fair share of buffer space, and it will block you from sending any more data until the receiver acknowledges delivery of some earlier data (and the OS can safely discard the copy that it has been storing). In the case of IP Multicast (and UDP), there is no back-channel to provide such data. Thus, there will never be any acknowledgment of received data and no cue to discard stored data. Instead, most network stack implementations (that I am aware of) discard the data immediately after it is transmitted over the wire. Consider the case where your processor can generate data faster than the network can transmit it, which is almost certainly the case in your environment. Now consider what happens when you use up your "fair share" of buffer space. What is the OS to do? It boils to down to three choices: 1) block the process, 2) discard some data silently, and 3) report an error. All three options have been taken by various OSs at some point in the past. Today most systems implement either (2) or (3). My system (FreeBSD) implements (3), but previous BSD systems have implemented (2). I don't immediately remember what Linux does. Now let's look at your sender loop: > loop do > data = data_source.read(1024) > data_length += data.length > if (data_length % 102400) == 0 > puts data_length > end > socket.send(data, 0, MULTICAST_ADDR, PORT) > end You are looping here over and over with no intervening pause. You're almost certainly spending most of your time calling into the OS and having it discard data. The fact that your system doesn't report any error on the socket.send makes me think that your OS implements technique (2) above and silently files the data in the bit bucket. So why does the Perl program behave "better?" I don't know, and frankly there are so many possible reasons that it's not worth my while chasing. In order to make your program work (as described above), I introduced a millisecond sleep every time the OS reported that it had discarded data. (And I still flooded my screen with notifications of the discards.) Instead, you should take this exercise as an early warning that your "blaster" application won't work as currently designed. If you pore through networking chatter, you'll find lots of sage advice by the networking grey beards that if you're going to use unreliable datagrams instead of TCP, you will have to work our for yourself how to achieve the level of reliability that you need for your application. Since TCP is not an option for you, you're going to have to solve the problem. Typically, in order to send large amounts of data via datagrams (including via IP multicast), you will need to monitor the amount of data that you are sending, rate-limit it as appropriate, and possibly negotiate reserved bandwidth if that should prove necessary. You will also need to figure out what to do about dropped packets, because the system WILL drop some packets -- even on single node and/or dedicated network systems. You will either need to send redundant data (as most audio/video systems do), or create an auxiliary data correction channel (as some of the P2P file distribution use). > ~james Good luck, - dmw -- . Douglas Wells . Connection Technologies . . Internet: -sp9804- -at - contek.com- .