[svlug] select/poll and EOF

Krishnan Nagarajan slugnet at xanadu.nagarajan.net
Fri Aug 15 03:26:11 PDT 2003

On Thu, 14 Aug 2003 22:44:14 -0700 (PDT)
Scott Hess <scott at doubleu.com> wrote:

> Hmm, sounds like you're sort of screwed, then.  You need to spool the data 
> to a queue _somewhere_.  If it's a file, then you can't select on growth.  
> If it's something that's selectable, the system isn't going to provide 
> enough queue to make it worthwhile.
> As you probably got from your research, select() on file descriptors which
> corrospond to a file is broken by design.  In the past when I've brought 
> the issue up with OS people, they basically dismiss it as an issue, even 
> though it can cause substantial problems with certain architectures 
> (single-process select-based servers are very efficient for handling 
> network, pipe, and tty traffic - it would be nice to apply that approach 
> to servers which need disk I/O, without having to go to async I/O).
> Well, polling shouldn't be _too_ horrible, performance-wise.  You're
> looking at a tight loop of usleep/stat calls, which should be pretty low
> overhead, so long as you're only doing it like 8 or 16 times per second
> (fast enough for a human, but slow by computer standards).
> Another option would be to look for a way to receive metadata
> notifications.  FreeBSD has some stuff around kqueue/kevent (I believe)
> which lets you effectively do a select-like operation waiting for things
> like file size changes.  Perhaps Linux has something similar.
> Good luck,
> scott
> On Fri, 15 Aug 2003, M. Drew Streib wrote:
> > On Thu, Aug 14, 2003 at 10:18:12PM -0700, Scott Hess wrote:
> > > Does it need to be a real file, or similar to a file?  If you can live
> > > with a named pipe (aka fifo), then select() should work.  mknod(1) to
> > > create (or mknod(2), of course), then open for reading and run your
> > > select() loop as normal, and have the other side write to it.
> > 
> > Unfortunately, the writing process is recording the session independently
> > of any reading processes, so a fifo may be out.
> > 
> > In particular, the current implementation is at http://dtype.org/nethack,
> > or just telnet to nethack.alt.org and (w)atch games in progress.
> > The tty playback engine is pretty cool now, and allows you to view tty
> > sessions in progress (nethack games), but is doing so now at a 1/4
> > second read() loop. I can't help but think there must be some way
> > to block on data available to read somehow...
> > 
> > I thought about using a fifo and a 'cat' process, but 'cat' ends at
> > eof. I could 'tail' into a fifo, but tail also uses a cheese method
> > of only read()ing at certain intervals.
> > 
> > Since the writing process is sort of independent, and the write/read
> > isn't 1to1, I think fifo isn't my solution. :/ 
> > 
> > Thanks though. Any further thoughts?
> > 
> > -drew
> > 
> > 
> _______________________________________________
> svlug mailing list
> svlug at lists.svlug.org
> http://lists.svlug.org/lists/listinfo/svlug

Here's a highly custom-built approach that might solve this problem.
Stepping away from the idea of two processes communicating through
a file's contents, and instead create a third server process with which each of the
tow original processes (the reader and writer) communicate - e.g. through 
two sockets.
This server process would serve the following purposes :-
1) Accept connect and write requests from writers, and perform the actual 
write to file.
2) Accept connect and read requests from readers, maintain reader state within
the file, and provide the callback when new data is available.

Only new writes by writers would need to be considered to wake up sleeping

More information about the svlug mailing list