Re: Multithreading, callbacks from unknown threads [OT]

[William Ahern <william@...>]

On Tue, Mar 20, 2012 at 10:40:07AM -0700, Sam Roberts wrote:
> On Sat, Mar 17, 2012 at 3:59 AM, Gaspard Bucher <gaspard@teti.ch> wrote:
> > On Sat, Mar 17, 2012 at 9:41 AM, Ross Bencina <rossb-lists@audiomulch.com>
> >> On 17/03/2012 7:11 PM, Gaspard Bucher wrote:
> >>> https://github.com/lubyk/lubyk/blob/master/modules/lk/include/lubyk/Fifo.h
> 
> Its not clear why you are going through the trouble of not passing
> your data through the pipe.
> 
> Since a pipe has FIFO semantics, and you have small amount of data,
> couldn't you just write/read the entire packed data through the pipe,
> instead of just a notification byte? Are you not doing this because
> you benchmarked performance, and found it to be a bottleneck?
> 
> You're already suffering context switching overhead, the memory copy
> path through the kernel might not be much worse than the user-space
> memory copying you are doing.
> 
> Cheers,
> Sam

Premature optimization ;) I work on an application which does on-the-fly
multimedia transcoding (WMA->AAC, FLV->RTP, RTSP->HTTP, 44.1khz->48khz, etc)
and can do well over a dozen streams (including resampling) on a single core
(circa 2008 xeon). The pipeline uses socket pairs between processes, because
sockets already handle flow control and I figured I could always optimize
later. I've never had to return to it to optimize ;) I know deep down that I
could reduce memory I/O by reducing that extra copy, and yet in reality it's
just not worth it yet. (Memory I/O is _the_ bottleneck, mind you, but
there are other, easier optimizations that can be had.)

It's amazing how being judicious with the complexity of your code can result
in good performance. I can knock the socks off of FFmpeg, VLC, and
Quicktime, and I'm hardly even trying.