Re: Multithreading, callbacks from unknown threads [OT]

[Gaspard Bucher <gaspard@...>]

On Tue, Mar 20, 2012 at 7:27 PM, William Ahern <william@25thandclement.com> wrote:

On Tue, Mar 20, 2012 at 10:40:07AM -0700, Sam Roberts wrote:
> On Sat, Mar 17, 2012 at 3:59 AM, Gaspard Bucher <gaspard@teti.ch> wrote:
> > On Sat, Mar 17, 2012 at 9:41 AM, Ross Bencina <rossb-lists@audiomulch.com>
> >> On 17/03/2012 7:11 PM, Gaspard Bucher wrote:
> >>> https://github.com/lubyk/lubyk/blob/master/modules/lk/include/lubyk/Fifo.h
>
> Its not clear why you are going through the trouble of not passing
> your data through the pipe.
>
> Since a pipe has FIFO semantics, and you have small amount of data,
> couldn't you just write/read the entire packed data through the pipe,
> instead of just a notification byte? Are you not doing this because
> you benchmarked performance, and found it to be a bottleneck?
>
> You're already suffering context switching overhead, the memory copy
> path through the kernel might not be much worse than the user-space
> memory copying you are doing.
>
> Cheers,
> Sam

Premature optimization ;) I work on an application which does on-the-fly
multimedia transcoding (WMA->AAC, FLV->RTP, RTSP->HTTP, 44.1khz->48khz, etc)
and can do well over a dozen streams (including resampling) on a single core
(circa 2008 xeon). The pipeline uses socket pairs between processes, because
sockets already handle flow control and I figured I could always optimize
later. I've never had to return to it to optimize ;) I know deep down that I
could reduce memory I/O by reducing that extra copy, and yet in reality it's
just not worth it yet. (Memory I/O is _the_ bottleneck, mind you, but
there are other, easier optimizations that can be had.)

It's amazing how being judicious with the complexity of your code can result
in good performance. I can knock the socks off of FFmpeg, VLC, and
Quicktime, and I'm hardly even trying.

No, the motivation is not optimization (the Fifo is hardly used), it's to keep the object simple from the user's perspective. I do not want to require library users to write code to serialize data into a stream. Such code is not simple to write depending on what the Fifo is used for.

Not using mutex is not optimization either. I do not like mutex objects because they tend to not play nicely with signals.

As you can see, a memory barrier macro really solves my issues.

                                                               Gaspard