From: Jeff Garland (jeff_at_[hidden])
Date: 2004-05-22 09:45:15
On Sat, 22 May 2004 20:20:07 +1000, Arash Partow wrote
> Hi all,
>
> I was wondering if there would be any interest in a template
> library based around producers and consumers.
Sure, but....
> I've developed something already and have a series of working
> examples, I propose a library that consist of 3 basic entities, a
> producer a consumer and a hybrid entity called an interchange.
>
> the entities themselves will be threaded and will be linked together
Making each element of the processing chain a thread is rather constraining.
What if I want to use a leaders / followers approach to threading my chain of
transformers? So when I get a callback on my socket I chain thru all the
transformations in the callback thread thus avoiding several thread context
switches just to process the data.
> via a linking mechanism that is itself a base template but which can
> be extended to provide data transfer between same system processes
> and threads and also extended versions which provide data transfer over
> pipes, sockets, serial lines etc..
>
> the idea of the producer-consumer is very simple the producer
> produces a type of data ie: string, a class, a struct etc.. and a
> consumer that is instantiated as a consumer of the type will consume
> the data provided by the producer and do something with it.
How does this differ from the signals and slots library?
http://www.boost.org/doc/html/signals.html
Couldn't we just create a slot endpoint for the various types? Can I connect
2 or more consumers to a particular interchange? For this to be useful, I
should be able to.
>...snip detail...
>
> The link mechanism is a basically a type that supports the common
> type between a producer and its consumer, it is a one way flow of
> data, the consumer is signaled when data is ready for it to collect
Again, seems like signals and slots to me...
>...snip...
>
> A link has a basic get and add functionality but depending what
> entity is being implemented they will only see it via an interface
> and hence only be able to invoke 1 kind of method, a producer can
> only invoke "add" where-as a consumer can only invoke a get, an
> interchange can invoke both because it is instantiated with both a
> consumer and producer link interface, but the links are separate for
> an interchange, because it requires 2 links, one link for the
> incoming data and one link for the out going data. both link maybes
> of different types, ie:
>
> Producer <string> ----> Interchange <string, AClass> ----> Consumer <AClass>
>
> I need some opinions, about whether or not people will find this
> library useful and hence have it as part of Boost, if so could people
> give me their opinions about my design, its just a simple first trial
> run at designing it, i intended for the producers and interchange
> entities to be policy driven as far as production of data is
> concerned.
Well, I think it could be useful and wouldn't mind having it in boost, but it
seems like we need other basic infrastructure like a reactor, callback timers,
and sockets before there will be much advantage.
> I think this library can be used in distributed computing, in network
> packet assembly, and also for interfacing and passing data up and
> down threaded software layers.
Yes, it is a useful design in 'data flow' systems. Distributed
publish-subscribe type systems which have various data transformers use this
type of design (or they should).
Jeff