Asynchronous Reductions


template<typename Ctxt , typename ITag , typename IType , typename ITuner , typename CType , typename CTuner , typename OTag , typename OTuner , typename ReduceOp , typename Select >
graph * make_reduce_graph (CnC::context< Ctxt > &ctxt, const std::string &name, CnC::item_collection< ITag, IType, ITuner > &in, CnC::item_collection< OTag, CType, CTuner > &cnt, CnC::item_collection< OTag, IType, OTuner > &out, const ReduceOp &op, const IType &idty, const Select &sel)

Function Documentation

graph* CnC::make_reduce_graph ( CnC::context< Ctxt > &  ctxt,
const std::string &  name,
CnC::item_collection< ITag, IType, ITuner > &  in,
CnC::item_collection< OTag, CType, CTuner > &  cnt,
CnC::item_collection< OTag, IType, OTuner > &  out,
const ReduceOp &  op,
const IType &  idty,
const Select &  sel 

Creates a graph for asynchronous reductions.

Takes an input collection and reduces its content with a given operation and selection mechanism. The computation is done while new items arrive. Not all items need to be available to start or make progress. Data input is provided by normal puts into the input collection. The final reduced value for a reduction is put into the output collection.

Supports multiple concurrent reductions (with the same operation) identified by a reduction id. For this, a selector functor can be provided to tell which data-item goes to which reduction (maps a data-tag to a reduction-id).

The number reduced items per reduction-id needs to be provided through a second input collection. You can signal no more incoming values by putting a count < 0. Providing counts late reduces communication and potentially improves performance.

Each reduction is independent of other reductions and can finish independently while others are still processing. Connected graphs can get the reduced values with a normal get-calls (using the desired reduction-id as the tag).

The implementation is virtually lock-free. On distributed memory the additional communication is also largely asynchronous.

See also Re-using CnC graphs (reductions, cross/join...)

ctxtthe context to which the graph belongs
namethe name of this reduction graph instance
ininput collection, every item that's put here is applied to sel and potentially takes part in a reduction
cntinput collection; number of items for each reduction expected to be put here (tag is reduction-id, value is count)
outoutput collection, reduced results are put here with tags as returned by sel
opthe reduction operation:
IType (*)(const IType&, const IType&) const
usually a functor
idtythe identity/neutral element for the given operation
selfunctor, called once for every item put into "in":
bool (*)( const ITag & itag, OTag & otag ) const
must return true if given element should be used for a reduction, otherwise false;
if true, it must set otag to the tag of the reduction it participates in

Definition at line 95 of file reduce.h.

        return new reduction< ITag, IType, ITuner, CType, CTuner, OTag, OTuner, ReduceOp, Select >( ctxt, name, in, cnt, out, op, idty, sel );
 All Classes Namespaces Functions Variables Typedefs Enumerator Friends