|
Kyoto Tycoon
|
#include <ktdbext.h>
Classes | |
| class | FlushThread |
| Cache flusher. | |
| class | MapChecker |
| Checker for the map process. | |
| class | MapVisitor |
| Visitor for the map process. | |
| struct | MergeLine |
| Front line of a merging list. | |
| class | ReduceTaskQueue |
| Task queue for parallel reducer. | |
| class | ValueIterator |
| Value iterator for the reducer. More... | |
Public Types | |
| enum | Option { XNOLOCK = 1 << 0, XPARAMAP = 1 << 1, XPARARED = 1 << 2, XPARAFLS = 1 << 3, XNOCOMP = 1 << 8 } |
| Execution options. More... | |
Public Member Functions | |
| MapReduce () | |
| Default constructor. | |
| virtual | ~MapReduce () |
| Destructor. | |
| virtual bool | map (const char *kbuf, size_t ksiz, const char *vbuf, size_t vsiz)=0 |
| Map a record data. | |
| virtual bool | reduce (const char *kbuf, size_t ksiz, ValueIterator *iter)=0 |
| Reduce a record data. | |
| virtual bool | preprocess () |
| Preprocess the map operations. | |
| virtual bool | midprocess () |
| Mediate between the map and the reduce phases. | |
| virtual bool | postprocess () |
| Postprocess the reduce operations. | |
| virtual bool | log (const char *name, const char *message) |
| Process a log message. | |
| bool | execute (TimedDB *db, const std::string &tmppath="", uint32_t opts=0) |
| Execute the MapReduce process about a database. | |
| void | tune_storage (int32_t dbnum, int64_t clim, int64_t cbnum) |
| Set the storage configurations. | |
| void | tune_thread (int32_t mapthnum, int32_t redthnum, int32_t flsthnum) |
| Set the thread configurations. | |
Protected Member Functions | |
| bool | emit (const char *kbuf, size_t ksiz, const char *vbuf, size_t vsiz) |
| Emit a record from the mapper. | |
MapReduce framework.
| kyototycoon::MapReduce::MapReduce | ( | ) | [explicit] |
Default constructor.
| virtual kyototycoon::MapReduce::~MapReduce | ( | ) | [virtual] |
Destructor.
| virtual bool kyototycoon::MapReduce::map | ( | const char * | kbuf, |
| size_t | ksiz, | ||
| const char * | vbuf, | ||
| size_t | vsiz | ||
| ) | [pure virtual] |
Map a record data.
| kbuf | the pointer to the key region. |
| ksiz | the size of the key region. |
| vbuf | the pointer to the value region. |
| vsiz | the size of the value region. |
| virtual bool kyototycoon::MapReduce::reduce | ( | const char * | kbuf, |
| size_t | ksiz, | ||
| ValueIterator * | iter | ||
| ) | [pure virtual] |
Reduce a record data.
| kbuf | the pointer to the key region. |
| ksiz | the size of the key region. |
| iter | the iterator to get the values. |
| virtual bool kyototycoon::MapReduce::preprocess | ( | ) | [virtual] |
Preprocess the map operations.
| virtual bool kyototycoon::MapReduce::midprocess | ( | ) | [virtual] |
Mediate between the map and the reduce phases.
| virtual bool kyototycoon::MapReduce::postprocess | ( | ) | [virtual] |
Postprocess the reduce operations.
| virtual bool kyototycoon::MapReduce::log | ( | const char * | name, |
| const char * | message | ||
| ) | [virtual] |
Process a log message.
| name | the name of the event. |
| message | a supplement message. |
| bool kyototycoon::MapReduce::execute | ( | TimedDB * | db, |
| const std::string & | tmppath = "", |
||
| uint32_t | opts = 0 |
||
| ) |
Execute the MapReduce process about a database.
| db | the source database. |
| tmppath | the path of a directory for the temporary data storage. If it is an empty string, temporary data are handled on memory. |
| opts | the optional features by bitwise-or: MapReduce::XNOLOCK to avoid locking against update operations by other threads, MapReduce::XNOCOMP to avoid compression of temporary databases. |
| void kyototycoon::MapReduce::tune_storage | ( | int32_t | dbnum, |
| int64_t | clim, | ||
| int64_t | cbnum | ||
| ) |
Set the storage configurations.
| dbnum | the number of temporary databases. |
| clim | the limit size of the internal cache. |
| cbnum | the bucket number of the internal cache. |
| void kyototycoon::MapReduce::tune_thread | ( | int32_t | mapthnum, |
| int32_t | redthnum, | ||
| int32_t | flsthnum | ||
| ) |
Set the thread configurations.
| mapthnum | the number of threads for the mapper. |
| redthnum | the number of threads for the reducer. |
| flsthnum | the number of threads for the internal flusher. |
| bool kyototycoon::MapReduce::emit | ( | const char * | kbuf, |
| size_t | ksiz, | ||
| const char * | vbuf, | ||
| size_t | vsiz | ||
| ) | [protected] |
Emit a record from the mapper.
| kbuf | the pointer to the key region. |
| ksiz | the size of the key region. |
| vbuf | the pointer to the value region. |
| vsiz | the size of the value region. |
1.7.6.1