Skip to content

Async

Xiating Xie requested to merge xiext/wukong:async into master

Support async data fetch interface such that engine can call rpc to fetch data asynchronously and switch to another query.

Add a new abstraction: edge engine. It decides how to fetch data.

Add a new daemon thread: edge daemon, which handles rpc request from edge engine.

Current configuration works as:

  1. With RDMA, query engines fetch data using RDMA one-sided READ as origin.
  2. With TCP, query engines fetch data asynchronously through edge engine.

With TCP, throughput is 97K while origin fork-and-join method's throughput is 32K. (LUBM_2560, 4 machines, 1 daemon)

Evaluation shows that with TCP, increase of number of daemons does not improve performance.

Merge request reports