Skip to content

Implement general KV and RDFGraph

Zihang Yao requested to merge yaozh/wukong:refactor_gstore into master

Motivation

Currently, the implementation of GStore tightly binds the implementation of our RDMA-friendly KV.

Ideally, the GStore is implemented based on a general KVS, which should be configurable, as the size of Key (#bits), and only provides some low-level operations (read, write, etc). Thus, we could easily support multiple graphs, various types of Value, various indexes, and so on.

Solution

This merge request splits the RDF-related operations and KV-related operations into two classes: KVStore and RDFGraph.

A general KVStore based on C++ template is implemented.

Further, Static/Dynamic KVStore and Segment/Dynamic RDFGraph are split from the old Static/Dynamic GStore.

TODO

  • Gchecker is still implemented based on KV-store. My future plan is to write some unit tests to replace it.

Others

  • Refactor the Stats to collect statistics on RDF Graph but not on KV.
  • Refactor the RDFEngine to load triple data into RDF Graph.
  • Decouple the partition function (hash) and implement a macro: PARTITION(vid).
  • Refactor the meta-data(type-pred, edge-pred) part in graph loader and RDFGraph.
  • Config Doxygen and Sphinx docs.
  • Use clang-format and cpplint to format codes under src/core/store.

Evaluation

  • Tests on LUBM Q1-Q11, WSDTS 20 queries, YAGO Y1-Y4, DPSBD confirm correctness.
  • Also test DYNAMIC_GSTORE=ON & VERSATILE=ON & GPU=ON
Edited by Zihang Yao

Merge request reports