Setup YCSB & MapKeeper to benchmark LevelDB

LevelDB gets its own benchmark binary called “db_bench” under the directory “out-shared” after make. But there are other tools could benchmark LevelDB like MapKeeper. YCSB is a Yahoo! cloud service benchmark without the support for LevelDB. It is a possibility to constitute YCSB into MapKeeper to benchmark LevelDB. Here are my so far steps on doing so with an RHEL 7 on AWS.

  1. Install Java & Maven
    1. sudo yum install java-1.7.0-openjdk-devel
       wget http://www-eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
       sudo tar xzf apache-maven-3.3.9-bin.tar.gz & cd /opt
       sudo ln -s apache-maven-3.3.9 maven
       sudo vi /etc/profile.d/maven.sh -- to setup environment variable
       export M2_HOME=/opt/maven
       export PATH=${M2_HOME}/bin:${PATH}
  2. source /etc/profile.d/maven.sh
  3. mvn -version -- to make sure maven is installed with the right version
  • Install Libevent
    1. wget http://monkey.org/~provos/libevent-2.0.12-stable.tar.gz
      tar xfvz libevent-2.0.12-stable.tar.gz
      cd libevent-2.0.12-stable
      ./configure --prefix=/usr/local
      make && sudo make install
  • Install Boost
    1. Boost 1.48.0
      1. wget http://superb-sea2.dl.sourceforge.net/project/boost/boost/1.48.0/boost_1_48_0.tar.gz
        tar xfvz boost_1_48_0.tar.gz
        cd boost_1_48_0
        ./bootstrap.sh --prefix=/usr/local
        sudo ./b2 install 
    2. Boost 1.53
      1. sudo yum install boost-devel
  • Install Thrift
    1. Steps
      1. wget http://archive.apache.org/dist/thrift/0.8.0/thrift-0.8.0.tar.gz
        tar xfvz thrift-0.8.0.tar.gz
        cd thrift-0.8.0
        ./configure --prefix=/usr/local
    2. Make sure that
      1. Building C++ Library ......... : yes
        Building TNonblockingServer .. : yes
      2. make && sudo make install
        Bug report:
        1. Include patch: https://issues.apache.org/jira/browse/THRIFT-2367
        2. Modified /usr/local/include/thrift/protocol/TProtocol.h and
          /usr/local/include/thrift/TApplicationException.h to include <stdint.h>
        3. Include patch: https://svn.boost.org/trac/boost/attachment/ticket/6165/libstdcpp3.hpp.patch
  • Install YCSB in MapKeeper
    1. cd $MKROOT/ycsb
      git clone git://github.com/brianfrankcooper/YCSB.git
      cd YCSB
      mvn clean package
    2. ./bin/ycsb load mapkeeper -P ./workloads/workloada -- to see if it work
  • Install LevelDB on CentOS 7/RHEL 7 p.s. Libs work a little different on CentOS than it on Ubuntu or macOS
    git clone https://github.com/google/leveldb.git
     cd leveldb && make
     sudo cp include/leveldb /usr/local/include -rf
     sudo cp out-shared/libleveldb.* /usr/local/lib
     * sudo vi /etc/ld.so.conf.d/usrlocallib.conf -- let bash know you lib are here by adding the following lines
     /usr/local/lib
     sudo /sbin/ldconfig –v -- flush the settings
     ./db_bench to see if LevelDB works
  • Run the benchmark in MapKeeper
    1. cd mapkeeper/leveldb && make
      1. possible bug 1: libstdcpp3.hpp
        1. solution: add this patch into the .hpp file: https://svn.boost.org/trac/boost/attachment/ticket/6165/libstdcpp3.hpp.patch
      2. possible bug 2: TIME_UTC is not correct
        1. solution: change all TIME_UTC into TIME_UTC_ in that .xtime file
      3. possible bug 3: lboost-thread not found
        1. solution: install a boost with version higher than 1.48.0
    2. ./bin/ycsb load mapkeeper -P ./workloads/workloada
      1. I haven't done it yet without bugs so hope anyone could help? :)

Good Enough? Maybe