Wednesday, 24 February 2016

Installing and running Dell Dvd store (dvdstore ds2.1) benchmark

Dell DVD store simulates and online dvd store for ordering DVDs. I stumbled upon it while reading a VMware paper [Link]. As it was being used by VMware, I thought it would be a decent workload to do some memory benchmarks on a few linux containers. The instructions for download and usage can be found here. The gzip files available for download also have a lot of text files for explanation.

Following the instructions given on the site, as well as in the README files, I was not able to run the benchmark. Before delving into the problems, and the way I was able to solve them, let me first introduce you to the way the benchmark works.

Firstly, Dell DVD store wants us to set up the VMs we want to test. Then a machine separate from the one no which these VMs are located is required to be set up (The machine should be separate to limit  interference to the tests). This machine is also known as the driver, and is used to run the benchamarks on the VMs, and to collect statistics fro them. The driver sends a bunch of queries to the database running in the machines under test, and measures performance.
In my case I had set up two Linux containers, each running ubuntu (consider Linux container as a VM if you are not sure about what it means), and mysql as the database. The driver machine was a Windows 7 desktop (Although Linux too can be used, with mono (for .Net framework) installed, but some other steps that I am going to show, may not be as straightforward on Linux).

For setting up the machines under test, run "" in them. This scipt will generate data files which have to be loaded into a database. Now open a mysql prompt (I had mysql databse, however dvdstore supports many other as well), and type the folowing commands.

mysql --password='MySqlpasswordOfYourUser' < mysqlds2_create_db.sql
mysql --password='MySqlpasswordOfYourUser' < mysqlds2_create_ind.sql
mysql --password='MySqlpasswordOfYourUser' < mysqlds2_create_sp.sql 
Note: Before issuing these commands, you will have to edit these file and replace all occurrences the word TYPE by ENGINE. TYPE corresponds to an earlier version of syntax, which is no longer supported.

These will be present in build directory.

For setting up the driver machine run "" in it, and answer the prompts.

Finally the driver can be run with the following command.
./ds2mysqldriver.exe --config_file=../DriverConfig.txt
Where "DriverConfig.txt" is the file created by "" script.

Following is the list of problems I encountered. These arose mainly because the benchmark seems to be an old one, which is no longer updated.
  1. ./ds2_create_cust not found : No such file or directory
  2. ./ds2_create_inv not found : No such file or directory
  3. ./ds2_create_prod not found : No such file or directory
  4. Could not load file or assembly 'Mysql.Data' . The exact error message was:

     Could not load file or assembly 'MySql.Data, Version=, Culture=neutral, PublicKeyToken=c5687fc88969c44d' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
  5. Then there was issue of granting access to mysql database, so that the driver can connect to it and query. This was not a problem in the benchmark, only a configuration issue that I had to sort before the benchmark ran without any errors.
Disclaimer: The steps shown will allow you to run the benchmark, but I do not guarantee that the benchmark will perform as expected. In my case the benchmark has been running for almost 5 hours, I can comment once it successfully completes.

The first 3 errors are relatively straightforward to solve. The reason for them is that my containers were running 64 bit Ubuntu, and the binaries were compiled for 32-bit OS. I agree that the error reported is a bit misleading. So to make them run on a 64 bit OS, either they can be recompiled for 64 bit (code present along with the benchmark), or support for 32 bit binaries can be added to 64-bit Ubuntu. Latter is much easier, and we just have to do an "apt-get install".

sudo apt-get install libc6-i386
The 4th Problem wasn't that easy to solve, and required a fair bit of effort. I had to decompile the binary "ds2mysqldriver.exe", present in mysqlds2 folder. Here are the steps.

  • Download ILSpy from
  • Open ds2mysqldriver.exe in ILSpy, it will then decompile the executable.
  • Select on the folder, ds2mysqldriver, and then click on File->Save Code. It will look like the following.

  • Now you'll have a visual studio project for ds2mysqldriver.exe in the directory you saved ILSpy code.
  • Open the project in Visual Studio (Or any other IDE that supports C#).
  • Under the references of the project (under "solution explorer" tab), remove MySql.Data, and then add a new reference using "Add Reference ..." option. Select MySql.Data in the dialogue box that appears. This step will basically change the dependency of the code from an older version of the MySql.Data to a newer version. 
  • Save the solution.
  • Before building the project, you'll have to do some minor corrections in the code also. I got the following errors.

       a. Constant value '-1' cannot be converted to a 'ulong' (use 'unchecked' syntax to override) ds2mysqldriver - Use unchecked{} block around the erroneous code.
    b.  Cannot implicitly convert type 'int' to 'string' ds2mysqldriver - Do an explicit type conversion to string using "ToString()" method.
  • Now build the project, and use the new ds2mysqldriver.exe created, instead of the older one.
Fixing the 5th problem, that is allowing the driver program to access mysql databases running on the test machines.

  1. Open mysqld.cnf (present in directory /etc/mysql/mysql.conf.d/ ), and comment out the line bind-address = .
  2. Open a mysql prompt, and issue the following command.

Now the driver can be successfully by the following command.

./ds2mysqldriver.exe --config_file=../DriverConfig.txt