Instrument your experiment with OML

Introduction

OML is one of the supported experiment instrumentation tools in Fed4FIRE. In this section we will guide you through a simple example based on the Virtual Wall and w-iLab.t testbeds and the iPerf example run by OMF experiment control (Control your experiment with OMF).

OML makes it possible to send all measurement data of your experiment (running on multiple nodes, on multiple testbeds) to a single OML server.

As such OML consists of two main components:

  • OML client library: there have been developed libraries for C, Python and Ruby which can be added to your software and which contain all logic for sending measurement data to the OML server
  • OML server: the OML server component is responsible for collecting and storing measurements inside a database. Currently, SQLite3 and PostgreSQL are supported as database backends.

In this tutorial, we will show you, based on a tutorial from NICTA, how to create an OML Wrapper (ruby script) around an existing binary (iPerf) and run this on Fed4FIRE testbeds. This wrapper will parse the standard output of the iPerf program and send the fields of your choice to an OML server that we will set up.

Setting up the experiment

Follow Control your experiment with OMF to prepare everything.

Log in on the EC and do hostname.

As extra, log in on the server and do the following to install the wrapper and OML library:

cd /
sudo wget http://fed4fire-testbeds.test.iminds.be/OML/iperfwrap.rb
sudo chmod 755 iperfwrap.rb
sudo -s bash -c "source /etc/profile.d/rvm.sh && gem install oml4r"
sudo apt-get install ruby1.8
sudo pico -w /usr/local/rvm/gems/ruby-1.9.3-p286@omf/gems/oml4r-2.10.3/lib/oml4r.rb
Change require 'oml4r/version' into require '/usr/local/rvm/gems/ruby-1.9.3-p286@omf/gems/oml4r-2.10.3/lib/oml4r/version.rb'

Install the OML server

We will install the OML server on the EC node in our experiment:

sudo sh -c "echo 'deb http://download.opensuse.org/repositories/home:/cdwertmann:/oml/xUbuntu_12.04/ /' >> /etc/apt/sources.list.d/oml2.list"
wget http://download.opensuse.org/repositories/home:cdwertmann:oml/xUbuntu_12.04/Release.key
sudo apt-key add - < Release.key
sudo apt-get update
sudo apt-get install oml2

Check that the OML server is running correctly with:

ps aux|grep oml2-server
netstat -an|grep 3003

Running our experiment

On the EC node:

wget http://fed4fire-testbeds.test.iminds.be/OML/iperfoml.rb

Edit this script, and change the script as follows as the DNS names of the client and server depend on the experiment name you have chosen.

Fetch the DNS name of the client by running hostname in an SSH terminal on the client. This looks e.g. like client.omftes4.wall2-ilabt-iminds-be.wilab2.ilabt.iminds.be. Substitute the yyy in the iperf.rb script by this hostname.

Fetch the DNS name of the server by running hostname in an SSH terminal on the server. Substitute the xxx (2 times !) in the iperf.rb script by this hostname.

Now run the experiment script as follows on the EC:

sudo su
source /etc/profile.d/rvm.sh
omf_ec -u amqp://labwiki.test.atlantis.ugent.be exec --oml_uri tcp:`hostname`:3003  iperfoml.rb

This will now do the following (have a look in iperfoml.rb):

  • after 20 seconds the iperf server will be started (the iPerf server will now be run through the OML wrapper which will send the measurement info to the OML server)
  • after 5 more seconds, the iperf client will be started
  • after 15 more seconds, the experiment is stopped (iperf runs for 10 seconds)

You should see the output from iPerf in your terminal as shown below. The difference with the pure OMF run is that the output per second is not visible as it is now parsed by the OML wrapper.

_images/omlecoutput.PNG

Access the measurement data

In this example the OML server stores its data in a local SQLite backend. Every run will create a database in /var/lib/oml2/ on the EC. You can access your data as follows:

sqlite3 /var/lib/oml2/2014-02-23T14\:25\:50.777Z.sq3
.tables
select * from _senders;
select * from iperfwrap_iperfmp;
.quit

Be careful: if your experiment is swapped out, these directories will be lost, so you are responsible to copy this data to another location.

Note: if you change to TCP instead of UDP for the iPerf experiment, the output is changed and the wrapper has to be adapted.

Support for OML by the testbeds

Support of OML by testbeds requires the availability of a suitable OML library on the resources. The intention of Fed4FIRE is that all testbeds within the federation will support OML. However, today this is not yet the case. To know if a testbed that you intend to use supports OML, please check the documentation about that specific testbed at Testbeds.

Also please note that if your have root access to a node, that probably you can install the needed OML libraries yourself, in case your testbed does not yet provide them for you by default.

More information on OML

See OML.