MongoChemWeb

The following steps are required for the deployment of MongoChemWeb:

Building VTK with web support

VTK needs to be configured with the Web group enabled, and Python wrapping turned on.

$ git clone git://vtk.org/VTK.git VTK
$ mkdir VTK-build
$ cd VTK-build
$ cmake -DVTK_Group_Web:BOOL=ON -DVTK_WRAP_PYTHON:BOOL=ON ../VTK 
$ make

Building with Mesa

For offscreen rendering you will need to build OSMesa. The Mesa 10.0.3 OSMesa Gallium llvmpipe state-tracker is the preferred Mesa back-end renderer for VTK. The following shows how to configure it with system installed LLVM. Our strategy is to configure Mesa with the minimal number of options needed for OSMesa. This greatly simplifies the build, as many of the other drivers/renderers depend on X11 or other libraries. The following set of options are from the Mesa v10.0.3 release. Older or newer releases may require slightly different options. Consult ./configure --help for the details.

$ autoreconf -fi
$ ./configure \
   CXXFLAGS="-O2 -g -DDEFAULT_SOFTWARE_DEPTH_BITS=31" \
   CFLAGS="-O2 -g -DDEFAULT_SOFTWARE_DEPTH_BITS=31" \
   --disable-xvmc \
   --disable-glx \
   --disable-dri \
   --with-dri-drivers="" \
   --with-gallium-drivers="swrast" \
   --enable-texture-float \
   --disable-shared-glapi \
   --disable-egl \
   --with-egl-platforms="" \
   --enable-gallium-osmesa \
   --enable-gallium-llvm=yes \
   --with-llvm-shared-libs \
   --prefix=/opt/mesa/10.0.3/llvmpipe

$ make -j5
$ make install

Some explanation of these options: DEFAULT_SOFTWARE_DEPTH_BITS=31 This sets the internal depth buffer precision for the OSMesa rendering context. In our experience, this is necessary to avoid z-buffer fighting during parallel rendering. Note that we have used this in-place of --with-osmesa-bits=32, which sets both depth buffer and color buffers to 32 bit precision. Because of a bug in Mesa, this introduces over 80 ctest regression failures in VTK related to line drawing. --enable-texture-float Floating point textures are disabled by default due to patent restrictions. They must be enabled for many advanced VTK algorithms. To use Mesa with VTK you will need to set the following options before when configuring VTK:

 $ cmake -DVTK_Group_Web:BOOL=ON -DVTK_WRAP_PYTHON:BOOL=ON \
   -DVTK_USE_X:BOOL=OFF -DVTK_OPENGL_HAS_OSMESA:BOOL=ON \
   -DOPENGL_gl_LIBRARY:FILEPATH="" -DOPENGL_glu_LIBRARY:FILEPATH="" \
   -DOPENGL_INCLUDE_DIR:PATH=/opt/mesa/10.0.3/llvmpipe/include \
   -DOSMESA_INCLUDE_DIR:PATH=/opt/mesa/10.0.3/llvmpipe/include \
   -DOSMESA_LIBRARY:FILEPATH=/opt/mesa/10.0.3/llvmpipe/lib/libOSMesa.so \
   -DCMAKE_BUILD_TYPE:STRING=Release ../VTK

Building OpenChemistry

Clone the OpenChemistry repository containg the CJSON to CML conversion executable.

$ git clone --recursive https://github.com/OpenChemistry/openchemistry.git openchemistry

Now follow instruction to build the OpenChemistry project starting at the Build step.

Apache compile, install and configuration

Apache 2.4.7 has support for rewriting websocket URLs so can be used to proxy VTK web connections, this is the easiest way to configure Apache. However, if you are using a older version of Apache you can use the custom proxy code based on mod_python.

Using Apache 2.4.7

Build / Installation

Unless Apache 2.4.7 is packaged your distribution you will need to compile from source.

Obtain the necessary source tarballs. You will need httpd source, as well as apr and apr-util sources.

$ wget http://mirrors.sonic.net/apache/httpd/httpd-2.4.7.tar.gz -O httpd.tgz
$ wget http://apache.petsads.us/apr/apr-1.5.0.tar.gz -O apr.tgz
$ wget http://apache.petsads.us/apr/apr-util-1.5.3.tar.gz -O apr-util.tgz

Now unpack everything in the right places.

$ mkdir apache-2.4.7-src
$ cd apache-2.4.7-src
$ tar zxvf ../httpd.tgz
$ cd httpd-2.4.7/srclib
$ tar zxvf ../../../apr.tgz
$ mv apr-1.5.0 apr
$ tar zxvf ../../../apr-util.tgz
$ mv apr-util-1.5.3 apr-util

Now configure and build.

$ cd apache-2.4.7-src/httpd-2.4.7
$ ./configure --with-included-apr --enable-proxy
$ make
$ sudo make install

Configuration

Add a virtual host to the httpd-vhosts.conf file, which will be located in /usr/local/apache2/conf/extra/.

<VirtualHost *:80>
    # The hostname need to be updated
    ServerName hostname
    ServerAdmin webmaster@example-host.example.com
    ErrorLog "logs/mongochemweb-error_log"
    CustomLog "logs/mongochemweb-access_log" common
    # This rule is used to pass session requests to the launcher
    ProxyPass /session http://localhost:9000/session
    # Proxy rest of traffic to tangelo, note the ~mongochem may need to be updated
    # if tangelo is being run as a different user.
    ProxyPass / 
    # Turn on the rewrite engine
    RewriteEngine On
    # This is the path the mapping file the launcher will create
    RewriteMap session-to-port txt:/tmp/mapping.txt
    # This is the rewrite condition. Look for anything with a sessionId= in the query part of the  URL and capture the value to use below.
    RewriteCond %{QUERY_STRING}     ^sessionId=(.*)$ [NC]
    # This does the rewrite using the mapping file and the sessionId
    RewriteRule    ^/proxy.*$  ws://${session-to-port:%1}/ws  [P]
</VirtualHost>

Be sure to update the ProxyPass paths statements to match your configuration. For example if tangelo is going to to be run as user bob. Then / should be forwarded to http://localhost:8080/~bob/mongochemweb/

Include this virtual host in the main httpd configuration file. Find the following line in /usr/local/apache2/conf/httpd.conf and uncomment it.

Include conf/extra/httpd-vhosts.conf

Find the following lines in the httpd.conf file and uncomment them in order to load some necessary modules.

LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
LoadModule rewrite_module modules/mod_rewrite.so

We need to ensure the mapping file exists before starting Apache. Issue the following command to this.

$ touch /tmp/mapping.txt

Start the httpd daemon.

$ sudo /usr/local/apache2/bin/apachectl -k start

Using Apache 2.2.X

Build / Installation

Old versions of Apache are likely to packaged for your distribution for example Apache 2.2 can be install on Ubuntu LTS using the following command.

$ sudo apt-get install apache2

Once Apache is installed the mod_python Apache module needs to be installed ( this can be done on Ubuntu) by issuing the following command.

$ sudo apt-get install libapache2-mod-python

Ensure that the correct link was create to load the module.

$ ls -l /etc/apache2/mods-enabled/python.load

This should give:

$ lrwxrwxrwx 1 root root 29 Sep  4 14:50 /etc/apache2/mods-enabled/python.load -> ../mods-available/python.load

Autobahn is needed by the proxy to make websocket connections to the backend. It can be install using pip.

$ sudo pip install autobahn

The pywebsocket module is required to listen for incoming websocket. It is installed by the following.

$ wget http://pywebsocket.googlecode.com/files/mod_pywebsocket-0.7.8.tar.gz
$ tar xvf mod_pywebsocket-0.7.8.tar.gz
$ cd mod_pywebsocket-0.7.8/src/
$ python setup.py build
$ sudo python setup.py install

Websocket proxy configuration

The websocket proxy code now needs to be downloaded and configured.

$ sudo mkdir /var/websocket_proxy
$ cd /tmp
$ wget http://pvw.kitware.com/guides/apache_setup/data/ApacheWebsocketProxy.tgz
$ cd /var/websocket_proxy
$ tar xvf /tmp/ApacheWebsocketProxy.tgz

Update proxy.json with the following values

"loggingConfiguration": "/var/websocket_proxy/logging.json"
"sessionMappingFile": "/tmp/session.map"

The proxy configuration needs to be in home directory of the user who runs Apache. This is usually /var/www.

$ sudo mv proxy.json /var/www

The following lines need to be added to /etc/apache2/apache2.conf to configure the mod_python module

PythonPath "sys.path+['/usr/local/lib/python2.7/dist-packages/mod_pywebsocket']"
PythonOption mod_pywebsocket.handler_root /var/websocket_proxy
PythonHeaderParserHandler mod_pywebsocket.headerparserhandler
PythonInterpreter main_interpreter

You will also need to create a site configuration file in /etc/apache2/sites-available/, so create a file called mongochemweb contain the following configuration:

<VirtualHost *:80>
        # Replace with real hostname
        ServerName hostname
        DocumentRoot /var/websocket_proxy
        # Jetty session manager
        ProxyPass /session http://localhost:9000/sessionmgr/paraview ttl=500
        # Tangelo, you may need to substitute the user name you are using if its not mongochem 
        ProxyPass /  http://localhost:8080/~mongochem/mongochemweb/
        <Directory /var/websocket_proxy>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
                AddHandler mod_python .py
                PythonHandler mod_python.publisher
                PythonDebug On
        </Directory>
        ErrorLog        logs/mongochemweb-error.log
        CustomLog       logs/mongochemweb-access.log 
</VirtualHost>

You then need to create symlink in the enabled sites directory

$ /etc/apache2/sites-enabled
$ sudo ln -s ../sites-available/mongochemweb .

Configuring a process launcher

A launcher is used to start python processes that provide the visualization pipeline for a MongoChemWeb session. If you are using Apache 2.4.7 you should use the Python launcher that comes with VTK, otherwise you will need to use the Jetty based session manager that will do the same job.

Using the VTK web launcher

The launcher can be found in the VTK source tree:

<VTK source directory>/Web/Python/launcher.py

The launcher takes a configuration file specifies the parameters to use when launching the vtkpython processes.

Create a file launcher.conf containing the following JSON specification:

{

      "configuration": {
        "host" : "localhost",
        "port" : 9000,
        "endpoint": "session",
        "proxy_file" : "/tmp/mapping.txt",
        "sessionURL" : "ws://${host}/proxy?sessionId=${id}",
        "timeout" : 5,
        "log_dir" : "/tmp",
        "fields" : ["file", "host", "port", "updir"]
      },

      "resources" : [ { "host" : "localhost", "port_range" : [9001, 9999] } ],


      "properties" : {
        "mongochemweb_dir": "/home/cjh/tangelo_html/mongochemweb",
        "python_exec" : "/home/cjh/work/build/VTK/bin/vtkpython"
      },


      "apps" : {
        "mol" : {
          "cmd" : [
            "${python_exec}", "${mongochemweb_dir}/service/vtk_web_molecule.py", "--port", "$port"
              ],
          "ready_line" : "Starting factory"
        }
      }
}

To start the launcher issue the following command:

$ python <VTK source directory>/Web/Python/launcher.py launcher.conf

Go to testing launcher to check that things are configured correctly.

Using the Jetty Session Manager

If you are using Apache 2.2.X you will need to use the Jetty session manager. You will need to install Java (on Ubuntu) this can be done using the following command:

$ sudo apt-get install openjdk-7-jre

How you need to download and configure the session manager.

$ wget http://paraview.org/files/dependencies/ParaViewWeb/JettySessionManager-Server-1.0.jar
Configuration
In order to customize and configure the Jetty session manager web server, you will need to create a configuration file, named pw-config.properties, as follows:
pw.web.port=9000
# Process logs
pw.logging.dir=/tmp
# ==================================================
pw.mol.cmd=<VTK build dir>/bin/vtkpython /home/mongochem/tangelo_html/mongochemweb/service/vtk_web_molecule.py --port PORT
pw.mol.cmd.run.dir=/home/mongochem/tangelo_html/mongochemweb/service
pw.mol.cmd.map=PORT:getPort

# Resources informations
pw.resources=localhost:9001-9999

# Factory
pw.factory.proxy.adapter=com.kitware.paraviewweb.external.JsonFileProxyConnectionAdapter
pw.factory.session.url.generator=com.kitware.paraviewweb.external.GenericSessionURLGenerator
pw.factory.resource.manager=com.kitware.paraviewweb.external.SimpleResourceManager
pw.factory.visualization.launcher=com.kitware.paraviewweb.external.ProcessLauncher
pw.factory.websocket.proxy=com.kitware.paraviewweb.external.SimpleWebSocketProxyManager
pw.factory.session.manager=com.kitware.paraviewweb.external.MemorySessionManager

# External configurations
pw.factory.proxy.adapter.file=/tmp/session.map
pw.factory.session.url.generator.pattern=ws://hostname/proxy?sessionId=SESSION_ID

pw.process.launcher.wait.keyword=Starting factory
pw.process.launcher.wait.timeout=10000

pw.session.public.fields=id,sessionURL,name,description,sessionManagerURL,application,idleTimeout,startTime

You will need to update the hostname and path to the mongochemweb directory.

The session manager can now be started using the following command:wil

java -jar JettySessionManager-Server-1.0.jar /path_to_your_config_file/pw-config.properties

Testing your launcher configuration

Regardless of the launcher you are using you can use the following procedure to test that it is responsing to requests.

To test the launcher use curl issue the following request.

curl http://localhost:9000/session/123

The launcher should response with:

{"error": "No session with id: 123"}

Now check that the launcher requests are successfully proxies through Apache by issuing the same request to the external URL.

curl http://localhost/session/123

You should see the same result

Tangelo installation and configuration

Install tangelo by issuing the following command:

$ sudo pip install tangelo

Now start tangelo by issuing the following command:

$ tangelo start

MongoChemWeb app installation and configuration

The MongoChemWeb app requires two Python modules which can be installed as follows:

$ sudo pip install requests
$ sudo pip install pymongo

We now need to setup the application that will be run by tangelo. As the user who will used to run tangelo issue the following commands:

$ cd ~
$ mkdir tangelo_html
$ cd tangelo_html
$ git clone https://github.com/OpenChemistry/mongochemweb.git

The configuration on mongochemweb needs to be adjust to machine the installation. The configuration file an be found in mongochemweb/config/

chemical.json

{
  "server": "mongochem",
  "db": "cep",
  "heliumUrl": "http://mongochem:8088"
}

server - This is the hostname of the mongo database that MongoChemWeb will use to search for molecular data. The data needs to conform to this schema

db - This is the database on the server to use

heliumUrl - Is the URL that the Helium similarity search can be found (optional)

conversion.json

{
  "baseUrl": "http://localhost:8080/~cjh/mongochemweb/",
  "cjsonToCmlPath": "<OpenChemistry build dir>/avogadrolibs/bin/avocjsontocml"
}

baseUrl - This is the URL as which the MongoChemWeb app is be served by tangelo cjsonToCmlPath - This is the path to executable used to convert from CJSON to CML which we compiled earilier.

Now we need to create a symlink to ensure that the VTK web client can be found by tangelo.

$ cd ~/tangelo_html/mongochemweb/
$ ln -s <VTK build dir>/www vtk-web

Once the configuration is updated you should restart Tangelo

tangelo restart

You should now be able point your browser at http://localhost or http:://<yourhostname> and the application should load up.

Troubleshooting

There are three places to look for errors:

• The Apache logs /usr/local/apache2/logs or /var/log/apache2. This is the place to look for error related to proxying and rewrite rules.
• The Tangelo log /<user running tangelo home>/.config/tangelo/tangelo.log. This is the place to log for application related errors.
• When a vtkpython process is started by the launcher the stderr and stdout are redirected to a file that will appear in /tmp/<session id>.txt if you change the launcher configuration the directory may be different. This is the place to look for errors related to visualization pipeline.
• If no 3D view is display the first thing to do it check that the vtkpython processes are be launched

$ ps -ef | grep vtkpython

If not process is found check your launcher configuration and check /tmp for logs.