MoleQueue JSON-RPC Specification

JSON-RPC specification

This is the proposed communication protocol, it is neither final nor implemented!

The JSON-RPC 2.0 specification is here. In a nutshell, the protocol consists of three message types: requests, responses, and notifications. Every request must be met with a reply, and a notification is simply a request object without the id member.

Requests are JSON objects with three name/value pairs:

• method (String): The method requested.
• params (Array or Object): A parameters for the requested method.
• id (Value): A unique identifier.

Each request is met with a reply of the following format:

• result (Object, Array, or Value): The result of the request. Must not exist if there was an error.
• error (Error Object): A description of the error. Must not exist if no error occurred.
• id (Value): Must be the same as the identifier used in the request.

The Error object must contain an integer error code ("code") and a descriptive error string ("message"). An optional "data" object may provide more detailed information.

As mentioned above, notifications follow the same format as requests, but will omit the id name/value pair.

Implementation

Click for details about the MoleQueue JSON-RPC Implementation.

Specifying files

Files may be represented by either referring to a location on a local filesystem or providing a file's contents. The {FileSpec} object in the following sample messages refers to either of the following forms:

Filesystem path:

{
  "filepath" : "/path/to/file"
}

or filename and contents:

{
  "filename" : "filename.extension", // no path
  "contents" : "string containing file contents\n" 
}

Supported Messages

The following sections detail the requests, responses, and notifications that MoleQueue recognizes:

List of available Queues/Programs

To obtain a list of the available Queues and the programs supported by them, submit a request for the method "listQueues" with no parameters. MoleQueue will reply with an object containing a dictionary with user-set names identifying each queue, along with an array of the programs supported by each queue.

Client request:

{
  "jsonrpc": "2.0",
  "method": "listQueues",
  "id": "XXX"
}

Server response:

{
    "jsonrpc": "2.0",
    "result": {
        "Big cluster SGE": [
            "GAMESS",
            "MOPAC",
            "Gaussian",
            "NWChem"
        ],
        "Remote cluster PBS": [
            "GAMESS",
            "MOPAC",
            "Gaussian",
            "NWChem"
        ],
        "Local": [
            "GAMESS",
            "MOPAC",
            "Gaussian",
            "NWChem"
        ]
    },
    "id": "XXX"
}

Request new job submission

Client request:

{
  "jsonrpc": "2.0",
  "method": "submitJob",
  "params": {
    "queue": "Remote cluster PBS",
    "program": "MOPAC",
    "description": "PM6 H2 optimization",
    "inputFile": {FileSpec}
  },
  "id": "XXX"
}

Additional (optional) parameters may be used to tweak job behavior further. Default values are provided in parenthesis:

• additionalInputFiles: ([]) An array of additional {FileSpec} objects indicating supplemental files needed for the job to run. These are placed in the working directories of the job, but not explicitly called by the executable.
• cleanRemoteFiles: (false) If true, any remote working directories are deleted.
• retrieveOutput: (true) If true, output files are copied back into the local working directory.
• outputDirectory: (null) If not null, this must be a string specifying a path to place completed output files, instead of the local working directory.
• cleanLocalWorkingDirectory: (false) If true, the local temporary working directory will be removed on successful job completion.
• hideFromGui: (false) If true, the job will not be displayed in the MoleQueue GUI by default (useful for automated batch jobs).
• popupOnStateChange: (true) If true, a notification popup will be displayed system tray when the job's state changes.
• maxWallTime: (-1) The maximum time limit for a job to run in minutes. If less than or equal to zero, a queue-specific walltime is used instead. This option only has effect on remote queuing systems.
• numberOfCores: (1) The number of processor cores to request for this job.
• keywords: ({}) A dictionary of string pairs used to populate the launch script template. Dictionary keys identify keywords (enclosed in "$$<it>[keyword]</it>$$") in the launch script template to replace with the associated value. For example, if this member is

  "keywords" : {
    "postProcessStep" : "mkdir output\nmv *.out *.dat output\ntar -xzvf output.tgz output && rm -rf output"
  }

and the launch script template is

#!/bin/bash

# setup, run program...

$$postProcessStep$$

then the job-specific generated launch script will be generated as

#!/bin/bash

# setup, run program...

mkdir output
mv *.out *.dat output
tar -xzvf output.tgz output && rm -rf output

This option can be used to add custom job-specific functionality for sufficiently advanced clients. Note that some keywords are hardcoded into MoleQueue; these are documented in the "Template Help" dialog of the MoleQueue server GUI. Any unrecognized keywords in the launcher template will be removed.

Server response:

• If successful, some identifiers are returned:

{
  "jsonrpc": "2.0",
  "result": {
    "moleQueueId": 17,
    "workingDirectory": "/tmp/MoleQueue/17/"
  },
  "id": "XXX"
}

moleQueueId is a identifier that is unique to this instance of MoleQueue.

• If unsuccessful, an error is returned:

{
  "jsonrpc": "2.0",
  "error": {
    "code": 32,
    "message": "Cannot write to local directory \"/tmp/MoleQueue/17/\". Permission denied."
  },
  "id": "XXX"
}

Cancel submitted job

Client request:

Parameter is the MoleQueue id of the job to cancel:

{
  "jsonrpc": "2.0",
  "method": "cancelJob",
  "params": {"moleQueueId" : 17},
  "id": "XXX"
}

Server response:

Result is the MoleQueue id of the cancel job:

{
  "jsonrpc": "2.0",
  "result": 17,
  "id": "XXX"
}

Notification of job state change

When a job changes state, MoleQueue will send a notification to all active connections of the following form:

{
  "jsonrpc": "2.0",
  "method": "jobStateChanged",
  "params": {
    "moleQueueId": 17,
    "oldState": "RemoteQueued",
    "newState": "Running"
  }
}

Valid states are:

• None: Initial state of job, should never be entered.
• Accepted: Job has been received and is being prepared (Writing input files, etc).
• LocalQueued: Job is being queued locally, either waiting for local execution or remote submission.
• Submitted: Job has been submitted to a remote queuing system.
• RemoteQueued: Job is pending execution on a remote queuing system.
• RunningLocal: Job is running locally.
• RunningRemote: Job is running remotely.
• Finished: Job has completed.
• Killed: Job has been canceled by the user.
• Error: The job has encountered an error.
• Unknown: The job's status is not known or invalid.

Request Job Information

A client can request information about a Job by submitting a lookupJob request and providing a MoleQueue id:

{
  "jsonrpc": "2.0",
  "method": "lookupJob",
  "params": {"moleQueueId" : 17},
  "id": "XXX"
}

The server will respond with an object describing the Job's state using a similar JSON object as the submission request:

{
  "jsonrpc" : "2.0",
  "result" : {
     "cleanLocalWorkingDirectory" : false,
     "cleanRemoteFiles" : false,
     "description" : "spud slicer 28",
     "hideFromGui" : false,
     "inputFile" : {FileSpec},
     "additionalInputFiles" : [ {FileSpec}, {FileSpec}, ... ],
     "jobState" : 0,
     "localWorkingDirectory" : "/some/local/path",
     "maxWallTime" : 1440,
     "moleQueueId" : 17,
     "numberOfCores" : 4,
     "outputDirectory" : "",
     "popupOnStateChange" : true,
     "program" : "Quantum Tater",
     "queue" : "Some big ol' cluster",
     "queueId" : 7366,
     "retrieveOutput" : true
  },
  "id": "XXX"
}

If the lookup fails (i.e. the request MoleQueue id is not known to the server), a simple error reply is sent with the requested MoleQueue id as the error.data value:

{
  "error" : {
    "data" : 17,
    "code" : 0,
    "message" : "Unknown MoleQueue ID"
  },
  "jsonrpc" : "2.0",
  "id" : "XXX"
}