MoleQueue JSON-RPC Specification

From wiki.openchemistry.org
Jump to navigation Jump to search

JSON-RPC specification

The JSON-RPC 2.0 specification is here. In a nutshell, the protocol consists of three message types: requests, responses, and notifications. Every request must be met with a reply, and a notification is simply a request object without the id member.

Requests are JSON objects with three name/value pairs:

  • method (String): The method requested.
  • params (Array or Object): A parameters for the requested method.
  • id (Value): A unique identifier.

Each request is met with a reply of the following format:

  • result (Object, Array, or Value): The result of the request. Must not exist if there was an error.
  • error (Error Object): A description of the error. Must not exist if no error occurred.
  • id (Value): Must be the same as the identifier used in the request.

The Error object must contain an integer error code ("code") and a descriptive error string ("message"). An optional "data" object may provide more detailed information.

As mentioned above, notifications follow the same format as requests, but will omit the id name/value pair.

Implementation

Click for details about the MoleQueue JSON-RPC Implementation.

Specifying files

Files may be represented by either referring to a location on a local filesystem or providing a file's contents. The {FileSpec} object in the following sample messages refers to either of the following forms:

Filesystem path: 

{
  "path" : "/path/to/file"
}

or filename and contents: 

{
  "filename" : "filename.extension", // no path
  "contents" : "string containing file contents\n" 
}

Supported Messages

The following sections detail the requests, responses, and notifications that MoleQueue recognizes:

List of available Queues/Programs

To obtain a list of the available Queues and the programs supported by them, submit a request for the method "listQueues" with no parameters. MoleQueue will reply with an object containing a dictionary with user-set names identifying each queue, along with an array of the programs supported by each queue.

Client request: 

{
  "jsonrpc": "2.0",
  "method": "listQueues",
  "id": "XXX"
}

Server response: 

{
    "jsonrpc": "2.0",
    "result": {
        "Big cluster SGE": [
            "GAMESS",
            "MOPAC",
            "Gaussian",
            "NWChem"
        ],
        "Remote cluster PBS": [
            "GAMESS",
            "MOPAC",
            "Gaussian",
            "NWChem"
        ],
        "Local": [
            "GAMESS",
            "MOPAC",
            "Gaussian",
            "NWChem"
        ]
    },
    "id": "XXX"
}

Request new job submission

Client request: 

{
  "jsonrpc": "2.0",
  "method": "submitJob",
  "params": {
    "queue": "Remote cluster PBS",
    "program": "MOPAC",
    "description": "PM6 H2 optimization",
    "inputFile": {FileSpec}
  },
  "id": "XXX"
}

Additional (optional) parameters may be used to tweak job behavior further. Default values are provided in parenthesis:

  • additionalInputFiles: ([]) An array of additional {FileSpec} objects indicating supplemental files needed for the job to run. These are placed in the working directories of the job, but not explicitly called by the executable.
  • cleanRemoteFiles: (false) If true, any remote working directories are deleted.
  • retrieveOutput: (true) If true, output files are copied back into the local working directory.
  • outputDirectory: (null) If not null, this must be a string specifying a path to place completed output files, instead of the local working directory.
  • cleanLocalWorkingDirectory: (false) If true, the local temporary working directory will be removed on successful job completion.
  • hideFromGui: (false) If true, the job will not be displayed in the MoleQueue GUI by default (useful for automated batch jobs).
  • popupOnStateChange: (true) If true, a notification popup will be displayed system tray when the job's state changes.
  • maxWallTime: (-1) The maximum time limit for a job to run in minutes. If less than or equal to zero, a queue-specific walltime is used instead. This option only has effect on remote queuing systems.
  • numberOfCores: (1) The number of processor cores to request for this job.
  • keywords: ({}) A dictionary of string pairs used to populate the launch script template. Dictionary keys identify keywords (enclosed in "$$[keyword]$$") in the launch script template to replace with the associated value. For example, if this member is
  "keywords" : {
    "postProcessStep" : "mkdir output\nmv *.out *.dat output\ntar -xzvf output.tgz output && rm -rf output"
  }

and the launch script template is 

#!/bin/bash

# setup, run program...

$$postProcessStep$$

then the job-specific generated launch script will be generated as 

#!/bin/bash

# setup, run program...

mkdir output
mv *.out *.dat output
tar -xzvf output.tgz output && rm -rf output

This option can be used to add custom job-specific functionality for sufficiently advanced clients. Note that some keywords are hardcoded into MoleQueue; these are documented in the "Template Help" dialog of the MoleQueue server GUI. Any unrecognized keywords in the launcher template will be removed.

Server response:

  • If successful, some identifiers are returned:
{
  "jsonrpc": "2.0",
  "result": {
    "moleQueueId": 17,
    "workingDirectory": "/tmp/MoleQueue/17/"
  },
  "id": "XXX"
}

moleQueueId is a identifier that is unique to this instance of MoleQueue.

  • If unsuccessful, an error is returned:
{
  "jsonrpc": "2.0",
  "error": {
    "code": 32,
    "message": "Cannot write to local directory \"/tmp/MoleQueue/17/\". Permission denied."
  },
  "id": "XXX"
}

Cancel submitted job

Client request:

Parameter is the MoleQueue id of the job to cancel: 

{
  "jsonrpc": "2.0",
  "method": "cancelJob",
  "params": {"moleQueueId" : 17},
  "id": "XXX"
}

Server response:

Result is the MoleQueue id of the canceled job: 

{
  "jsonrpc": "2.0",
  "result": {"moleQueueId" : 17},
  "id": "XXX"
}

If the job does not exist or some other error occurs, a JSON-RPC error response is sent with the requested MoleQueue id as the error.data value: 

{
  "error" : {
    "data" : {"moleQueueId" : 17},
    "code" : 0,
    "message" : "Unknown MoleQueue ID"
  },
  "jsonrpc" : "2.0",
  "id" : "XXX"
}

Notification of job state change

When a job changes state, MoleQueue will send a notification to all active connections of the following form: 

{
  "jsonrpc": "2.0",
  "method": "jobStateChanged",
  "params": {
    "moleQueueId": 17,
    "oldState": "QueuedRemote",
    "newState": "Running"
  }
}

Valid states are:

  • None: Initial state of job, should never be entered.
  • Accepted: Job has been received and is being prepared (Writing input files, etc).
  • QueuedLocal: Job is being queued locally, either waiting for local execution or remote submission.
  • Submitted: Job has been submitted to a remote queuing system.
  • QueuedRemote: Job is pending execution on a remote queuing system.
  • RunningLocal: Job is running locally.
  • RunningRemote: Job is running remotely.
  • Finished: Job has completed.
  • Killed: Job has been canceled by the user.
  • Error: The job has encountered an error.
  • Unknown: The job's status is not known or invalid.

Request Job Information

A client can request information about a Job by submitting a lookupJob request and providing a MoleQueue id: 

{
  "jsonrpc": "2.0",
  "method": "lookupJob",
  "params": {"moleQueueId" : 17},
  "id": "XXX"
}

The server will respond with an object describing the Job's state using a similar JSON object as the submission request: 

{
  "jsonrpc" : "2.0",
  "result" : {
     "cleanLocalWorkingDirectory" : false,
     "cleanRemoteFiles" : false,
     "description" : "spud slicer 28",
     "hideFromGui" : false,
     "inputFile" : {FileSpec},
     "additionalInputFiles" : [ {FileSpec}, {FileSpec}, ... ],
     "jobState" : "Accepted",
     "localWorkingDirectory" : "/some/local/path",
     "maxWallTime" : 1440,
     "moleQueueId" : 17,
     "numberOfCores" : 4,
     "outputDirectory" : "",
     "popupOnStateChange" : true,
     "program" : "Quantum Tater",
     "queue" : "Some big ol' cluster",
     "queueId" : 7366,
     "retrieveOutput" : true
  },
  "id": "XXX"
}

If the lookup fails (i.e. the request MoleQueue id is not known to the server), a simple error reply is sent with the requested MoleQueue id as the error.data value: 

{
  "error" : {
    "data" : {"moleQueueId" : 17},
    "code" : 0,
    "message" : "Unknown MoleQueue ID"
  },
  "jsonrpc" : "2.0",
  "id" : "XXX"
}

Configuring File Handlers

Right-clicking on a job entry in MoleQueue will list applications that are able to open files in the job's working/output directories. MoleQueue maintains a collection of file handlers that store an executable along with a set of filename patterns that the application is known to handle. New handlers can be added with the registerOpenWith method: 

{
    "jsonrpc": "2.0",
    "method": "registerOpenWith",
    "params": {
        "name": "My Spiffy Client",
        "method": { MethodSpec },
        "patterns": [
            {
                "regexp": "spiff[\\d]*\\.(?:dat|out)"
            },
            {
                "wildcard": "*.spiffyout",
                "caseSensitive": false
            }
        ]
    },
    "id": "777"
}

The parameters to this method are:

  • name: The name of the application. Must be a non-empty string.
  • method: Specification of how to open the file. See MethodSpec definition below.
  • executable: The name of / path to the executable. If the path is not specified, the executable should be in the user's path.
  • patterns (optional): An array of JSON objects that specify filename patterns that may be opened by the executable (see below). There is no limit on the number of patterns that may be specified for a single handler. If no pattern is specified, a single catch-all pattern ({"wildcard": "*"}) is used by default.

If the handler is successfully registered with MoleQueue, a JSON-RPC response is sent with the string "success" as the result. If a handler with the same name already exists, a JSON-RPC error response is sent with an error code of 1.

Method specification

The MethodSpec may either specify an executable or an RPC call. The executable method requires an executable that will be used to open files. If the executable is not in the user's path, the full path to the executable must be specified: 

{
    "jsonrpc": "2.0",
    "method": "registerOpenWith",
    "params": {
        "name": "My Spiffy Client",
        "method": { "executable": "/path/to/executable" },
        "patterns": [ ... ]
    },
    "id": "777"
}

To open files via an RPC call, the server and method must be specified as follows: 

{
    "jsonrpc": "2.0",
    "method": "registerOpenWith",
    "params": {
        "name": "My Spiffy Client",
        "method": {
            "rpcServer": "myLocalSocketName",
            "rpcMethod": "myOpenFileMethod"
        },
        "patterns": [ ... ]
    },
    "id": "777"
}

MoleQueue will use this information to send a message to a local socket of the form: 

{
    "jsonrpc": "2.0",
    "method": "myOpenFileMethod",
    "params": {
        "fileName": "<name of file>",
    },
    "id": "777"
}

The server must send a response to the request. The contents of the response are not important, MoleQueue will only check for a valid JSON-RPC 2.0 reponse. If the MoleQueue does not receive a response within 3 seconds, it will alert the user that the request was not handled.

Pattern specification

Each pattern in the patterns array must have a member named "regexp" or "wildcard". This member specifies a string pattern to match filenames against. "regexp" entries are regular expressions that follow the QRegExp RegExp2 format. The "wildcard" matching syntax is specified by the "WildCardUnix" QRegExp format documented on the same page. Both pattern types support an additional "caseSensitive" boolean option that can be used to toggle the case-sensitivity of the pattern. If not specified, case-sensitive matching is assumed.

Retrieved from "http://wiki.openchemistry.org/index.php?title=MoleQueue_JSON-RPC_Specification&oldid=312"