MoleQueue JSON-RPC Specification: Difference between revisions

From wiki.openchemistry.org
Jump to navigation Jump to search
Line 117: Line 117:
* <tt>maxWallTime</tt>: (<tt>-1</tt>) The maximum time limit for a job to run in minutes. If less than or equal to zero, a queue-specific walltime is used instead. This option only has effect on remote queuing systems.
* <tt>maxWallTime</tt>: (<tt>-1</tt>) The maximum time limit for a job to run in minutes. If less than or equal to zero, a queue-specific walltime is used instead. This option only has effect on remote queuing systems.
* <tt>numberOfCores</tt>: (<tt>1</tt>) The number of processor cores to request for this job.
* <tt>numberOfCores</tt>: (<tt>1</tt>) The number of processor cores to request for this job.
* <tt>keywords</tt>: (<tt>{}</tt>) A dictionary of string pairs used to populate the launch script template. Dictionary keys identify keywords (enclosed in "$$keyword$$") in the launch script template to replace with the associated value. For example, if this member is
* <tt>keywords</tt>: (<tt>{}</tt>) A dictionary of string pairs used to populate the launch script template. Dictionary keys identify keywords (enclosed in "$$<it>[keyword]</it>$$") in the launch script template to replace with the associated value. For example, if this member is
<source lang="JavaScript">
<source lang="JavaScript">
   "keywords" : {
   "keywords" : {
Line 123: Line 123:
   }
   }
</source>
</source>
Thus, if the launch script template is
and the launch script template is
<source lang="bash">
<source lang="bash">
#!/bin/bash
#!/bin/bash
Line 131: Line 131:
$$postProcessStep$$
$$postProcessStep$$
</source>
</source>
it will be generated as
then the job-specific generated launch script will be generated as
<source lang="bash">
<source lang="bash">
#!/bin/bash
#!/bin/bash

Revision as of 15:21, 22 August 2012

JSON-RPC specification

This is the proposed communication protocol, it is neither final nor implemented!

The JSON-RPC 2.0 specification is here. In a nutshell, the protocol consists of three message types: requests, responses, and notifications. Every request must be met with a reply, and a notification is simply a request object without the id member.

Requests are JSON objects with three name/value pairs:

  • method (String): The method requested.
  • params (Array or Object): A parameters for the requested method.
  • id (Value): A unique identifier.

Each request is met with a reply of the following format:

  • result (Object, Array, or Value): The result of the request. Must not exist if there was an error.
  • error (Error Object): A description of the error. Must not exist if no error occurred.
  • id (Value): Must be the same as the identifier used in the request.

The Error object must contain an integer error code ("code") and a descriptive error string ("message"). An optional "data" object may provide more detailed information.

As mentioned above, notifications follow the same format as requests, but will omit the id name/value pair.

Implementation

Click for details about the MoleQueue JSON-RPC Implementation.

Specifying files

Files may be represented by either referring to a location on a local filesystem or providing a file's contents. The {FileSpec} object in the following sample messages refers to either of the following forms:

Filesystem path:

{
  "filepath" : "/path/to/file"
}

or filename and contents:

{
  "filename" : "filename.extension", // no path
  "contents" : "string containing file contents\n" 
}

Supported Messages

The following sections detail the requests, responses, and notifications that MoleQueue recognizes:

List of available Queues/Programs

To obtain a list of the available Queues and the programs supported by them, submit a request for the method "listQueues" with no parameters. MoleQueue will reply with an object containing a dictionary with user-set names identifying each queue, along with an array of the programs supported by each queue.

Client request:

{
  "jsonrpc": "2.0",
  "method": "listQueues",
  "id": "XXX"
}

Server response:

{
    "jsonrpc": "2.0",
    "result": {
        "Big cluster SGE": [
            "GAMESS",
            "MOPAC",
            "Gaussian",
            "NWChem"
        ],
        "Remote cluster PBS": [
            "GAMESS",
            "MOPAC",
            "Gaussian",
            "NWChem"
        ],
        "Local": [
            "GAMESS",
            "MOPAC",
            "Gaussian",
            "NWChem"
        ]
    },
    "id": "XXX"
}

Request new job submission

Client request:

{
  "jsonrpc": "2.0",
  "method": "submitJob",
  "params": {
    "queue": "Remote cluster PBS",
    "program": "MOPAC",
    "description": "PM6 H2 optimization",
    "inputFile": {FileSpec}
  },
  "id": "XXX"
}

Additional (optional) parameters may be used to tweak job behavior further. Default values are provided in parenthesis:

  • additionalInputFiles: ([]) An array of additional {FileSpec} objects indicating supplemental files needed for the job to run. These are placed in the working directories of the job, but not explicitly called by the executable.
  • cleanRemoteFiles: (false) If true, any remote working directories are deleted.
  • retrieveOutput: (true) If true, output files are copied back into the local working directory.
  • outputDirectory: (null) If not null, this must be a string specifying a path to place completed output files, instead of the local working directory.
  • cleanLocalWorkingDirectory: (false) If true, the local temporary working directory will be removed on successful job completion.
  • hideFromGui: (false) If true, the job will not be displayed in the MoleQueue GUI by default (useful for automated batch jobs).
  • popupOnStateChange: (true) If true, a notification popup will be displayed system tray when the job's state changes.
  • maxWallTime: (-1) The maximum time limit for a job to run in minutes. If less than or equal to zero, a queue-specific walltime is used instead. This option only has effect on remote queuing systems.
  • numberOfCores: (1) The number of processor cores to request for this job.
  • keywords: ({}) A dictionary of string pairs used to populate the launch script template. Dictionary keys identify keywords (enclosed in "$$<it>[keyword]</it>$$") in the launch script template to replace with the associated value. For example, if this member is
  "keywords" : {
    "postProcessStep" : "mkdir output\nmv *.out *.dat output\ntar -xzvf output.tgz output && rm -rf output"
  }

and the launch script template is

#!/bin/bash

# setup, run program...

$$postProcessStep$$

then the job-specific generated launch script will be generated as

#!/bin/bash

# setup, run program...

mkdir output
mv *.out *.dat output
tar -xzvf output.tgz output && rm -rf output

This option can be used to add custom job-specific functionality for sufficiently advanced clients. Note that some keywords are hardcoded into MoleQueue; these are documented in the "Template Help" dialog of the MoleQueue server GUI. Any unrecognized keywords in the launcher template will be removed.

Server response:

  • If successful, some identifiers are returned:
{
  "jsonrpc": "2.0",
  "result": {
    "moleQueueId": 17,
    "workingDirectory": "/tmp/MoleQueue/17/"
  },
  "id": "XXX"
}

moleQueueId is a identifier that is unique to this instance of MoleQueue.

  • If unsuccessful, an error is returned:
{
  "jsonrpc": "2.0",
  "error": {
    "code": 32,
    "message": "Cannot write to local directory \"/tmp/MoleQueue/17/\". Permission denied."
  },
  "id": "XXX"
}

Cancel submitted job

Client request:

Parameter is the MoleQueue id of the job to cancel:

{
  "jsonrpc": "2.0",
  "method": "cancelJob",
  "params": {"moleQueueId" : 17},
  "id": "XXX"
}

Server response:

Result is the MoleQueue id of the cancel job:

{
  "jsonrpc": "2.0",
  "result": 17,
  "id": "XXX"
}

Notification of job state change

When a job changes state, MoleQueue will send a notification to all active connections of the following form:

{
  "jsonrpc": "2.0",
  "method": "jobStateChanged",
  "params": {
    "moleQueueId": 17,
    "oldState": "RemoteQueued",
    "newState": "Running"
  }
}

Valid states are:

  • None: Initial state of job, should never be entered.
  • Accepted: Job has been received and is being prepared (Writing input files, etc).
  • LocalQueued: Job is being queued locally, either waiting for local execution or remote submission.
  • Submitted: Job has been submitted to a remote queuing system.
  • RemoteQueued: Job is pending execution on a remote queuing system.
  • RunningLocal: Job is running locally.
  • RunningRemote: Job is running remotely.
  • Finished: Job has completed.
  • Killed: Job has been canceled by the user.
  • Error: The job has encountered an error.
  • Unknown: The job's status is not known or invalid.

Request Job Information

A client can request information about a Job by submitting a lookupJob request and providing a MoleQueue id:

{
  "jsonrpc": "2.0",
  "method": "lookupJob",
  "params": {"moleQueueId" : 17},
  "id": "XXX"
}

The server will respond with an object describing the Job's state using a similar JSON object as the submission request:

{
  "jsonrpc" : "2.0",
  "result" : {
     "cleanLocalWorkingDirectory" : false,
     "cleanRemoteFiles" : false,
     "description" : "spud slicer 28",
     "hideFromGui" : false,
     "inputFile" : {FileSpec},
     "additionalInputFiles" : [ {FileSpec}, {FileSpec}, ... ],
     "jobState" : 0,
     "localWorkingDirectory" : "/some/local/path",
     "maxWallTime" : 1440,
     "moleQueueId" : 17,
     "numberOfCores" : 4,
     "outputDirectory" : "",
     "popupOnStateChange" : true,
     "program" : "Quantum Tater",
     "queue" : "Some big ol' cluster",
     "queueId" : 7366,
     "retrieveOutput" : true
  },
  "id": "XXX"
}

If the lookup fails (i.e. the request MoleQueue id is not known to the server), a simple error reply is sent with the requested MoleQueue id as the error.data value:

{
  "error" : {
    "data" : 17,
    "code" : 0,
    "message" : "Unknown MoleQueue ID"
  },
  "jsonrpc" : "2.0",
  "id" : "XXX"
}