Weaver CLI and Client¶

Once Weaver package is installed (see Installation), it provides a command line interface (CLI) as well as a Python weaver.cli.WeaverClient to allow simplified interactions through shell calls or Python scripts.

This offers to the user methods to use file references (e.g.: local CWL Application Package definition) to rapidly operate with functionalities such as Deploy, Describe, Execute and any other operation described in Managing processes included in Weaver ADES/EMS section.

Please refer to following sections for more details.

Python Client Commands ¶

For details about using the Python weaver.cli.WeaverClient, please refer directly to its class documentation and its underlying methods.

Shell CLI Commands ¶

Following are the detail for the shell CLI which provides the same features.

Run Weaver operations.

usage: weaver [-h] [--stdout] [--log LOG]
              [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
              [--version]
              {deploy,undeploy,capabilities,processes,describe,execute,monitor,dismiss,status,results,upload}
              ...

Optional Arguments ¶

--version, -V: Display the version of the package.

Logging Options ¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Operations ¶

Name of the operation to run.

operation: Possible choices: deploy, undeploy, capabilities, processes, describe, execute, monitor, dismiss, status, results, upload

Sub-commands:¶

deploy ¶

Deploy a process.

weaver deploy [--stdout] [--log LOG]
              [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
              [-h] -u URL [-p PROCESS_ID] [-b BODY] [--cwl CWL | --wps WPS]
              [-t TOKEN] [-U USERNAME] [-P PASSWORD] [-D]

Optional Arguments¶

-U, --username: Username to compute the authentication token for Docker image retrieval from a private registry.
-P, --password: Password to compute the authentication token for Docker image retrieval from a private registry.

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Required Arguments¶

-u, --url: URL of the instance to run operations.

Optional Arguments¶

-p, --id, --process

Process identifier for deployment. If no --body is provided, this is required. Otherwise, provided value overrides the corresponding ID in the body.

-b, --body

Deployment body directly provided. Allows both JSON and YAML format when using file reference. If provided in combination with process ID or CWL, they will override the corresponding content. Can be provided either with a local file, an URL or literal string contents formatted as JSON.

--cwl

Application Package of the process defined using Common Workflow Language (CWL) as JSON or YAML format when provided by file reference. File reference can be a local file or URL location. Can also be provided as literal string contents formatted as JSON. Provided contents will be inserted into an automatically generated request deploy body if none was specified with --body option (note: --process must be specified instead in that case). Otherwise, it will override the appropriate execution unit section within the provided deploy body.

--wps

Reference URL to a specific process under a Web Processing Service (WPS) to package as OGC-API Process.

-t, --token

Authentication token to retrieve a Docker image reference from a private registry during execution.

-D, --delete, --undeploy

Perform undeploy step as applicable prior to deployment to avoid conflict with exiting process.

Default: False

undeploy ¶

Undeploy an existing process.

weaver undeploy [--stdout] [--log LOG]
                [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
                [-h] -u URL -p PROCESS_ID

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Required Arguments¶

-u, --url: URL of the instance to run operations.
-p, --id, --process: Identifier of the process to run undeploy operation.

capabilities (processes)¶

List available processes.

weaver capabilities [--stdout] [--log LOG]
                    [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
                    [-h] -u URL

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Required Arguments¶

-u, --url: URL of the instance to run operations.

describe ¶

Obtain an existing process description.

weaver describe [--stdout] [--log LOG]
                [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
                [-h] -u URL -p PROCESS_ID [-S {OGC,OLD}]

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Required Arguments¶

-u, --url: URL of the instance to run operations.
-p, --id, --process: Identifier of the process to run describe operation.

Optional Arguments¶

-S, --schema

Possible choices: OGC, OLD

Representation schema of the returned process description.

Default: “OGC”

execute ¶

Submit a job execution for an existing process.

weaver execute [--stdout] [--log LOG]
               [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
               [-h] -u URL -p PROCESS_ID -I INPUTS [-M] [-T TIMEOUT]
               [-W INTERVAL]

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Required Arguments¶

-u, --url

URL of the instance to run operations.

-p, --id, --process

Identifier of the process to run execute operation.

-I, --inputs

Literal input definitions, or a file path or URL reference to JSON or YAML contents defining job inputs with OGC-API or CWL schema. This parameter is required.

To provide inputs using a file reference, refer to relevant CWL Job schema or API request schema for selected format. Both mapping and listing formats are supported.

To execute a process without any inputs (e.g.: using its defaults), supply an explicit empty input (i.e.: -I "" or loaded from file as {}).

To provide inputs using literal command-line definitions, inputs should be specified using <id>=<value> convention, with distinct -I options for each applicable input value.

Values that require other type than string to be converted for job submission can include the type following the ID using a colon separator (i.e.: <id>:<type>=<value>). For example, an integer could be specified as follows: number:int=1 while a floating point number would be: number:float=1.23.

File references (href) should be specified using File as the type (i.e.: input:File=http://...). Note that File in this case is expected to be an URL location where the file can be download from. When a local file is supplied, Weaver will automatically convert it to a remote Vault File in order to upload it and make it available for the remote process.

Array input (maxOccurs > 1) should be specified using semicolon (;) separated values. The type of an item of this array can also be provided (i.e.: array:int=1;2;3).

Example: -I message='Hello Weaver' -I value:int=1234

Optional Arguments¶

-M, --monitor

Automatically perform the monitoring operation following job submission to retrieve final results. If not requested, the created job status location is directly returned.

Default: False

-T, --timeout

Wait timeout (seconds) of the maximum monitoring duration of the job execution (default: 60s). If this timeout is reached but job is still running, another call directly to the monitoring operation can be done to resume monitoring. The job execution itself will not stop in case of timeout.

Default: 60

-W, --wait, --interval

Wait interval (seconds) between each job status polling during monitoring (default: 5s).

Default: 5

monitor ¶

Monitor a pending or running job execution until completion or up to a maximum wait time.

weaver monitor [--stdout] [--log LOG]
               [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
               [-h] [-u URL] -j JOB_REFERENCE [-T TIMEOUT] [-W INTERVAL]

Required Arguments¶

-j, --job: Job URL or UUID to run monitor operation. If full Job URL is provided, the instance --url parameter can be omitted.

Optional Arguments¶

-u, --url

URL of the instance to run operations.

-T, --timeout

Wait timeout (seconds) of the maximum monitoring duration of the job execution (default: 60s). If this timeout is reached but job is still running, another call directly to the monitoring operation can be done to resume monitoring. The job execution itself will not stop in case of timeout.

Default: 60

-W, --wait, --interval

Wait interval (seconds) between each job status polling during monitoring (default: 5s).

Default: 5

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

dismiss ¶

Dismiss a pending or running job, or wipe any finished job results.

weaver dismiss [--stdout] [--log LOG]
               [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
               [-h] [-u URL] -j JOB_REFERENCE

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Required Arguments¶

-j, --job: Job URL or UUID to run dismiss operation. If full Job URL is provided, the instance --url parameter can be omitted.

Optional Arguments¶

-u, --url: URL of the instance to run operations.

status ¶

Obtain the status of a job using a reference UUID or URL. This is equivalent to doing a single-shot ‘monitor’ operation without any pooling or retries.

weaver status [--stdout] [--log LOG]
              [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
              [-h] [-u URL] -j JOB_REFERENCE

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Required Arguments¶

-j, --job: Job URL or UUID to run status operation. If full Job URL is provided, the instance --url parameter can be omitted.

Optional Arguments¶

-u, --url: URL of the instance to run operations.

results ¶

Obtain the output results description of a job. This operation can also download them from the remote server if requested.

weaver results [--stdout] [--log LOG]
               [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
               [-h] [-u URL] -j JOB_REFERENCE [-D] [-O OUT_DIR]

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Required Arguments¶

-j, --job: Job URL or UUID to run results operation. If full Job URL is provided, the instance --url parameter can be omitted.

Optional Arguments¶

-u, --url

URL of the instance to run operations.

-D, --download

Download all found job results file references to output location. If not requested, the operation simply displays the job results (default: False).

Default: False

-O, --outdir

Output directory where to store downloaded files from job results if requested (default: ${CURDIR}/{JobID}/<outputs.files>).

upload ¶

Upload a local file to the remote server vault for reference in process execution inputs. This operation is accomplished automatically for all execution inputs submitted using local files. [note: feature only available for Weaver instances]

weaver upload [--stdout] [--log LOG]
              [--quiet | --debug | --verbose | --log-level {DEBUG,ERROR,INFO,WARN,debug,error,info,warn}]
              [-h] -u URL [-c CONTENT_TYPE] -f FILE

Logging Options¶

Options that configure output logging.

--stdout

Enforce logging to stdout for display in console.

Default: False

--log, --log-file

Output file to write generated logs.

--quiet, -q

Do not output anything else than error.

Default: False

--debug, -d

Enable extra debug logging.

Default: False

--verbose, -v

Output informative logging details.

Default: False

--log-level, -l

Possible choices: DEBUG, ERROR, INFO, WARN, debug, error, info, warn

Explicit log level to employ (default: None).

Required Arguments¶

-u, --url: URL of the instance to run operations.
-f, --file: Local file path to upload to the vault.

Optional Arguments¶

-c, --content-type: Content-Type of the file to apply. This should be an IANA Media-Type, optionally with additional parameters such as charset. If not provided, attempts to guess it based on the file extension.

CLI and Client Examples ¶

Following sections present different typical usage of the Shell CLI Commands and Python Client Commands. Operations are equivalent between the CLI and Python client.

Note that more operations and option parameters are available, and are not all necessarily represented in below examples.

For each of the following examples, the client is created as follows:

client = WeaverClient(url="{WEAVER_URL}")

Deploy Example ¶

Todo

example

Undeploy Example ¶

Todo

example

GetCapabilities Example ¶

Accomplishes the GetCapabilities request to obtain a list of available Process.

weaver capabilities -u {WEAVER_URL}

client.capabilities()

Sample Output:

{
  "description": "Listing of available processes successful.",
  "processes": [
    "docker-demo-cat",
    "docker-python-script",
    "Echo",
    "file_index_selector",
    "file2string_array",
    "image-utils",
    "jsonarray2netcdf",
    "las2tif",
    "metalink2netcdf",
    "sleep"
  ],
  "page": 0,
  "total": 25
}

DescribeProcess Example ¶

Accomplishes the DescribeProcess request to obtain the Process definition.

weaver describe -u {WEAVER_URL} -p jsonarray2netcdf

client.describe("jsonarray2netcdf")

Sample Output:

{
  "id": "jsonarray2netcdf",
  "title": "JSON array to NetCDF",
  "version": "1.1",
  "abstract": "Extracts and fetches NetCDF files from a JSON file containing an URL string array, and provides them on the output directory.",
  "keywords": [],
  "metadata": [],
  "inputs": {
    "input": {
      "title": "input",
      "formats": [
        {
          "mimeType": "application/json",
          "default": true
        }
      ],
      "minOccurs": "1",
      "maxOccurs": "1"
    }
  },
  "outputs": {
    "output": {
      "title": "output",
      "formats": [
        {
          "mimeType": "application/x-netcdf",
          "default": true
        }
      ]
    }
  },
  "visibility": "public",
  "processEndpointWPS1": "http://localhost:4002/wps",
  "processDescriptionURL": "http://localhost:4002/processes/jsonarray2netcdf/jobs",
  "executeEndpoint": "http://localhost:4002/processes/jsonarray2netcdf/jobs",
  "links": [
    {
      "type": "application/json",
      "title": "Process description.",
      "hreflang": "en-CA",
      "href": "http://localhost:4002/processes/jsonarray2netcdf",
      "rel": "self"
    },
    {
      "type": "application/json",
      "title": "Process description.",
      "hreflang": "en-CA",
      "href": "http://localhost:4002/processes/jsonarray2netcdf",
      "rel": "process-desc"
    },
    {
      "type": "application/json",
      "title": "Process execution endpoint for job submission.",
      "hreflang": "en-CA",
      "href": "http://localhost:4002/processes/jsonarray2netcdf/execution",
      "rel": "execute"
    },
    {
      "type": "application/json",
      "title": "List of registered processes.",
      "hreflang": "en-CA",
      "href": "http://localhost:4002/processes",
      "rel": "collection"
    },
    {
      "type": "application/xml",
      "title": "Service definition.",
      "hreflang": "en-CA",
      "href": "http://localhost:4002/ows/wps?service=WPS&request=GetCapabilities",
      "rel": "service-desc"
    }
  ]
}

Execute Example ¶

Accomplishes the Execute request to obtain launch a Job with the specified Process and provided inputs.

Todo

example

When running the execute operation, any detected local file reference will be automatically uploaded as Vault file in order to make it available for the remote Weaver server for Process execution.