Configuration

After you have installed Weaver, you can customize its behaviour using multiple configuration settings.

Configuration Settings

All settings are configured using a weaver.ini configuration file. A weaver.ini.example file is provided with default values to help in the configuration process. Explanations of respective settings are also available in this example file.

The configuration file tell the application runner (e.g. Gunicorn, pserve or similar WSGI HTTP Server), how to execute Weaver as well as all settings to provide in order to personalize the application. All settings specific to Weaver employ the format weaver.<setting>.

Most configuration parameters for the manager portion of Weaver (i.e.: WSGI HTTP server for API endpoints) are defined in the [app:main] section of weaver.ini.example, while parameters specific to the worker (task queue handler) are within [celery] section. Note that multiple settings are shared between the two applications, such as the mongodb.[...] configuration or weaver.configuration options. When parameters are shared, they are usually expected to be placed in [app:main] section.

Following is a partial list of most predominant settings specific to Weaver. Many parameters provide alternative or extended functionality when employed in conjunction with other settings. Others are sometimes not necessarily required to be defined if default behaviour is desired. Refer to the relevant details that will describe in which condition they are optional and which default value or operation is applied in each situation.

Note

Refer to weaver.ini.example for the extended list of applicable settings. Some advanced configuration settings are also described in other sections of this page.

  • weaver.configuration = ADES|EMS|HYBRID|DEFAULT
    (default: DEFAULT)

    Tells the application in which mode to run.

    Enabling ADES for instance will disable some EMS-specific operations such as dispatching Workflow process steps to known remote ADES servers. ADES should be used to only run processes locally (as the working unit). EMS will always dispatch execution of jobs to other ADES except for Workflow processes that chains them.
    When HYBRID is specified, Weaver will assume both ADES and EMS roles simultaneously, meaning it will be able to execute local processes by itself and monitor dispatched execution of registered remote providers.
    Finally, DEFAULT configuration will provide very minimalistic operations as all other modes will be unavailable.
  • weaver.url = <url>
    (default: http://localhost:4001)

    Defines the full URL (including HTTP protocol/scheme, hostname and optionally additional path suffix) that will be used as base URL for all other URL settings of Weaver.

Note

This is the URL that you want displayed in responses (e.g.: processDescriptionURL or job location). For the effective URL employed by the WSGI HTTP server, refer to [server:main] section of weaver.ini.example.

  • weaver.schema_url = <url>
    (default: ${weaver.url}/json#/definitions)

    Defines the base URL of schemas to be reported in responses.

    When not provided, the running Web Application instance OpenAPI JSON path will be employed to refer to the schema definitions section. The configuration setting is available to override this endpoint by another static URL location where the corresponding schemas can be found if desired.

New in version 4.0.0.

  • weaver.wps = true|false
    (default: true)

    Enables the WPS-1/2 endpoint.

Warning

At the moment, this setting must be true to allow job execution as the worker monitors this endpoint. This could change with future developments (see issue #21).

  • weaver.wps_path = <url-path>
    weaver.wps_url = <full-url>
    (default: path /ows/wps)

    Defines the URL to employ as WPS-1/2 endpoint.

    It can either be the explicit full URL to use or the path relative to weaver.url.
    Setting weaver.wps_path is ignored if its URL equivalent is defined.
    The path variant SHOULD start with / for appropriate concatenation with weaver.url, although this is not strictly enforced.
  • weaver.wps_output_s3_bucket = <s3-bucket>
    (default: None)

    AWS S3 bucket where to store WPS outputs. Used in conjunction with weaver.wps_output_s3_region.

    When this parameter is defined, any job result generated by a process execution will be stored (uploaded) to that location. If no bucket is specified, the outputs fall back to using the location specified by weaver.wps_output_dir.

New in version 1.13.0.

See also

Configuration of AWS S3 Buckets

  • weaver.wps_output_s3_region = <s3-region>
    (default: None)

    AWS S3 region to employ for storing WPS outputs. Used in conjunction with weaver.wps_output_s3_bucket.

    When this parameter is defined as well as weaver.wps_output_s3_bucket, it is employed to define which S3 to write output files to. If not defined but weaver.wps_output_s3_bucket is specified, Weaver attempt to retrieve the region from the profile defined in AWS configuration files or environment variables.

New in version 1.13.0.

See also

Configuration of AWS S3 Buckets

  • weaver.wps_output_dir = <directory-path>
    (default: path /tmp)

    Location where WPS outputs (results from Job) will be stored for stage-out.

    When weaver.wps_output_s3_bucket is specified, only WPS XML status and log files are stored under this path. Otherwise, Job results are also located under this directory with a sub-directory named with the Job ID.
    This directory should be mapped to Weaver’s WPS output URL to serve them externally as needed.

Changed in version 4.3.0: The output directory could be nested under a contextual directory if requested during Job submission. See Outputs Location and below weaver.wps_output_context parameter for more details.

  • weaver.wps_output_context = <sub-directory-path>
    (default: None)

    Default sub-directory hierarchy location to nest WPS outputs (Job results) under.

    If defined, this parameter is used as substitute context when X-WPS-Output-Context header is omitted. When not defined, X-WPS-Output-Context header can still take effect, but omitting it will store results directly under weaver.wps_output_dir instead of default context location.

New in version 4.3.0.

See also

See Outputs Location for more details about this feature and implications of this setting.

  • weaver.wps_output_path = <url-path>
    weaver.wps_output_url = <full-url>
    (default: path /wpsoutputs)

    Endpoint that will be employed as prefix to refer to WPS outputs (Job results).

    It can either be the explicit full URL to use or the path relative to weaver.url.
    Setting weaver.wps_output_path is ignored if its URL equivalent is defined.
    The path variant SHOULD start with / for appropriate concatenation with weaver.url, although this is not strictly enforced.

Note

The resulting weaver.wps_output_url endpoint, whether directly provided or indirectly resolved by weaver.url and weaver.wps_output_path will not be served by Weaver itself. This location is returned for reference in API responses, but it is up to the infrastructure that hosts Weaver service to make this location available online as deemed necessary.

  • weaver.wps_workdir = <directory-path>
    (default: uses automatically generated temporary directory if none specified)

    Prefix where process job worker should execute the process from.
  • weaver.wps_restapi = true|false
    (default: true)

    Enable the WPS-REST endpoint.

Warning

Weaver looses most, if not all, of its useful features without this, and there won’t be much point in using it without REST endpoint, but it should technically be possible to run it as WPS-1/2 only if desired.

  • weaver.wps_restapi_path = <url-path>
    weaver.wps_restapi_url = <full-url>
    (default: path /)

    Endpoint that will be employed as prefix to refer to WPS-REST requests
    (including but not limited to OGC API - Processes schemas).

    It can either be the explicit full URL to use or the path relative to weaver.url.
    Setting weaver.wps_restapi_path is ignored if its URL equivalent is defined.
    The path variant SHOULD start with / for appropriate concatenation with weaver.url, although this is not strictly enforced.
  • weaver.wps_metadata_[...] (multiple settings)

    Metadata fields that will be rendered by either or both the WPS-1/2 and WPS-REST endpoints (Access registered process(es) (GetCapabilities, DescribeProcess)).
  • weaver.wps_email_[...] (multiple settings)

    Defines configuration of email notification functionality on job completion.

    Encryption settings as well as custom email templates are available. Default email template defined in email-template is employed if none is provided. Email notifications are sent only on job completion if an email was provided in the Execution of a process (Execute) request body (see also: Email Notification).

Note

Since Weaver employs Celery as task queue manager and MongoDB as backend, relevant settings for the configuration of Celery and the configuration of MongoDB Backend should be referred to. Processing of task jobs and results reporting is accomplished according to the specific implementation of these services. Therefore, all applicable settings and extensions should be available for custom server configuration and scaling as needed.

Configuration of AWS S3 Buckets

Any AWS S3 bucket accessed by Weaver needs to be accessible by the application, whether it is to fetch input files or to store output results. This can require from the server administrator to specify credentials by one of reference supported methodologies to provide necessary role and/or permissions. See also reference AWS Configuration which list various options that will be considered when working with S3 buckets.

Note that Weaver expects the AWS Configuration to define a default profile from which the AWS client can infer which region it needs to connect to. The S3 bucket to store files should be defined by weaver.wps_output_s3_bucket setting as presented in the previous section.

The S3 file references for input and output in Weaver are expected to be formatted as:

s3://<bucket>/<file-key>

This implicitly tells Weaver to employ the S3 bucket it was configured with as well as the automatically retrieved region from the AWS server configuration.

Alternatively, the reference can be provided with the more explicit AWS S3 link such as:

https://s3.[region-name.]amazonaws.com/<bucket>/<file-key>

In this situation, Weaver will parse it as equivalent to the prior shorthand reference format, as long as the AWS server configuration matches with all associated details from the HTTP URL variant. If this is not the case, Weaver will still attempt to fetch the file as standard HTTP reference, but read access should be granted accordingly to the corresponding bucket and file such that Weaver can access it.

Finally, in the above references, file-key is used as anything after the bucket name. In other words, this value can contain any amount of / separators and details. For example, Weaver will store process output results to S3 using file-key as a combination of <WPS-UUID>/<output-id>.<ext>, therefore forming the full job result file references as:

https://s3.<region-name>.amazonaws.com/<bucket>/<WPS-UUID>/<output-id>.<ext>

Note

Value of WPS-UUID can be retrieved from Weaver internal job storage from weaver.datatypes.Job.wps_id(). It refers to the process execution identifier that accomplished the WPS request to run the Application Package.

Configuration of WPS Processes

Weaver allows the configuration of services or processes auto-deployment using definitions from a file formatted as wps_processes.yml.example. On application startup, provided references in processes list will be employed to attempt deployment of corresponding processes locally. Given that the resources can be correctly resolved, they will immediately be available from Weaver’s API without further request needed.

For convenience, every reference URL in the configuration file can either refer to explicit process definition (i.e.: endpoint and query parameters that resolve to Access registered process(es) (GetCapabilities, DescribeProcess) response), or a group of processes under a common WPS server to iteratively register, using a Access registered process(es) (GetCapabilities, DescribeProcess) WPS endpoint. Please refer to wps_processes.yml.example for explicit format, keywords supported, and their resulting behaviour.

Note

Processes defined under processes section registered into Weaver will correspond to a local snapshot of the remote resource at that point in time, and will not update if the reference changes. On the other hand, their listing and description offering will not require the remote service to be available at all time until execution.

New in version 1.14.0: When references are specified using providers section instead of processes, the registration only saves the remote WPS provider endpoint to dynamically populate WPS processes on demand.

Using this registration method, the processes will always reflect the latest modification from the remote WPS provider.

To specify a custom YAML file, you can define the setting named weaver.wps_processes_file with the appropriate path within the employed weaver.ini file that starts your application. By default, this setting will look for the provided path as absolute location, then will attempt to resolve relative path (corresponding to where the application is started from), and will also look within the weaver/config directory. If none of the files can be found, the operation is skipped.

To ensure that this feature is disabled and to avoid any unexpected auto-deployment provided by this functionality, simply set setting weaver.wps_processes_file as undefined (i.e.: nothing after = in weaver.ini).

See also

Configuration of Request Options

Todo

complete docs

weaver.ssl_verify

New in version 1.8.0.

request_options.yml.example

Starting the Application

Todo

complete docs

make start (or similar command)