Weaver¶
Workflow Execution Management Service (EMS)
Application, Deployment and Execution Service (ADES)
- Weaver (the nest-builder)
Weaver birds build exquisite and elaborate nest structures that are a rival to any human feat of engineering. Some of these nests are the largest structures to be built by birds. (Eden).
Although weavers are named for their elaborately woven nests, some are notable for their selective parasitic nesting habits instead. (Wikipedia)
Weaver is an Execution Management Service (EMS) that allows the execution of workflows chaining various applications and Web Processing Services (WPS) inputs and outputs. Remote execution is deferred by the EMS to an Application Deployment and Execution Service (ADES), as defined by Common Workflow Language (CWL) configurations.
Weaver can be launched either as an EMS or an ADES according to configuration values it is deployed with. For more details, see Configuration section.
dependencies |
|
---|---|
build status |
|
releases |
Summary¶
Weaver is primarily an Execution Management Service (EMS) that allows the execution of workflows chaining various applications and Web Processing Services (WPS) inputs and outputs. Remote execution of each process in a workflow chain is dispatched by the EMS to one or many registered Application Deployment and Execution Service (ADES) by ensuring the transfer of files accordingly between instances when located across multiple remote locations.
Weaver can also accomplish the ADES role in order to perform application deployment at the data source using the application definition provided by Common Workflow Language (CWL) configuration. It can then directly execute a registered process execution with received inputs from a WPS request to expose output results for a following ADES in a EMS workflow execution chain.
Weaver extends the OGC API - Processes (WPS-REST bindings) by providing additional functionalities such as more detailed job log routes, adding more process management request options than required by the standard, and supporting remote providers to name a few. Because of this, not all features offered in Weaver are guaranteed to be applicable on other similarly behaving ADES and/or EMS instances. The reference specification is tracked to preserve the minimal conformance requirements and provide feedback to Open Geospatial Consortium (OGC) in this effect.
Weaver can be launched either as an EMS or an ADES according to configuration values it is deployed with. For more details, see Configuration and Documentation sections.
Links¶
Docker image repositories:
CRIM registry: ogc/weaver
OGC processes: ogc-public
DockerHub: pavics/weaver
$ docker pull pavics/weaver:1.14.0
For convenience, following tags are also available:
weaver:1.14.0-manager
: Weaver image that will run the API for WPS process and job management.weaver:1.14.0-worker
: Weaver image that will run the process job runner application.
Following links correspond to existing servers with Weaver configured as EMS/ADES instances respectively.
ADES Test server: https://ogc-ades.crim.ca/weaver/
EMS Test server: https://ogc-ems.crim.ca/weaver/
EMS Extra server: https://ogc.crim.ca/ems/
Note
The test servers will not necessarily be up-to-date with the latest version.
Request the ${server}/weaver/versions
route to verify the running version.
Configuration¶
All configuration settings can be overridden using a weaver.ini
file that will be picked during
instantiation of the application. An example of such file is provided here: weaver.ini.example.
Setting Weaver’s operational mode (EMS/ADES) is accomplished using the
weaver.configuration
field of weaver.ini
.
For more configuration details, please refer to Documentation.
Documentation¶
The REST API documentation is auto-generated and served under any running Weaver application on route
{WEAVER_URL}/api/
. This documentation will correspond to the version of the executed Weaver application.
For the latest documentation, you can refer to the OpenAPI Specification served directly on readthedocs.
More ample details about installation, configuration and usage are also provided on readthedocs. These are generated from corresponding information provided in docs source directory.
Extra Details & Sponsors¶
The project was initially developed upon OGC Testbed-14 – ESA Sponsored Threads – Exploitation Platform findings and following improvements. It is also advanced with sponsorship of U.S. Department of Energy to support common API of the Earth System Grid Federation (ESGF). The findings are reported on the OGC Testbed-14 thread, and more explicitly in the ADES & EMS Results and Best Practices Engineering Report.
The project has been employed for OGC Testbed-15 - ML Thread to demonstrate the use of Machine Learning interactions with OGC web standards in the context of natural resources applications. The advancements are reported through the OGC Testbed-15: Machine Learning Engineering Report.
Developments are continued in OGC Testbed-16 to improve methodologies in order to provide better interoperable geospatial data processing in the areas of Earth Observation Application Packages.
The project is furthermore developed through the Data Analytics for Canadian Climate Services (DACCS) initiative.
Weaver is a prototype implemented in Python with the Pyramid web framework. It is part of PAVICS and Birdhouse ecosystems.
Table of Contents¶
Installation¶
The installation is using the Python distribution system Miniconda (default installation of no conda
found)
to maintain software dependencies. Any conda
installation should work the same. To use a pre-installed conda
distribution, simply make sure that it can be found on the shell path.
Requirements¶
The installation works on Linux 64 bit distributions (tested on Ubuntu 16.04).
Note
Windows is not officially supported, but some patches have been applied to help using it. If you find some problems, please submit a new issue or open a pull request with fixes.
From GitHub Sources¶
Install weaver as normal user from GitHub sources:
git clone https://github.com/crim-ca/weaver.git
cd weaver
make install
If no conda
environment is activated, the install
process will setup a new or reuse the conda environment
named weaver
and install all dependency packages. If an environment is activated, Weaver will be installed in
that environment. You can also enforce a specific environment using:
make CONDA_ENV=<my-env> install
Please refer to Configuration and Running sections for following steps.
Configuration¶
After you have installed Weaver, you can customize its behaviour using multiple configuration settings.
Configuration Settings¶
All settings are configured using a weaver.ini
configuration file. A weaver.ini.example file is provided
with default values to help in the configuration process. Explanations of respective settings are also available in
this example file.
The configuration file tell the application runner (e.g. Gunicorn, pserve
or similar WSGI HTTP Server), how to
execute Weaver as well as all settings to provide in order to personalize the application. All settings specific to
Weaver employ the format weaver.<setting>
.
Most configuration parameters for the manager portion of Weaver (i.e.: WSGI HTTP server for API endpoints) are
defined in the [app:main]
section of weaver.ini.example, while parameters specific to the worker (task queue
handler) are within [celery]
section. Note that multiple settings are shared between the two applications, such as
the mongodb.[...]
configuration or weaver.configuration
options. When parameters are shared, they are usually
expected to be placed in [app:main]
section.
Following is a partial list of most predominant settings specific to Weaver. Many parameters provide alternative or extended functionality when employed in conjunction with other settings. Others are sometimes not necessarily required to be defined if default behaviour is desired. Refer to the relevant details that will describe in which condition they are optional and which default value or operation is applied in each situation.
Note
Refer to weaver.ini.example for the extended list of applicable settings. Some advanced configuration settings are also described in other sections of this page.
weaver.configuration = ADES|EMS
(default:DEFAULT
)Tells the application in which mode to run.Enabling ADES for instance will disable some EMS-specific operations such as dispatching Workflow process steps to known remote ADES servers. ADES should be used to run processes locally (as the working unit).Default configuration will provide very minimalistic operations as both other variants will be unavailable.weaver.url = <url>
(default:https://localhost
)Defines the full URL (including HTTP protocol/scheme, hostname and optionally additional path suffix) that will be used as base URL for all other URL settings of Weaver.
Note
This is the URL that you want displayed in responses (e.g.: processDescriptionURL
or job location
).
For the effective URL employed by the WSGI HTTP server, refer to [server:main]
section of weaver.ini.example.
weaver.wps = true|false
(default:true
)Enables the WPS-1/2 endpoint.
Warning
At the moment, this setting must be
true
to allow job execution as the worker monitors this endpoint. This could change with future developments (see issue #21).
weaver.wps_path = <url-path>
weaver.wps_url = <full-url>
(default: path/ows/wps
)Defines the URL to employ as WPS-1/2 endpoint.It can either be the explicit full URL to use or the path relative toweaver.url
.Settingweaver.wps_path
is ignored if its URL equivalent is defined.The path variant SHOULD start with/
for appropriate concatenation withweaver.url
, although this is not strictly enforced.weaver.wps_output_s3_bucket = <s3-bucket>
(default:None
)AWS S3 bucket where to store WPS outputs. Used in conjunction withweaver.wps_output_s3_region
.When this parameter is defined, any job result generated by a process execution will be stored (uploaded) to that location. If no bucket is specified, the outputs fall back to using the location specified byweaver.wps_output_dir
.
New in version 1.13.0.
See also
weaver.wps_output_s3_region = <s3-region>
(default:None
)AWS S3 region to employ for storing WPS outputs. Used in conjunction withweaver.wps_output_s3_bucket
.When this parameter is defined as well asweaver.wps_output_s3_bucket
, it is employed to define which S3 to write output files to. If not defined butweaver.wps_output_s3_bucket
is specified, Weaver attempt to retrieve the region from the profile defined in AWS configuration files or environment variables.
New in version 1.13.0.
See also
weaver.wps_output_dir = <directory-path>
(default:/tmp
)Location where WPS outputs (results from jobs) will be stored for stage-out.Whenweaver.wps_output_s3_bucket
is specified, only WPS XML status and log files are stored under this path. Otherwise, job results are also located under this directory with a sub-directory named with the job ID.This directory should be mapped to Weaver’s WPS output URL to serve them externally as needed.weaver.wps_output_path = <url-path>
weaver.wps_output_url = <full-url>
(default: path/wpsoutputs
)Endpoint that will be employed as prefix to refer to WPS outputs (job results).It can either be the explicit full URL to use or the path relative toweaver.url
.Settingweaver.wps_output_path
is ignored if its URL equivalent is defined.The path variant SHOULD start with/
for appropriate concatenation withweaver.url
, although this is not strictly enforced.weaver.wps_workdir = <directory-path>
(default: uses automatically generated temporary directory if none specified)Prefix where process job worker should execute the process from.weaver.wps_restapi = true|false
(default:true
)Enable the WPS-REST endpoint.
Warning
Weaver looses most, if not all, of its useful features without this, and there won’t be much point in using it without REST endpoint, but it should technically be possible to run it as WPS-1/2 only if desired.
weaver.wps_restapi_path = <url-path>
weaver.wps_restapi_url = <full-url>
(default: path/
)Endpoint that will be employed as prefix to refer to WPS-REST requests(including but not limited to OGC API - Processes schemas).It can either be the explicit full URL to use or the path relative toweaver.url
.Settingweaver.wps_restapi_path
is ignored if its URL equivalent is defined.The path variant SHOULD start with/
for appropriate concatenation withweaver.url
, although this is not strictly enforced.weaver.wps_metadata_[...]
(multiple settings)Metadata fields that will be rendered by either or both the WPS-1/2 and WPS-REST endpoints (Access registered process(es) (GetCapabilities, DescribeProcess)).weaver.wps_email_[...]
(multiple settings)Defines configuration of email notification functionality on job completion.Encryption settings as well as custom email templates are available. Default email template defined in email-template is employed if none is provided. Email notifications are sent only on job completion if an email was provided in the Execution of a process (Execute) request body (see also: Email Notification).
Note
Since Weaver employs Celery as task queue manager and MongoDB as backend, relevant settings for the configuration of Celery and the configuration of MongoDB Backend should be referred to. Processing of task jobs and results reporting is accomplished according to the specific implementation of these services. Therefore, all applicable settings and extensions should be available for custom server configuration and scaling as needed.
Configuration of AWS S3 Buckets¶
Any AWS S3 bucket accessed by Weaver needs to be accessible by the application, whether it is to fetch input files or to store output results. This can require from the server administrator to specify credentials by one of reference supported methodologies to provide necessary role and/or permissions. See also reference AWS Configuration which list various options that will be considered when working with S3 buckets.
Note that Weaver expects the AWS Configuration to define a default profile from which the AWS
client can infer which region it needs to connect to. The S3 bucket to store files should be defined by
weaver.wps_output_s3_bucket
setting as presented in the previous section.
The S3 file references for input and output in Weaver are expected to be formatted as:
s3://<bucket>/<file-key>
This implicitly tells Weaver to employ the S3 bucket it was configured with as well as the automatically retrieved region from the AWS server configuration.
Alternatively, the reference can be provided with the more explicit AWS S3 link such as:
https://s3.[region-name.]amazonaws.com/<bucket>/<file-key>
In this situation, Weaver will parse it as equivalent to the prior shorthand reference format, as long as the AWS server configuration matches with all associated details from the HTTP URL variant. If this is not the case, Weaver will still attempt to fetch the file as standard HTTP reference, but read access should be granted accordingly to the corresponding bucket and file such that Weaver can access it.
Finally, in the above references, file-key
is used as anything after the bucket name. In other words, this value
can contain any amount of /
separators and details. For example, Weaver will store process output results to S3
using file-key
as a combination of <WPS-UUID>/<output-id>.<ext>
, therefore forming the full job result file
references as:
https://s3.<region-name>.amazonaws.com/<bucket>/<WPS-UUID>/<output-id>.<ext>
Note
Value of WPS-UUID
can be retrieved from Weaver internal job storage from weaver.datatypes.Job.wps_id()
.
It refers to the process execution identifier that accomplished the WPS request to run the Application Package.
Configuration of WPS Processes¶
Weaver allows the configuration of services or processes auto-deployment using definitions from a file formatted
as wps_processes.yml.example. On application startup, provided references in processes
list will be employed
to attempt deployment of corresponding processes locally. Given that the resources can be correctly resolved, they
will immediately be available from Weaver’s API without further request needed.
For convenience, every reference URL in the configuration file can either refer to explicit process definition (i.e.: endpoint and query parameters that resolve to Access registered process(es) (GetCapabilities, DescribeProcess) response), or a group of processes under a common WPS server to iteratively register, using a Access registered process(es) (GetCapabilities, DescribeProcess) WPS endpoint. Please refer to wps_processes.yml.example for explicit format, keywords supported, and their resulting behaviour.
Note
Processes defined under processes
section registered into Weaver will correspond to a local snapshot of
the remote resource at that point in time, and will not update if the reference changes. On the other hand, their
listing and description offering will not require the remote service to be available at all time until execution.
New in version 1.14.0: When references are specified using providers
section instead of processes
, the registration
only saves the remote WPS provider endpoint to dynamically populate WPS processes on demand.
Using this registration method, the processes will always reflect the latest modification from the remote WPS provider.
To specify a custom YAML file, you can define the setting named weaver.wps_processes_file
with the appropriate path
within the employed weaver.ini
file that starts your application. By default, this setting will look for the
provided path as absolute location, then will attempt to resolve relative path (corresponding to where the application
is started from), and will also look within the weaver/config
directory. If none of the files can be found, it will
try to use a copy of wps_processes.yml.example.
To disable this feature and avoid any auto-deployment provided by this functionality, simply set setting
weaver.wps_processes_file
as undefined (i.e.: nothing after =
in weaver.ini
).
Starting the Application¶
Todo
complete docs
make start
(or similar command)
need to start
gunicorn/pserve
(example Dockerfile-manager)need to start
celery
worker (example Dockerfile-worker)
Running Weaver¶
Running Weaver Service¶
Before running Weaver, you must make sure that the required MongoDB connection is accessible according to
specified connection settings in weaver/config/weaver.ini
.
See also
Weaver installation comes with a Makefile which provides a shortcut command to start the application with Gunicorn:
$ make start # start Weaver WSGI application server
Weaver should be running after this operation.
It will be available under the configured URL endpoint in weaver/config/weaver.ini
.
If everything was configured correctly, calling this URL (default: http://localhost:4001
) should
provide a response containing a JSON body with basic information about Weaver.
Execution Details¶
To execute, Weaver requires two type of application executed in parallel. First, it requires a WSGI HTTP server
that will run the application to provide API endpoints. This is referred to as weaver-manager
in the provided
docker images. Second, Weaver requires a Celery task queue handler to execute submitted process jobs. This
is referred to as weaver-worker
in built Docker images.
For specific details about configuration of both applications, please refer to Configuration section.
The typical commands that need to be executed for the manager and worker applications should be similar to the following calls. Obviously, additional arguments supported by the corresponding applications can be provided.
# manager
pserve <weaver-root>/config/weaver.ini
# worker
celery worker -A pyramid_celery.celery_app --ini <weaver-root>/config/weaver.ini
Tutorial¶
Using the WPS application included in Weaver¶
Install Weaver (see: Installation) and make sure all required components are started and running (see: Configuration).
Then, execute the desired WPS or WPS-REST request according to desired operation mode and version.
For all following examples, ${WEAVER_URL}
is used to specify your application URL endpoint configuration.
By default, this value should be localhost:4001
.
Note
This tutorial section is a minimal introduction to available requests and endpoints. Please refer to processes for further details, such as detailed request payload contents, types of processes and additional operations that compose a typical process execution workflow. Similarly, refer to Application Package for further details about the definition of the reference application executed by the deployed processes.
WPS-1/2 requests¶
Specifying the appropriate version=<1.0.0|2.0.0>
parameter in the URL as required.
Run a WPS-1/WPS-2 GetCapabilities
request:
$ curl -k "${WEAVER_URL}/ows/wps?service=wps&request=getcapabilities"
You should receive an XML response listing service details and available processes.
Run a WPS-1/WPS-2 DescribeProcess
request (built-in process jsonarray2netcdf
):
$ curl -k "${WEAVER_URL}/ows/wps?service=wps&request=describeprocess&identifier=jsonarray2netcdf&version=1.0.0"
This will provide you with an XML response listing the specific process details such and inputs/outputs and description.
We can now use the process to execute a WPS request. To do so, we will need some input data files to call it. First, let’s create a JSON file with some dummy NetCDF file reference for demonstration purpose.
$ echo 'Test WPS' > /tmp/test.nc
$ echo '["file:///tmp/test.nc"]' > /tmp/test.json
Then, run the WPS-1/WPS-2 Execute
request (built-in process jsonarray2netcdf
) as follow:
$ curl -k "${WEAVER_URL}/ows/wps?service=wps&request=execute&identifier=jsonarray2netcdf&version=1.0.0 \
&DataInputs=input=file:///tmp/test.json"
The execution of the process should read the JSON list with our dummy NetCDF file and make it available (as a copy)
on the output parameter named output
with a path matching the configured output WPS path of the application.
Note
All above WPS-1/2 requests suppose that configuration setting weaver.wps_path /ows/wps
(default value).
The request URL have to be adjusted accordingly if this parameter is modified.
Also, the provided file reference is relative where Weaver application is running. If you want to employ a remote server instance, you will have to either place the file on this server file system at a location Weaver has access to, or provide the file through an HTTP URL.
WPS-3 requests¶
All previous operations for listing available processes (GetCapabilities
), describing or executing a WPS-1/2
process can also be accomplished using the WPS-3 REST JSON interface. For instance, listing processes is done like so:
$ curl -k "${WEAVER_URL}/processes"
Individual process details (DescribeProcess
) can be obtained with the following method
(e.g.: built-in process jsonarray2netcdf
in this case):
$ curl -k "${WEAVER_URL}/processes/jsonarray2netcdf"
And execution of this process can be accomplished with the following request:
$ curl -X POST "${WEAVER_URL}/processes/jsonarray2netcdf/jobs" \
-H "Content-Type: application/json" \
-d '{"inputs": [{"id": "input", "href": "file:///tmp/test.json"}],
"outputs": [{"id": "output", "transmissionMode": "reference"}],
"response": "document",
"mode": "async"}'
The JSON response should provide a location
field specifying where the job status can be verified.
Upon successful job completion, an output
reference URL should have been generated just as with
the WPS-1/2 example.
The WPS-3 interface allows further operations such as job monitoring, specific output listing, log reporting, etc.
For all available operations and specific details about them, please refer to OpenAPI schemas (they will also be
rendered on route ${WEAVER_URL}/api
when running Weaver application).
Endpoint Content-Type¶
Todo
wps-1/2 xml default, json supported wps-2
Todo
wps-rest json only (for now, xml also if implemented) https://github.com/crim-ca/weaver/issues/125 https://github.com/crim-ca/weaver/issues/126
Next Steps¶
Have a look to the Processes, Package and FAQ sections.
The full OpenAPI Specification is also available for request details.
Processes¶
Type of Processes¶
Weaver supports multiple type of processes, as listed below. Each one of them are accessible through the same API interface, but they have different implications.
WPS-REST (a.k.a.: WPS-3, OGC API - Processes)
See also
Section Examples provides multiple concrete use cases of Deploy and Execute request payloads for diverse set of applications.
Builtin¶
These processes come pre-packaged with Weaver. They will be available directly on startup of the application and re-updated on each boot to make sure internal database references are updated with any source code changes.
Theses processes typically correspond to utility operations. They are specifically useful when employed as
step
within a Workflow process that requires data-type conversion between input/output of similar, but not
perfectly, compatible definitions.
For example, process weaver.processes.builtin.jsonarray2netcdf
takes a single input JSON file which its
content contains an array-list of NetCDF file references, and returns them directly as the corresponding list of output
files. These two different file formats (single JSON to multiple NetCDF) can then be used to map two processes with
these respective output and inputs.
As of the latest release, following builtin processes are available:
All builtin processes are marked with weaver.processes.constants.CWL_REQUIREMENT_APP_BUILTIN
in the
CWL hints
section and are all defined in weaver.processes.builtin
.
WPS-1/2¶
This kind of process corresponds to a traditional WPS XML or JSON endpoint (depending of supported version) prior to WPS-REST specification. When the WPS-REST process is deployed in Weaver using an URL reference to an WPS-1/2 process, Weaver parses and converts the XML or JSON body of the response and registers the process locally using this definition. This allows a remote server offering limited functionalities (e.g.: no REST bindings supported) to provide them through Weaver.
A minimal Deploy request body for this kind of process could be as follows:
{
"processDescription": {
"process": {
"id": "my-process-reference"
}
},
"executionUnit": [
{
"href": "https://example.com/wps?service=WPS&request=DescribeProcess&identifier=my-process&version=1.0.0"
}
]
}
This would tell Weaver to locally deploy the my-process-reference
process using the WPS-1 URL reference that is
expected to return a DescribeProcess
XML schema. Provided that this endpoint can be resolved and parsed according
to typical WPS specification, this should result into a successful process registration. The deployed process would
then be accessible with DescribeProcess requests.
The above deployment procedure can be automated on startup using Weaver’s wps_processes.yml
configuration file.
Please refer to Configuration of WPS Processes section for more details on this matter.
Warning
Because Weaver creates a snapshot of the reference process at the moment it was deployed, the local process definition could become out-of-sync with the remote reference where the Execute request will be sent. Refer to Remote Provider section for more details to work around this issue.
See also
WPS-REST¶
This process type is the main component of Weaver. All other process types are converted to this one either through some parsing (e.g.: WPS-1/2) or with some requirement indicators (e.g.: Builtin, Workflow) for special handling.
When deploying one such process directly, it is expected to have a reference to a CWL Application Package. This is most of the time employed to wrap a reference docker image process. The reference package can be provided in multiple ways as presented below.
Note
When a process is deployed with any of the below supported Application Package formats, additional parsing of this CWL as well as complementary details directly within the WPS deployment body is accomplished. See Correspondance between CWL and WPS fields section for more details.
Package as Literal Unit Block¶
In this situation, the CWL definition is provided as is using JSON-formatted package embedded within the
POST {WEAVER_URL}/processes
(Deploy) request. The request payload would take the following shape:
{
"processDescription": {
"process": {
"id": "my-process-reference"
}
},
"executionUnit": [
{
"unit": {
"cwlVersion": "v1.0",
"class": "CommandLineTool",
"inputs": ["<...>"],
"outputs": ["<...>"],
"<...>": "<...>"
}
}
]
}
ESGF-CWT¶
For traditional WPS-1 process type, Weaver adds default values to CWL definition. As we can see in
weaver/processes/wps_package.py
, the following default values for the CWL package are:
cwl_package = OrderedDict([
("cwlVersion", "v1.0"),
("class", "CommandLineTool"),
("hints", {
CWL_REQUIREMENT_APP_WPS1: {
"provider": get_url_without_query(wps_service_url),
"process": process_id,
}}),
])
In ESGF-CWT processes, ESGF-CWTRequirement
hint must be used instead of usual WPS1Requirement
, contained
in the weaver.processes.constants.CWL_REQUIREMENT_APP_WPS1
variable. The handling of this technicality is
handled in weaver/processes/wps_package.py
. We can define ESGF-CWT processes using this syntax:
{
"cwlVersion": "v1.0",
"class": "CommandLineTool",
"hints": {
"ESGF-CWTRequirement": {
"provider": "https://edas.nccs.nasa.gov/wps/cwt",
"process": "xarray.subset"
}
}
}
Workflow¶
Processes categorized as Workflow
are very similar to WPS-REST processes. From the API standpoint, they
actually look exactly the same as an atomic process when calling DescribeProcess or Execute requests.
The difference lies within the referenced Application Package which uses a CWL Workflow instead of
typical CWL CommandLineTool, and therefore, modifies how the process is internally executed.
For Workflow
processes to be deploy-able and executable, it is mandatory that Weaver is configured as
EMS (see: Configuration Settings). This requirement is due to the nature of workflows that chain
processes that need to be dispatched to known remote ADES servers (see: Configuration of Data Sources
and Workflow Operations).
Given that a Workflow
process was successfully deployed and that all process steps can be resolved, calling
its Execute request will tell Weaver to parse the chain of operations and send step process execution requests
to relevant ADES picked according to data sources. Each step’s job will then gradually be monitored from the
remote ADES until completion, and upon successful result, the EMS will retrieve the data references to
pass it down to the following step. When the complete chain succeeds, the final results of the last step will be
provided as Workflow
output as for atomic processes. In case of failure, the error will be indicated in the log
with the appropriate step and message where the error occurred.
Note
Although chaining sub-workflow(s) within a bigger scoped workflow is technically possible, this have not yet been fully explored (tested) in Weaver. There is a chance that Data-Source resolution fails to identify where to dispatch the step in this situation. If this impacts you, please vote and indicate your concern on issue #171.
Remote Provider¶
Remote provider correspond to a remote service that provides similar interfaces as supported by Weaver (WPS-like). For example, a remote WPS-1 XML endpoint can be referenced as a provider. When an API Providers-scoped request is executed, for example to list is processes capabilities (see GetCapabilities), Weaver will send the corresponding request using the registered reference URL to access the remote server and reply with parsed response, as if they its processes were registered locally.
Since remote providers obviously require access to the remote service, Weaver will only be able to provide results if the service is accessible with respect to standard implementation features and supported specifications.
The main advantage of using Weaver’s endpoint rather than directly accessing the referenced remote provider processes is in the case of limited functionality offered by the service. For instance, WPS-1 do not always offer GetStatus feature, and there is no extensive job monitoring availability. Since Weaver wraps the original reference with its own endpoints, these features indirectly become employable. Similarly, although WPS-1 offer XML-only endpoints, the parsing operation accomplished by Weaver makes theses services available as WPS-REST JSON endpoints. On top of that, registering remote providers into Weaver allows the user to use it as a central hub to keep references to all his accessible services and dispatch jobs from a common location.
A remote provider differs from previously presented WPS-1/2 processes such that the underlying processes of the service are not registered locally. For example, if a remote service has two WPS processes, only top-level service URL will be registered locally (in Weaver’s database) and the application will have no explicit knowledge of these remote processes. When calling process-specific requests (e.g.: DescribeProcess or Execute), Weaver will re-send the corresponding request directly to the remote provider each time and return the result accordingly. On the other hand, a WPS-1/2 reference would be parsed and saved locally with the response at the time of deployment. This means that a deployed WPS-1/2 reference would act as a snapshot of the reference (which could become out-of-sync), while Remote Provider will dynamically update according to the re-fetched response from the remote service. If our example remote service was extended to have a third WPS process, it would immediately be reflected in GetCapabilities and DescribeProcess retrieved via Weaver Providers-scoped requests. This would not be the case for the WPS-1/2 reference that would need manual update (deploy the third process to register it in Weaver).
An example body of the register provider request could be as follows:
{
"id": "my-service",
"url": "https://example.com/wps",
"public": true
}
Then, processes of this registered remote-provider will be accessible. For example, if the referenced service by
the above URL add a WPS process identified by my-process
, its JSON description would be obtained with following
request (DescribeProviderProcess):
GET {WEAVER_URL}/providers/my-service/processes/my-process
Note
Process my-process
in the example is not registered locally. From the point of view of Weaver’s processes
(i.e.: route /processes/{id}
), it does NOT exist. You must absolutely use the provider-prefixed route
/providers/{id}/processes/{id}
to explicitly fetch and resolve this remote process definition.
Warning
API requests scoped under Providers are Weaver-specific implementation. These are not part of OGC API - Processes specification.
Managing processes included in Weaver ADES/EMS¶
Following steps represent the typical steps applied to deploy a process, execute it and retrieve the results.
Register a new process (Deploy)¶
Deployment of a new process is accomplished through the POST {WEAVER_URL}/processes
POST {WEAVER_URL}/processes
(Deploy) request.
The request body requires mainly two components:
processDescription
:Defines the process identifier, metadata, inputs, outputs, and some execution specifications. This mostly corresponds to information that corresponds to a traditional WPS definition.executionUnit
:Defines the core details of the Application Package. This corresponds to the explicit CWL definition that indicates how to execute the given application.
Upon deploy request, Weaver will either respond with a successful result, or with the appropriate error message, whether caused by conflicting ID, invalid definitions or other parsing issues. A successful process deployment will result in this process to become available for following steps.
Warning
When a process is deployed, it is not necessarily available immediately. This is because process visibility also
needs to be updated. The process must be made public to allow its discovery. For updating visibility, please
refer to the PUT {WEAVER_URL}/processes/{id}/visibility
(Visibility) request.
After deployment and visibility preconditions have been met, the corresponding process should become available through DescribeProcess requests and other routes that depend on an existing process.
Note that when a process is deployed using the WPS-REST interface, it also becomes available through the WPS-1/2 interface with the same identifier and definition. Because of compatibility limitations, some parameters in the WPS-1/2 might not be perfectly mapped to the equivalent or adjusted WPS-REST interface, although this concerns mostly only new features such as status monitoring. For most traditional use cases, properties are mapped between the two interfaces, but it is recommended to use the WPS-REST one because of the added features.
Access registered process(es) (GetCapabilities, DescribeProcess)¶
Available processes can all be listed using GET {WEAVER_URL}/processes
(GetCapabilities) request. This request will return all locally registered
process summaries. Other return formats and filters are also available according to provided request query parameters.
Note that processes not marked with public visibility will not be listed in this result.
For more specific process details, the GET {WEAVER_URL}/processes/{id}
(DescribeProcess) request should be used. This will return all information
that define the process references and expected inputs/outputs.
Note
For remote processes (see: Remote Provider), Provider requests are also available for more fine-grained
search of underlying processes. These processes are not necessarily listed as local processes, and will therefore
sometime not yield any result if using the typical DescribeProcess
endpoint.
All routes listed under Process requests should normally be applicable for remote processes by prefixing
them with /providers/{id}
.
Execution of a process (Execute)¶
Process execution (i.e.: submitting a Job) is accomplished using the POST {WEAVER_URL}/processes/{id}/jobs
(Execute) request. This section will
first describe the basics of this request format, and after go into details for specific use cases and parametrization
of various input/output combinations. Let’s employ the following example of JSON body sent to the Job execution
to better illustrate the requirements.
{
"mode": "async",
"response": "document",
"inputs": [
{
"id": "input-file",
"href": "<some-file-reference"
},
{
"id": "input-value",
"data": 1,
}
],
"outputs": [
{
"id": "output",
"transmissionMode": "reference"
}
]
}
Basic Details¶
The first field is mode
, it basically tells whether to run the Process in a blocking (sync
) or
non-blocking (async
) manner. Note that support is currently limited for mode sync
as this use case is often more
cumbersome than async
execution. Effectively, sync
mode requires to have a task worker executor available
to run the Job (otherwise it fails immediately due to lack of processing resource), and the requester must wait
for the whole execution to complete to obtain the result. Given that Process could take a very long time to
complete, it is not practical to execute them in this manner and potentially have to wait hours to retrieve outputs.
Instead, the preferred and default approach is to request an async
Job execution. When doing so, Weaver
will add this to a task queue for processing, and will immediately return a Job identifier and location where
the user can probe for its status, using GetStatus monitoring request. As soon as any task worker becomes available,
it will pick any leftover queued Job to execute it.
The second field is response
. At the time being, Weaver only supports document
value. This parameter is
present only for compatibility with other ADES implementation, but does not actually affects Weaver’s
response.
Following are the inputs
definition. This is the most important section of the request body. It defines which
parameters to forward to the referenced Process to be executed. All id
elements in this Job request
body must correspond to valid inputs
from the definition returned by DescribeProcess response. Obviously, all
formatting requirements (i.e.: proper file MIME-types), data types (e.g.: int
, string
, etc.) and
validations rules (e.g.: minOccurs
, AllowedValues
, etc.) must also be fulfilled. When providing files as input,
multiple protocols are supported. See later section File Reference Types for details.
Finally, the outputs
section defines, for each id
corresponding to the Process definition, how to
report the produced outputs from a successful Job completion. Again, Weaver only implement the
reference
result for the time being as this is the most common variation. In this case, the produced file is
stored locally and exposed externally with returned reference URL. The other (unimplemented) mode value
would
return the contents directly in the response instead of the URL.
Note
Other parameters can be added to the request to provide further functionalities. Above fields are the minimum requirements to request a Job. Please refer to the OpenAPI Execute definition for all applicable features.
Note
Since most of the time, returned files are not human readable or are simply too large to be displayed, the
transmissionMode: value
is rarely employed. Also, it is to be noted that outputs representing LiteralData
(which is even more uncommon) would automatically be represented as value
without explicitly requesting it,
as there would not be any file to return. If this poses problem or you encounter a valid use-case where value
would be useful for your needs, please submit a new issue to request the feature.
Execution Steps¶
Once the Job is submitted, its status should initially switch to accepted
. This effectively means that the
Job is pending execution (task queued), but is not yet executing. When a worker retrieves it for execution, the
status will change to started
for preparation steps (i.e.: allocation resources, retrieving required
parametrization details, etc.), followed by running
when effectively reaching the execution step of the underlying
Application Package operation. This status will remain as such until the operation completes, either with
succeeded
or failed
status.
At any moment during async
execution, the Job status can be requested using GET {WEAVER_URL}/processes/{id}/jobs/{id}
(GetStatus). Note that
depending on the timing at which the user executes this request and the availability of task workers, it could be
possible that the Job be already in running
state, or even failed
in case of early problem detected.
When the Job reaches its final state, multiple parameters will be adjusted in the status response to indicate its completion, notably the completed percentage, time it finished execution and full duration. At that moment, the requests for retrieving either error details or produced outputs become accessible. Examples are presented in GetResult section.
Process Operations¶
Todo
detail ‘operations’ accomplished (stage-in, exec-cwl, stage-out)
Workflow Operations¶
Todo
same as prev + ‘operations’ (deploy based on data-source, visibility, exec-remote for each step, pull-result)
File Reference Types¶
Most inputs can be categorized into two of the most commonly employed types, namely LiteralData
and ComplexData
.
The former represents basic values such as integers or strings, while the other represents a file reference.
Files in Weaver (and WPS in general) can be specified with any formats
as MIME-type.
See also
Correspondance between CWL and WPS fields
As for standard WPS, remote file references are usually limited to http(s)
scheme, unless the process
takes an input string and parses the unusual reference from the literal data to process it by itself. On the other hand,
Weaver supports all following reference schemes.
http(s)://
file://
opensearchfile://
[experimental]s3://
[experimental]
The method in which Weaver will handle such references depends on its configuration, in other words, whether it is running as ADES or EMS (see: Configuration), as well as depending on some other CWL package requirements. These use-cases are described below.
Warning
Missing schemes in URL reference are considered identical as if file://
was used. In most cases, if not always,
an execution request should not employ this scheme unless the file is ensured to be at the specific location where
the running Weaver application can find it. This scheme is usually only employed as byproduct of the fetch
operation that Weaver uses to provide the file locally to underlying CWL application package to be
executed.
When Weaver is able to figure out that the process needs to be executed locally in ADES mode, it will fetch all necessary files prior to process execution in order to make them available to the CWL package. When Weaver is in EMS configuration, it will always forward remote references (regardless of scheme) exactly as provided as input of the process execution request, since it assumes it needs to dispatch the execution to another ADES remote server, and therefore only needs to verify that the file reference is reachable remotely. In this case, it becomes the responsibility of this remote instance to handle the reference appropriately. This also avoids potential problems such as if Weaver as EMS doesn’t have authorized access to a link that only the target ADES would have access to.
When CWL package defines WPS1Requirement
under hints
for corresponding WPS-1/2 remote processes being
monitored by Weaver, it will skip fetching of http(s)
-based references since that would otherwise lead to useless
double downloads (one on Weaver and the other on the WPS side). It is the same in case of
ESGF-CWTRequirement
employed for ESGF-CWT processes. Because these processes do not always support S3
buckets, and because Weaver supports many variants of S3 reference formats, it will first fetch the S3
reference using its internal AWS Configuration, and then expose this downloaded file as https(s)
reference
accessible by the remote WPS process.
Note
When Weaver is fetching remote files with http(s)://
, it can take advantage of additional request options to
support unusual or server-specific handling of remote reference as necessary. This could be employed for instance
to attribute access permissions only to some given ADES server by providing additional authorization tokens
to the requests. Please refer to Configuration of Request Options for this matter.
When using S3 references, Weaver will attempt to retrieve the file using server AWS Configuration and AWS Credentials. Provided that the corresponding S3 bucket can be accessed by the running Weaver application, it will fetch the file and store it locally temporarily for CWL execution.
Note
When using S3 buckets, authorization are handled through typical AWS credentials and role permissions. This means that AWS access must be granted to the application in order to allow it fetching the file. There are also different formats of S3 reference formats handled by Weaver. Please refer to Configuration of AWS S3 Buckets for more details.
When using OpenSearch references, additional parameters are necessary to handle retrieval of specific file URL. Please refer to OpenSearch Data Source for more details.
Following table summarize the default behaviour of input file reference handling of different situations when received as input argument of process execution. For simplification, keyword <any> is used to indicate that any other value in the corresponding column can be substituted for a given row when applied with conditions of other columns, which results to same operational behaviour. Elements that behave similarly are also presented together in rows to reduce displayed combinations.
Configuration |
Process Type |
File Scheme |
Applied Operation |
---|---|---|---|
<any> |
<any> |
|
Query and re-process 1 |
ADES |
|
Convert to |
|
|
Nothing (left unmodified) |
||
|
Fetch and convert to |
||
|
Nothing (file already local) |
||
|
Fetch and convert to |
||
|
|||
EMS |
|
Convert to |
|
|
Nothing (left unmodified) |
||
|
Footnotes
- 1
References defined by
opensearch://
will trigger an OpenSearch query using the provided URL as well as other input additional parameters (see OpenSearch Data Source). After processing of this query, retrieved file references will be re-processed using the summarized logic in the table for the given use case.- 2(1,2,3)
When a
file://
(or empty scheme) maps to a local file that needs to be exposed externally for another remote process, the conversion tohttp(s)://
scheme employs settingweaver.wps_outputs_url
to form the result URL reference. The file is placed inweaver.wps_outputs_dir
to expose it as HTTP(S) endpoint.- 3(1,2)
When the process refers to a remote WPS-REST process (i.e.: remote WPS instance that supports REST bindings but that is not necessarily an ADES), Weaver simply wraps and monitor its remote execution, therefore files are handled just as for any other type of remote WPS-like servers. When the process contains an actual CWL Application Package that defines a
CommandLineTool
(including Docker images), files are fetched as it will be executed locally. See CWL CommandLineTool, WPS-REST and Remote Providers for further details.- 4
When an
s3://
file is fetched, is gets downloaded to a temporaryfile://
location, which is NOT necessarily exposed ashttp(s)://
. If execution is transferred to a remove process that is expected to not support S3 references, only then the file gets converted as in 2.- 5
Workflows are only available on EMS instances. Since they chain processes, no fetch is needed as the first sub-step process will do it instead. See Workflow process as well as CWL Workflow for more details.
Todo
method to indicate explicit fetch to override these? (https://github.com/crim-ca/weaver/issues/183)
Todo
add tests that validate each combination of operation
OpenSearch Data Source¶
Todo
EOImage with AOI/TOI/CollectionId for OpenSearch
Multiple Outputs¶
Although CWL allows output arrays, WPS does not support it directly, as only single values are allowed for WPS outputs according to original specification. To work around this, Metalink files can be used to provide a single output reference that embeds other references. This approach is also employed and preferred as described in PyWPS Multiple Outputs.
Todo
fix doc when Multiple Output is supported with metalink (https://github.com/crim-ca/weaver/issues/25)
Todo
add example of multi-output process definition
Todo
and how CWL maps them with WPS
Warning
This feature is being worked on (Weaver Issue #25). Direct support between
See also
Email Notification¶
When submitting a job for execution, it is possible to provide the notification_email
field.
Doing so will tell Weaver to send an email to the specified address with successful or failure details upon job
completion. The format of the email is configurable from weaver.ini.example file with email-specific settings
(see: Configuration).
Monitoring of a process (GetStatus)¶
Todo
job status body example (success vs fail)
Obtaining output results, logs or errors¶
In the case of successful Job execution, the outputs can be retrieved with GET {WEAVER_URL}/processes/{id}/jobs/{id}/result
(GetResult) request to list
each corresponding output id
with the generated file reference URL. Keep in mind that those URL’s purpose are
only to fetch the results (not persistent storage), and could therefore be purged after some reasonable amount of time.
The format should be similar to the following example, with minor variations according to Configurations:
{
"outputs": [
{
"id": "output",
"href": "{WEAVER_URL}/wpsoutputs/f93a15be-6e16-11ea-b667-08002752172a/output_netcdf.nc"
}
]
}
In situations where the Job resulted into failed
status, the GET {WEAVER_URL}/processes/{id}/jobs/{id}/exceptions
(GetLogs) can be use to retrieve
the potential cause of failure, by capturing any raised exception. Below is an example of such exception details.
[
"builtins.Exception: Could not read status document after 5 retries. Giving up."
]
The returned exception are often better understood when compared against, or in conjunction with, the logs that provide details over each step of the operation.
Any Job executed by Weaver will provide minimal information log, such as operation setup, the moment when it started execution and latest status. The extent of other log entries will more often than not depend on the verbosity of the underlying process being executed. When executing an Application Package, Weaver tries as best as possible to collect standard output and error steams to report them through log and exception lists.
Since Weaver can only report as much details as provided by the running application, it is recommended by
Application Package implementers to provide progressive status updates when developing their package
in order to help understand problematic steps in event of process execution failures. In the case of remote WPS
processes monitored by Weaver for example, this means gradually reporting process status updates
(e.g.: calling WPSResponse.update_status
if you are using |pywps|_, see: Progress and Status Report), while using print
and/or logging
operation for scripts or Docker images executed through CWL CommandLineTool
.
Note
Job logs and exceptions are a Weaver-specific implementation. They are not part of traditional OGC API - Processes.
A minimalistic example of logging output is presented below. This can be retrieved using GET {WEAVER_URL}/processes/{id}/jobs/{id}/logs
(GetLogs) request, at any
moment during Job execution (with logs up to that point in time) or after its completion (for full output).
Note again that the more the Process is verbose, the more tracking will be provided here.
[
"[2020-03-24 21:32:32] INFO [weaver.datatype.Job] 0:00:00 1% accepted Job task setup completed.",
"[2020-03-24 21:32:32] INFO [weaver.datatype.Job] 0:00:00 2% accepted Execute WPS request for process [jsonarray2netcdf]",
"[2020-03-24 21:32:33] INFO [weaver.datatype.Job] 0:00:01 4% accepted Fetching job input definitions.",
"[2020-03-24 21:32:33] INFO [weaver.datatype.Job] 0:00:01 6% accepted Fetching job output definitions.",
"[2020-03-24 21:32:33] INFO [weaver.datatype.Job] 0:00:01 8% accepted Starting job process execution",
"[2020-03-24 21:32:34] INFO [weaver.datatype.Job] 0:00:01 10% accepted Verifying job status location.",
"[2020-03-24 21:32:34] WARNING [weaver.datatype.Job] 0:00:01 10% accepted WPS status location could not be found",
"[2020-03-24 21:32:34] INFO [weaver.datatype.Job] 0:00:01 20% running Starting monitoring of job execution.",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:34] INFO [wps_package.jsonarray2netcdf] 1% running Preparing package logs done.",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:34] INFO [wps_package.jsonarray2netcdf] 2% running Launching package...",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:34] INFO [cwltool] Resolved '/tmp/tmpse3pi1gj/jsonarray2netcdf' to 'file:///tmp/tmpse3pi1gj/jsonarray2netcdf'",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:34] INFO [cwltool] ../../../../../tmp/tmpse3pi1gj/jsonarray2netcdf:1:1: Unknown hint",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded file:///tmp/tmpse3pi1gj/BuiltinRequirement",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:34] INFO [wps_package.jsonarray2netcdf] 5% running Loading package content done.",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:34] INFO [wps_package.jsonarray2netcdf] 6% running Retrieve package inputs done.",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:34] INFO [wps_package.jsonarray2netcdf] 8% running Convert package inputs done.",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:34] INFO [wps_package.jsonarray2netcdf] 10% running Running package...",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:34] INFO [cwltool] [job jsonarray2netcdf] /tmp/tmpqy1t8dp3$ python \\",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded /opt/weaver/processes/builtin/jsonarray2netcdf.py \\",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded -o \\",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded /tmp/tmpqy1t8dp3 \\",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded -i \\",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded /tmp/tmpla2utn2c/stgb5787338-4a34-4771-88c0-cae95f4d82dd/test_nc_array.json",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:41] INFO [cwltool] [job jsonarray2netcdf] Max memory used: 36MiB",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:41] INFO [cwltool] [job jsonarray2netcdf] completed success",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:32:41] INFO [wps_package.jsonarray2netcdf] 95% running Package execution done.",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:33:53] INFO [wps_package.jsonarray2netcdf] 98% running Generate package outputs done.",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded [2020-03-24 17:33:55] INFO [wps_package.jsonarray2netcdf] 100% succeeded Package complete.",
"[2020-03-24 21:33:59] INFO [weaver.datatype.Job] 0:01:26 90% succeeded Job succeeded (status: Package complete.).",
"[2020-03-24 21:34:45] INFO [weaver.datatype.Job] 0:01:26 90% succeeded Job succeeded.",
"[2020-03-24 21:34:45] INFO [weaver.datatype.Job] 0:01:26 100% succeeded Job task complete."
]
Special Weaver EMS use-cases¶
This section highlight the additional behaviour available only through an EMS-configured Weaver instance. Some other points are already described in other sections, but are briefly indicated here for conciseness.
ADES dispatching using Data-Sources¶
Todo
add details, data-source defines where to send request of known ADES
Todo
reference config weaver.data_sources
Application Package¶
The Application Package defines the internal script definition and configuration that will be executed by a Process. This package is based on Common Workflow Language (CWL). Using the extensive CWL Specification as backbone for internal execution of the process allows it to run multiple type of applications, whether they are referenced to by Docker image, scripts (bash, python, etc.), some remote Process and more.
Note
The large community and use cases covered by CWL makes it extremely versatile. If you encounter any issue running your Application Package in Weaver (such as file permissions for example), chances are that there exists a workaround somewhere in the CWL Specification. Most typical problems are usually handled by some flag or argument in the CWL definition, so this reference should be explored first. Please also refer to FAQ section as well as existing Weaver issues. Ultimately if no solution can be found, open an new issue about your specific problem.
All processes deployed locally into Weaver using a CWL package definition will have their full package
definition available with GET {WEAVER_URL}/processes/{id}/package
(Package) request.
Note
The package request is a Weaver-specific implementation, and therefore, is not necessarily available on other ADES/EMS implementation as this feature is not part of OGC API - Processes specification.
Typical CWL Package Definition¶
CWL CommandLineTool¶
Following CWL package definition represents the weaver.processes.builtin.jsonarray2netcdf
process.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | #!/usr/bin/env cwl-runner
cwlVersion: v1.0
class: CommandLineTool
# target the installed python pointing to weaver conda env to allow imports
baseCommand: python
arguments:
- "${WEAVER_ROOT_DIR}/weaver/processes/builtin/jsonarray2netcdf.py"
- "-o"
- "$(runtime.outdir)"
inputs:
input:
type: File
format: iana:application/json
inputBinding:
position: 1
prefix: "-i"
outputs:
output:
format: edam:format_3650
type:
type: array
items: File
outputBinding:
glob: "*.nc"
$namespaces:
iana: "https://www.iana.org/assignments/media-types/"
edam: "http://edamontology.org/"
|
The first main components is the class: CommandLineTool
that tells Weaver it will be a base process
(contrarily to CWL Workflow presented later).
The other important sections are inputs
and outputs
. These define which parameters will be expected and
produced by the described application. Weaver supports most formats and types as specified by CWL Specification.
See Inputs/Outputs Type for more details.
CWL Workflow¶
Weaver also supports CWL class: Workflow
. When an Application Package is defined this way, the
Process deployment operation will attempt to resolve each step
as another process. The reference to the CWL
definition can be placed in any location supported as for the case of atomic processes
(see details about supported package locations).
The following CWL definition demonstrates an example Workflow
process that would resolve each step
with
local processes of match IDs.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | {
"cwlVersion": "v1.0",
"class": "Workflow",
"requirements": [
{
"class": "StepInputExpressionRequirement"
}
],
"inputs": {
"tasmax": {
"type": {
"type": "array",
"items": "File"
}
},
"lat0": "float",
"lat1": "float",
"lon0": "float",
"lon1": "float",
"freq": {
"default": "YS",
"type": {
"type": "enum",
"symbols": ["YS", "MS", "QS-DEC", "AS-JUL"]
}
}
},
"outputs": {
"output": {
"type": "File",
"outputSource": "ice_days/output_netcdf"
}
},
"steps": {
"subset": {
"run": "ColibriFlyingpigeon_SubsetBbox.cwl",
"in": {
"resource": "tasmax",
"lat0": "lat0",
"lat1": "lat1",
"lon0": "lon0",
"lon1": "lon1"
},
"out": ["output"]
},
"json2nc": {
"run": "jsonarray2netcdf",
"in": {
"input": "subset/output"
},
"out": ["output"]
},
"ice_days": {
"run": "Finch_IceDays.cwl",
"in": {
"tasmax": "json2nc/output",
"freq": "freq"
},
"out": ["output_netcdf"]
}
}
}
|
For instance, the jsonarray2netcdf
(Builtin) middle step in this example corresponds to the
CWL CommandLineTool process presented in previous section. Other processes referenced in this Workflow
can be
found in Weaver Test Resources.
Steps processes names are resolved using the variations presented below. Important care also needs to be given to inputs and outputs definitions between each step.
Step Reference¶
In order to resolve referenced processes as steps, Weaver supports 3 formats.
- Process ID explicitly given.Any visible process from
GET {WEAVER_URL}/processes
(GetCapabilities) response should be resolved this way.(e.g.:jsonarray2netcdf
resolves to pre-deployedweaver.processes.builtin.jsonarray2netcdf
). Full URL to the process description endpoint, provided that it also offers a
GET {WEAVER_URL}/processes/{id}/package
(Package) endpoint (Weaver-specific).Full URL to the explicit CWL file (usually corresponding to (2) or the
href
provided in deployment body).
When an URL to the CWL process “file” is provided with an extension, it must be one of the supported values
defined in weaver.processes.wps_package.PACKAGE_EXTENSIONS
. Otherwise, Weaver will refuse it as it cannot
figure out how to parse it.
Because Weaver and the underlying CWL executor need to resolve all steps in order to validate their input and
output definitions correspond (id, format, type, etc.) in order to chain them, all intermediate processes MUST
be available. This means that you cannot Register a new process (Deploy) nor Execution of a process (Execute) a Workflow
-flavored
Application Package until all referenced steps have themselves been deployed and made visible.
Warning
Because Weaver needs to convert given CWL documents into equivalent WPS process definition,
embedded CWL processes within a Workflow
step are not supported currently. This is a known limitation
of the implementation, but not much can be done against it without major modifications to the code base.
See also issue #56.
See also
Step Inputs/Outputs¶
Inputs and outputs of connected steps are required to match types and formats in order for the workflow to be valid.
This means that a process that produces an output of type String
cannot be directly chained to a process that takes
as input a File
, even if the String
of the first process represents an URL that could be resolved to a valid
file reference. In order to chain two such processes, an intermediate operation would need to be defined to explicitly
convert the String
input to the corresponding File
output. This is usually accomplished using Builtin
processes, such as in the previous example.
Since formats must also match (e.g.: a process producing application/json
cannot be mapped to one producing
application/x-netcdf
), all mismatching formats must also be converted with an intermediate step if such operation
is desired. This ensures that workflow definitions are always explicit and that as little interpretation, variation or
assumptions are possible between each execution. Because of this, all application generated by Weaver will attempt to
preserve and enforce matching input/output format
definition in both CWL and WPS as long as it does
not introduce ambiguous results (see File Format for more details).
Correspondence between CWL and WPS fields¶
Because CWL definition and WPS process description inherently provide “duplicate” information, many fields can be mapped between one another. In order to handle any provided metadata in the various supported locations by both specifications, as well as to extend details of deployed processes, each Application Package get its details merged with complementary WPS description.
In some cases, complementary details are only documentation-related, but some information directly affect the format or
execution behaviour of some parameters. A common example is the maxOccurs
field provided by WPS that does
not have an exactly corresponding specification in CWL (any-sized array). On the other hand, CWL also
provides data preparation steps such as initial staging (i.e.: InitialWorkDirRequirement
) that doesn’t have an
equivalent under the WPS process description. For this reason, complementary details are merged and reflected
on both sides (as applicable), when non-ambiguous resolution is possible.
In case of conflicting metadata, the CWL specification will most of the time prevail over the WPS
metadata fields simply because it is expected that a strict CWL specification is provided upon deployment.
The only exceptions to this situation are when WPS specification help resolve some ambiguity or when
WPS enforces the parametrisation of some elements, such as with maxOccurs
field.
Note
Metadata merge operation between CWL and WPS is accomplished on per-mapped-field basis. In other
words, more explicit details such as maxOccurs
could be obtained from WPS and simultaneously the
same input’s format
could be obtained from the CWL side. Merge occurs bidirectionally for corresponding
information.
The merging strategy of process specifications also implies that some details can be omitted from one context if they
can be inferred from corresponding elements in the other. For example, the CWL and WPS context both
define keywords
(with minor naming variation) as a list of strings. Specifying this metadata in both locations
is redundant and only makes the process description longer. Therefore, the user is allowed to provide only one of the
two and Weaver will take care to propagate the information to the lacking location.
In order to help understand the resolution methodology between the contexts, following sub-section will cover supported mapping between the two specifications, and more specifically, how each field impacts the mapped equivalent metadata.
Warning
Merging of corresponding fields between CWL and WPS is a Weaver-specific implementation. The same behaviour is not necessarily supported by other implementations. For this reason, any converted information between the two contexts will be transferred to the other context if missing in order for both specification to reflect the similar details as closely as possible, wherever context the metadata originated from.
Inputs/Outputs ID¶
Inputs and outputs (I/O) id
from the CWL context will be respectively matched against corresponding
id
or identifier
field from I/O of WPS context. In the CWL definition, all of the allowed I/O
structures are supported, whether they are specified using an array list with explicit definitions, using “shortcut”
variant, or using key-value pairs (see CWL Mapping for more details). Regardless of array or mapping format,
CWL requires that all I/O have unique id
. On the WPS side, a list of I/O is always expected.
This is because WPS I/O with multiple values (array in CWL) are specified by repeating the id
with
each value instead of defining the value as a list of those values during Execution of a process (Execute) request (see also
Multiple Inputs).
To summarize, the following CWL and WPS I/O definitions are all equivalent and will result into the
same process definition after deployment. For simplification purpose, below examples omit all but mandatory fields
(only of the inputs
and outputs
portion of the full deployment body) to produce the same result.
Other fields are discussed afterward in specific sections.
CWL I/O objects array¶
|
CWL I/O key-value mapping¶
|
WPS I/O definition¶
|
The WPS example above requires a format
field for the corresponding CWL File
type in order to
distinguish it from a plain string. More details are available in Inputs/Outputs Type below about this requirement.
Finally, it is to be noted that above CWL and WPS definitions can be specified in the Register a new process (Deploy) request body with any of the following variations:
Both are simultaneously fully specified (valid although extremely verbose).
Both partially specified as long as sufficient complementary information is provided.
Only CWL I/O is fully provided (with empty or even unspecified
inputs
oroutputs
section from WPS).
Warning
Weaver assumes that its main purpose is to eventually execute an Application Package and will therefore
prioritize specification in CWL over WPS. Because of this, any unmatched id
from the WPS
context against provided CWL id
s of the same I/O section will be dropped, as they ultimately would
have no purpose during CWL execution.
This does not apply in the case of referenced WPS-1/2 processes since no CWL is available in the first place.
Inputs/Outputs Type¶
In the CWL context, the type
field indicates the type of I/O. Available types are presented in the
CWLType Symbols portion of the specification.
Warning
Weaver has two unsupported CWL type
, namely Any
and Directory
. This limitation is
intentional as WPS does not offer equivalents. Furthermore, both of these types make the process
description too ambiguous. For instance, most processes expect remote file references, and providing a
Directory
doesn’t indicate an explicit reference to which files to retrieve during stage-in operation of
a job execution.
In the WPS context, three data types exist, namely Literal
, BoundingBox
and Complex
data.
As presented in the example of the previous section, I/O in the WPS context does not require an explicit
indication of the type from one of Literal
, BoundingBox
and Complex
data. Instead, WPS type is
inferred using the matched API schema of the I/O. For instance, Complex
I/O (i.e.: file reference) requires the
formats
field to distinguish it from a plain string
. Therefore, specifying either format
in CWL
or formats
in WPS immediately provides all needed information for Weaver to understand that this I/O is
expected to be a file reference. A crs
field would otherwise indicate a BoundingBox
I/O
(see note). If none of the two previous schemas are matched, the I/O type resolution falls back
to Literal
data of string
type. To employ another primitive data type such as Integer
, an explicit
indication needs to be provided as follows.
1 2 3 4 5 6 | {
"id": "input",
"literalDataDomains": [
{"dataType": {"name": "integer"}}
]
}
|
Obviously, the equivalent CWL definition is simpler in this case (i.e.: only type: int
is required).
It is therefore recommended to take advantage of Weaver’s merging strategy in this case by providing only the
details through the CWL definition and have the corresponding WPS I/O type automatically deduced by
the generated process. If desired, literalDataDomains
can still be explicitly provided as above to ensure that
it gets parsed as intended type.
Note
As of the current version of Weaver, WPS data type BoundingBox
is not supported. The schema definition
exists in WPS context but is not handled by any CWL type conversion yet. This feature is reflected
by issue #51. It is possible to use a Literal
data of
type string
corresponding to WKT 1, 2 in the meantime.
File Format¶
An input or output resolved as CWL File
type, equivalent to a WPS ComplexData
, supports
format
specification. Every mimeType
field nested under formats
entries of the WPS definition
will be mapped against corresponding namespaced format
of CWL.
For example, the following input definitions are equivalent in both contexts.
WPS Format with MIME-type¶
|
CWL Format with Namespace¶
|
As demonstrated, both contexts accept multiple formats for inputs. These effectively represent supported formats by
the underlying application. The two MIME-types selected for this example are chosen specifically to demonstrate
how CWL formats must be specified. More precisely, CWL requires a real schema definition referencing to
an existing ontology to validate formats, specified through the $namespaces
section. Each format entry is then
defined as a mapping of the appropriate namespace to the identifier of the ontology. Alternatively, you can also provide
the full URL of the ontology reference in the format string.
Like many other fields, this information can become quite rapidly redundant and difficult to maintain. For this reason,
Weaver will automatically fill the missing detail if only one of the two corresponding information between CWL
and WPS is provided. In other words, an application developer could only specify the I/O’s formats
in the WPS portion during process deployment, and Weaver will take care to update the matching CWL
definition without any user intervention. This makes it also easier for the user to specify supported formats since it
is generally easier to remember names of MIME-types than full ontology references. Weaver has a large set of
commonly employed MIME-types that it knows how to convert to corresponding ontologies. Also, Weaver will look
for new MIME-types it doesn’t explicitly know about onto either the IANA or the EDAM ontologies
in order to attempt automatically resolving them.
When formats are resolved between the two contexts, Weaver applies information in a complimentary fashion. This means
for example that if the user provided application/x-netcdf
on the WPS side and iana:application/json
on
the CWL side, both resulting contexts will have both of those formats combined. Weaver will not favour one
location over the other, but will rather merge them if they can be resolved into different and valid entities.
Since format
is a required field for WPS ComplexData
definitions (see Inputs/Outputs Type) and
that MIME-types are easier to provide in this context, it is recommended to provide all of them in the
WPS definition.
Output File Format¶
Although WPS definition allows multiple supported formats for output that are later resolved to the applied
one onto the produced result of the job, CWL only considers the output format
that directly indicates the
applied schema. There is no concept of supported format in the CWL world. This is simply because CWL
cannot predict nor reliably determine which output will be produced by a given application execution without running it,
and therefore cannot expose consistent output specification before running the process. Because CWL requires to
validate the full process integrity before it can be executed, this means that only a single output format is
permitted in its context (providing many will raise a validation error when parsing the CWL definition).
To ensure compatibility with multiple supported formats outputs of WPS, any output that has more that one
format will have its format
field dropped in the corresponding CWL definition. Without any format
on
the CWL side, the validation process will ignore this specification and will effectively accept any type of
file. This will not break any execution operation with CWL, but it will remove the additional validation layer
of the format (which especially deteriorates process resolution when chaining processes inside a CWL Workflow).
If the WPS output only specifies a single MIME-type, then the equivalent format (after being resolved to a valid ontology) will be preserved on the CWL side since the result is ensured to be the unique one provided. For this reason, processes with specific single-format output are be preferred whenever possible. This also removes ambiguity in the expected output format, which usually requires a toggle input specifying the desired type for processes providing a multi-format output. It is instead recommended to produce multiple processes with a fixed output format for each case.
Allowed Values¶
Allowed values in the context of WPS LiteralData
provides a mean for the application developer to restrict
inputs to a specific set of values. In CWL, the same can be achieved using an enum
definition. Therefore,
the following two variants are equivalent and completely interchangeable.
WPS AllowedValues Input¶
|
CWL Enum Values¶
|
Weaver will ensure to propagate such definitions bidirectionally in order to update the CWL or WPS
correspondingly with the provided information in the other context if missing. The primitive type to apply to a missing
WPS specification when resolving it from a CWL definition is automatically inferred with the best
matching type from provided values in the enum
list.
Note that enum
such as these will also be applied on top of Multiple and Optional Values definitions
presented next.
Multiple and Optional Values¶
Inputs that take multiple values or references can be specified using minOccurs
and maxOccurs
in WPS
context, while they are specified using the array
type in CWL. While the same minOccurs
parameter with a
value of zero (0
) can be employed to indicate an optional input, CWL requires the type to specify null
or to use the shortcut ?
character suffixed to the base type to indicate optional input. Resolution between
WPS and CWL for the merging strategy implies all corresponding parameter combinations and checks in
this case.
Because CWL does not take an explicit amount of maximum occurrences, information in this case are not
necessarily completely interchangeable. In fact, WPS is slightly more verbose and easier to define in this case
than CWL because all details are contained within the same two parameters. Because of this, it is often
preferable to provide the minOccurs
and maxOccurs
in the WPS context, and let Weaver infer the
array
and/or null
type requirements automatically. Also, because of all implied parameters in this situation to
specify the similar details, it is important to avoid providing contradicting specifications as Weaver will have
trouble guessing the intended result when merging specifications. If unambiguous guess can be made, CWL will be
employed as deciding definition to resolve erroneous mismatches (as for any other corresponding fields).
Todo
update warning according to Weaver issue #25
Warning
Parameters minOccurs
and maxOccurs
are not permitted for outputs in the WPS context. Native
WPS therefore does not permit multiple output reference files. This can be worked around using a
Metalink file, but this use case is not covered by Weaver yet as it requires special mapping with CWL
that does support array
type as output (see issue #25).
Note
Although WPS multi-value inputs are defined as a single entity during deployment, special care must be taken to the format in which to specify these values during execution. Please refer to Multiple Inputs section of Execution of a process (Execute) request.
Following are a few examples of equivalent WPS and CWL definitions to represent multiple values under a given input. Some parts of the following definitions are purposely omitted to better highlight the concise details of multiple and optional information.
WPS Multi-Value Input (required)¶
|
CWL Multi-Value Input (required)¶
|
Todo
minOccurs/maxOccurs + array + WPS repeats IDs vs CWL as list
Todo
example multi-value + enum
It can be noted from the examples that minOccurs
and maxOccurs
can be either an integer
or a string
representing one. This is to support backward compatibility of older WPS specification that always employed
strings although representing numbers. Weaver understands and handles both cases. Also, maxOccurs
can have the
special string value "unbounded"
, in which case the input is considered to be allowed an unlimited amount if
entries (although often capped by another implicit machine-level limitation such as memory capacity). In the case of
CWL, an array
is always considered as unbounded, therefore WPS is the only context that can limit
this amount.
Metadata¶
Todo
(s:)keywords field, doc/label vs abstract/title per-I/O and overall process, etc?
Example: cwl-metadata
Process and Application Package Examples¶
The principal source of examples can be found directly within Weaver functional tests. Most Process listed
in this repository employ an Application Package defined using a locally provided CWL. Some other
cases will refer to pre-deployed WPS processes. Refer to corresponding Deploy request payloads,
also provided in that location, that will indicate the kind of Process definition employed through their
executionUnit
specification. Once successfully deployed on a ADES or EMS, the also provided
Execute request body can be employed as format reference to run the operation (input values can be modified).
The general naming convention is:
DeployProcess_<PROCESS_ID>.json
for the Deploy request payload.
Execute_<PROCESS_ID>.json
for the Execute request payload.
<PROCESS_ID>.cwl
for the CWL Application Package when applicable.
Note
There can be minor variations (camel/snake case, upper/lower case) of the exact <PROCESS_ID>
employed in the
file names according to the different reference repositories.
Further examples are also available following OGC Testbeds developments. The produced applications in this case can be found in OGC-Testbeds Applications. In that repository, the above naming convention will sometime be employed, but each Process is contained within its own sub-directory. Another naming conversion is sometime used, although the files provide equivalent details:
<PROCESS_ID>/deploy.json
for the Deploy request payload.
<PROCESS_ID>/execute.json
for the Execute request payload.
<PROCESS_ID>/package.cwl
for the CWL Application Package when applicable.
OpenAPI Specification¶
API Reference¶
This page contains auto-generated API reference documentation 1.
weaver
¶
Subpackages¶
weaver.database
¶
weaver.database.mongodb
¶-
class
weaver.database.mongodb.
MongoDatabase
(: AnySettingsContainer, registry: bool, reset_connection=False)[source]¶ Return the unique identifier of db type matching settings.
Initialize self. See help(type(self)) for accurate signature.
-
get_store
(self: Union[AnyStr, StoreInterface, MongodbStores], store_type: Any, *store_args: Any, **store_kwargs) → AnyStoreType[source]¶ Retrieve a store from the database.
- Parameters
store_type – type of the store to retrieve/create.
store_args – additional arguments to pass down to the store.
store_kwargs – additional keyword arguments to pass down to the store.
-
weaver.processes
¶
weaver.processes.builtin
¶weaver.processes.builtin.file2string_array
¶-
weaver.processes.builtin.file2string_array.
main
(input_file: argparse.FileType, output_dir: AnyStr) → None[source]¶
weaver.processes.builtin.jsonarray2netcdf
¶Extracts and fetches NetCDF files from a JSON file containing an URL string array, and provides them on the output directory.
weaver.processes.builtin.metalink2netcdf
¶Extracts and fetches NetCDF files from a Metalink file containing an URL, and outputs the NetCDF file at a given index of the list.
-
weaver.processes.builtin.
register_builtin_processes
(container: AnyDatabaseContainer) → None[source]¶ Registers every
builtin
CWL package to the processes database.CWL definitions must be located within the
weaver.processes.builtin
module.
weaver.processes.constants
¶weaver.processes.esgf_process
¶-
class
weaver.processes.esgf_process.
ESGFProcess
(provider: AnyStr, process: AnyStr, request: WPSRequest, update_status: UpdateStatusPartialFunction)[source]¶ Common interface for WpsProcess to be used is cwl jobs
Initialize self. See help(type(self)) for accurate signature.
-
execute
(self: JSON, workflow_inputs: AnyStr, out_dir: Dict[AnyStr, AnyStr], expected_outputs) → None[source]¶ Execute an ESGF process from cwl inputs
-
_prepare_inputs
(self: JSON, workflow_inputs) → List[cwt.Variable][source]¶ Convert inputs from cwl inputs to ESGF format
-
static
_get_files_urls
(workflow_inputs: JSON) → List[Tuple[str, str]][source]¶ Get all netcdf files from the cwl inputs
-
_run_process
(self: str, api_key: List[cwt.Variable], inputs: Optional[cwt.Domain], domain=None) → cwt.Process[source]¶ Run an ESGF process
-
_wait
(self: cwt.Process, esgf_process: float, sleep_time=2) → bool[source]¶ Wait for an ESGF process to finish, while reporting its status
-
weaver.processes.opensearch
¶-
weaver.processes.opensearch.
alter_payload_after_query
(payload)[source]¶ When redeploying the package on ADES, strip out any EOImage parameter
- Parameters
payload –
-
weaver.processes.opensearch.
query_eo_images_from_wps_inputs
(wps_inputs: Dict[str, Deque], eoimage_source_info: Dict[str, Dict], accept_mime_types: Dict[str, List[str]], settings: Optional[AnySettingsContainer] = None) → Dict[str, Deque][source]¶ Query OpenSearch using parameters in inputs and return file links.
eoimage_ids is used to identify if a certain input is an eoimage.
- Parameters
wps_inputs – inputs containing info to query
eoimage_source_info – data source info of eoimages
accept_mime_types – dict of list of accepted mime types, ordered by preference
settings – application settings to retrieve request options as necessary.
-
weaver.processes.opensearch.
replace_with_opensearch_scheme
(link)[source]¶ - Parameters
link – url to replace scheme
-
weaver.processes.opensearch.
load_wkt
(wkt)[source]¶ - Parameters
wkt (string) – to get the bounding box of
-
class
weaver.processes.opensearch.
OpenSearchQuery
(collection_identifier: str, osdd_url: str, catalog_search_field: str = 'parentIdentifier', settings: Optional[AnySettingsContainer] = None)[source]¶ - Parameters
collection_identifier – Collection ID to query
osdd_url – Global OSDD url for opensearch queries.
catalog_search_field – Name of the field for the collection identifier.
settings – application settings to retrieve request options as necessary.
-
_prepare_query_url
(self: str, template_url: Dict, params) → Tuple[str, Dict][source]¶ - Parameters
template_url – url containing query parameters
params – parameters to insert in formatted url
-
_query_features_paginated
(self: Dict, params) → Iterable[Dict, str][source]¶ - Parameters
params – query parameters
-
query_datasets
(self: Dict, params: Tuple, accept_schemes: List, accept_mime_types) → Iterable[AnyStr][source]¶ Loop on every opensearch result feature and yield url matching required mime-type and scheme. Log a warning if a feature cannot yield a valid url (either no compatible mime-type or scheme)
- Parameters
params – query parameters
accept_schemes – only return links of this scheme
accept_mime_types – list of accepted mime types, ordered by preference
- Raises
KeyError – If the feature doesn’t contain a json data section or an atom alternative link
-
weaver.processes.opensearch.
get_additional_parameters
(input_data: Dict) → List[Tuple[str, str]][source]¶ - Parameters
input_data – Dict containing or not the “additionalParameters” key
-
class
weaver.processes.opensearch.
EOImageDescribeProcessHandler
(: List[Dict], inputs)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
weaver.processes.opensearch.
get_eo_images_inputs_from_payload
(payload)[source]¶ - Parameters
payload –
-
weaver.processes.opensearch.
get_original_collection_id
(payload: Dict, wps_inputs: Dict[str, deque]) → Dict[str, deque][source]¶ When we deploy a Process that contains OpenSearch parameters, the collection identifier is modified. Ex: files -> collection Ex: s2 -> collection_s2, probav -> collection_probav This function changes the id in the execute request to the one in the deploy description. :param payload: :param wps_inputs: :return:
-
weaver.processes.opensearch.
get_eo_images_data_sources
(payload: Dict, wps_inputs: Dict[str, deque]) → Dict[str, Dict][source]¶ - Parameters
payload – Deploy payload
wps_inputs – Execute inputs
-
weaver.processes.opensearch.
get_eo_images_mime_types
(payload: Dict) → Dict[str, List][source]¶ From the deploy payload, get the accepted mime types. :param payload: Deploy payload
-
weaver.processes.opensearch.
insert_max_occurs
(payload: Dict, wps_inputs: Dict[str, Deque]) → None[source]¶ Insert maxOccurs value in wps inputs using the deploy payload. :param payload: Deploy payload :param wps_inputs: WPS inputs
weaver.processes.sources
¶-
weaver.processes.sources.
DATA_SOURCES
[source]¶ Data sources configuration.
Unless explicitly overridden, the configuration will be loaded from file as specified by
weaver.data_sources
setting. Following schema format is expected:{ "$schema": "http://json-schema.org/draft-07/schema#", "title": "Data Sources", "type": "object", "patternProperties": { ".*": { "type": "object", "required": [ "netloc", "ades" ], "additionalProperties": false, "properties": { "netloc": { "type": "string", "description": "Net location of a data source url use to match this data source." }, "ades": { "type": "string", "description": "ADES endpoint where the processing of this data source can occur." }, "default": { "type": "string", "description": "True indicate that if no data source match this one should be used (Use the first default)." } } } } }
weaver.processes.types
¶weaver.processes.utils
¶-
weaver.processes.utils.
_get_data
(output: owslib.wps.Output) → Optional[Any][source]¶ Extract the data from the output value.
-
weaver.processes.utils.
_read_reference
(url: AnyStr) → Optional[AnyStr][source]¶ Read a reference HTTP(S) URL and return the content.
-
weaver.processes.utils.
_get_multi_json_references
(output: owslib.wps.Output, container: Optional[AnySettingsContainer]) → Optional[List[JSON]][source]¶ Since WPS standard does not allow to return multiple values for a single output, a lot of process actually return a json array containing references to these outputs.
Because the multi-output references are contained within this JSON file, it is not very convenient to retrieve the list of URLs as one always needs to open and read the file to get them. This function goal is to detect this particular format and expand the references to make them quickly available in the job output response.
- Returns
Array of HTTP(S) references if the specified output is effectively a JSON containing that,
None
otherwise.
-
weaver.processes.utils.
map_progress
(progress: Number, range_min: Number, range_max: Number) → Number[source]¶ Calculates the relative progression of the percentage process within min/max values.
-
weaver.processes.utils.
jsonify_output
(output: owslib.wps.Output, process_description: owslib.wps.Process, container: Optional[AnySettingsContainer] = None) → JSON[source]¶ Utility method to jsonify an output element from a WPS1 process description.
In the case that a reference JSON output is specified and that it refers to a file that contains an array list of URL references to simulate a multiple-output, this specific output gets expanded to contain both the original URL
reference
field and the loaded URL list underdata
field for easier access from the response body.
-
weaver.processes.utils.
convert_process_wps_to_db
(service: Service, process: ProcessWPS, container: AnySettingsContainer) → ProcessDB[source]¶ Converts an owslib WPS Process to local storage Process.
-
weaver.processes.utils.
_check_deploy
(payload)[source]¶ Validate minimum deploy payload field requirements with exception handling.
-
weaver.processes.utils.
_get_deploy_process_info
(process_info, reference, package)[source]¶ Obtain the process definition from deploy payload with exception handling.
-
weaver.processes.utils.
deploy_process_from_payload
(payload: JSON, container: AnyContainer) → HTTPException[source]¶ Adds a
weaver.datatype.Process
instance to storage using the provided JSONpayload
matchingweaver.wps_restapi.swagger_definitions.ProcessDescription
.- Returns
HTTPOk if the process registration was successful
- Raises
HTTPException – otherwise
-
weaver.processes.utils.
parse_wps_process_config
(config_entry: Union[JSON, AnyStr]) → Tuple[AnyStr, AnyStr, List[AnyStr], bool][source]¶ Parses the available WPS provider or process entry to retrieve its relevant information.
- Returns
WPS provider name, WPS service URL, and list of process identifier(s).
- Raises
ValueError – if the entry cannot be parsed correctly.
-
weaver.processes.utils.
register_wps_processes_from_config
(wps_processes_file_path: Optional[FileSystemPathType], container: AnySettingsContainer) → None[source]¶ Loads a wps_processes.yml file and registers WPS-1 providers processes to the current Weaver instance as equivalent WPS-2 processes.
References listed under
processes
are registered. When the reference is a service (provider), registration of each WPS process is done individually for each of the specified providers with ID[service]_[process]
per listed process byGetCapabilities
.New in version 1.14.0: When references are specified using
providers
section instead ofprocesses
, the registration only saves the remote WPS provider endpoint to dynamically populate WPS processes on demand.See also
weaver.wps_processes.yml.example for additional file format details
weaver.processes.wps1_process
¶-
class
weaver.processes.wps1_process.
Wps1Process
(provider: AnyStr, process: AnyStr, request: WPSRequest, update_status: UpdateStatusPartialFunction)[source]¶ Common interface for WpsProcess to be used is cwl jobs
Initialize self. See help(type(self)) for accurate signature.
-
execute
(self, workflow_inputs, out_dir, expected_outputs)[source]¶ Execute a remote process using the given inputs. The function is expected to monitor the process and update the status. Retrieve the expected outputs and store them in the
out_dir
.- Parameters
workflow_inputs – CWL job dict
out_dir – directory where the outputs must be written
expected_outputs – expected value outputs as {‘id’: ‘value’}
-
weaver.processes.wps3_process
¶-
class
weaver.processes.wps3_process.
Wps3Process
(step_payload: JSON, joborder: JSON, process: AnyStr, request: WPSRequest, update_status: UpdateStatusPartialFunction)[source]¶ Common interface for WpsProcess to be used is cwl jobs
Initialize self. See help(type(self)) for accurate signature.
-
is_visible
(self) → Union[bool, None][source]¶ Gets the process visibility.
- Returns
True/False correspondingly for public/private if visibility is retrievable, False if authorized access but process cannot be found, None if forbidden access.
-
execute
(self, workflow_inputs, out_dir, expected_outputs)[source]¶ Execute a remote process using the given inputs. The function is expected to monitor the process and update the status. Retrieve the expected outputs and store them in the
out_dir
.- Parameters
workflow_inputs – CWL job dict
out_dir – directory where the outputs must be written
expected_outputs – expected value outputs as {‘id’: ‘value’}
-
weaver.processes.wps_default
¶-
class
weaver.processes.wps_default.
HelloWPS
(*args, **kwargs)[source]¶ - Parameters
handler – A callable that gets invoked for each incoming request. It should accept a single
pywps.app.WPSRequest
argument and return apywps.app.WPSResponse
object.identifier (string) – Name of this process.
title (string) – Human readable title of process.
abstract (string) – Brief narrative description of the process.
keywords (list) – Keywords that characterize a process.
inputs – List of inputs accepted by this process. They should be
LiteralInput
andComplexInput
andBoundingBoxInput
objects.outputs – List of outputs returned by this process. They should be
LiteralOutput
andComplexOutput
andBoundingBoxOutput
objects.metadata – List of metadata advertised by this process. They should be
pywps.app.Common.Metadata
objects.translations (dict[str,dict[str,str]]) – The first key is the RFC 4646 language code, and the nested mapping contains translated strings accessible by a string property. e.g. {“fr-CA”: {“title”: “Mon titre”, “abstract”: “Une description”}}
Initialize self. See help(type(self)) for accurate signature.
weaver.processes.wps_package
¶Functions and classes that offer interoperability and conversion between corresponding elements defined as CWL CommandLineTool/Workflow and WPS ProcessDescription in order to generate ADES/EMS Application Package.
See also
weaver.wps_restapi.api
conformance details
-
weaver.processes.wps_package.
retrieve_package_job_log
(execution: WPSExecution, job: Job) → None[source]¶ Obtains the underlying WPS execution log from the status file to add them after existing job log entries.
-
weaver.processes.wps_package.
get_process_location
(process_id_or_url: Union[Dict[AnyStr, Any], AnyStr], data_source: Optional[AnyStr] = None) → AnyStr[source]¶ Obtains the URL of a WPS REST DescribeProcess given the specified information.
- Parameters
process_id_or_url – process “identifier” or literal URL to DescribeProcess WPS-REST location.
data_source – identifier of the data source to map to specific ADES, or map to localhost if
None
.
- Returns
URL of EMS or ADES WPS-REST DescribeProcess.
-
weaver.processes.wps_package.
get_package_workflow_steps
(package_dict_or_url: Union[Dict[AnyStr, Any], AnyStr]) → List[Dict[AnyStr, AnyStr]][source]¶ - Parameters
package_dict_or_url – process package definition or literal URL to DescribeProcess WPS-REST location.
- Returns
list of workflow steps as {“name”: <name>, “reference”: <reference>} where name is the generic package step name, and reference is the id/url of a registered WPS package.
-
weaver.processes.wps_package.
complex2json
(data: Union[ComplexData, Any]) → Union[JSON, Any][source]¶ Obtains the JSON representation of a
ComplexData
or simply return the unmatched type.
-
weaver.processes.wps_package.
metadata2json
(meta: Union[ANY_Metadata_Type, Any], force: bool = False) → Union[JSON, Any][source]¶ Obtains the JSON representation of a
OwsMetadata
orpywps.app.Common.Metadata
. Otherwise, simply return the unmatched type. If requested, can enforce parsing a dictionary for the corresponding keys.
-
weaver.processes.wps_package.
get_process_definition
(process_offering: JSON, reference: Optional[AnyStr] = None, package: Optional[CWL] = None, data_source: Optional[AnyStr] = None) → JSON[source]¶ Returns an updated process definition dictionary ready for storage using provided WPS
process_offering
and a package definition passed byreference
orpackage
CWL content. The returned process information can be used later on to load an instance ofweaver.wps_package.WpsPackage
.- Parameters
process_offering – WPS REST-API (WPS-3) process offering as JSON.
reference – URL to CWL package definition, WPS-1 DescribeProcess endpoint or WPS-3 Process endpoint.
package – literal CWL package definition (YAML or JSON format).
data_source – where to resolve process IDs (default: localhost if
None
).
- Returns
updated process definition with resolved/merged information from
package
/reference
.
-
class
weaver.processes.wps_package.
WpsPackage
(**kw)[source]¶ - Parameters
handler – A callable that gets invoked for each incoming request. It should accept a single
pywps.app.WPSRequest
argument and return apywps.app.WPSResponse
object.identifier (string) – Name of this process.
title (string) – Human readable title of process.
abstract (string) – Brief narrative description of the process.
keywords (list) – Keywords that characterize a process.
inputs – List of inputs accepted by this process. They should be
LiteralInput
andComplexInput
andBoundingBoxInput
objects.outputs – List of outputs returned by this process. They should be
LiteralOutput
andComplexOutput
andBoundingBoxOutput
objects.metadata – List of metadata advertised by this process. They should be
pywps.app.Common.Metadata
objects.translations (dict[str,dict[str,str]]) – The first key is the RFC 4646 language code, and the nested mapping contains translated strings accessible by a string property. e.g. {“fr-CA”: {“title”: “Mon titre”, “abstract”: “Une description”}}
Creates a WPS-3 Process instance to execute a CWL package definition.
Process parameters should be loaded from an existing
weaver.datatype.Process
instance generated usingweaver.wps_package.get_process_definition()
.Provided
kw
should correspond toweaver.datatype.Process.params_wps()
-
setup_logger
(self: bool, log_stdout_stderr=True) → None[source]¶ Configures useful loggers to catch most of the common output and/or error messages during package execution.
-
insert_package_log
(self, result)[source]¶ Retrieves additional CWL sub-process logs captures to retrieve internal application output and/or errors.
After execution of this method, the WPS output log (which can be obtained by
retrieve_package_job_log()
) will have additionalstderr/stdout
entries extracted from the underlying application package tool execution.The outputs and errors are inserted as best as possible in the logical order to make reading of the merged logs appear as a natural and chronological order. In the event that both output and errors are available, they are appended one after another as merging in an orderly fashion cannot be guaranteed by outside CWL runner.
- Parameters
result – output results returned from the CWL package instance execution.
Todo
improve for realtime updates when using async routine (https://github.com/crim-ca/weaver/issues/131)
See also
-
update_requirements
(self)[source]¶ Inplace modification of
package
to remove invalid items that would break behaviour we must enforce.
-
update_effective_user
(self)[source]¶ Update effective user/group for the Application Package to be executed.
FIXME: (experimental) update user/group permissions
Reducing permissions is safer inside docker application since weaver/cwltool could be running as root but this requires that mounted volumes have the required permissions so euid:egid can use them.
Overrides
cwltool
’s function to retrieve user/group id for ones we enforce.
-
update_status
(self: AnyStr, message: Number, progress: AnyStatusType, status) → None[source]¶ Updates the PyWPS real job status from a specified parameters.
-
step_update_status
(self: AnyStr, message: Number, progress: Number, start_step_progress: Number, end_step_progress: AnyStr, step_name: AnyValue, target_host: AnyStr, status) → None[source]¶
-
log_message
(self: AnyStatusType, status: AnyStr, message: Optional[Number], progress: int = None, level=logging.INFO) → None[source]¶
-
exception_message
(self: Type[Exception], exception_type: Optional[Exception], exception: AnyStr = None, message: AnyStatusType = 'no message', status: int = STATUS_EXCEPTION, level=logging.ERROR) → Exception[source]¶
-
classmethod
map_step_progress
(cls: int, step_index: int, steps_total) → Number[source]¶ Calculates the percentage progression of a single step of the full process.
Note
The step procession is adjusted according to delimited start/end of the underlying CWL execution to provide a continuous progress percentage over the complete execution. Otherwise, we would have values that jump around according to whichever progress the underlying remote WPS or monitored CWL employs, if any is provided.
-
_handler
(self: WPSRequest, request: ExecuteResponse, response) → ExecuteResponse[source]¶ Method called when process receives the WPS execution request.
-
must_fetch
(self: AnyStr, input_ref) → bool[source]¶ Figures out if file reference should be fetched immediately for local execution. If anything else than local script/docker, remote ADES/WPS process will fetch it. S3 are handled here to avoid error on remote WPS not supporting it.
See also
-
make_location_input
(self: AnyStr, input_type: ComplexInput, input_definition) → JSON[source]¶ Generates the JSON content required to specify a CWL
File
input definition from a location.Note
If the process requires
OpenSearch
references that should be preserved as is, use scheme defined byweaver.processes.constants.OPENSEARCH_LOCAL_FILE_SCHEME
prefix instead ofhttp(s)://
.
-
make_outputs
(self: CWLResults, cwl_result) → None[source]¶ Maps CWL result outputs to corresponding WPS outputs.
-
make_location_output
(self: CWLResults, cwl_result: AnyStr, output_id) → None[source]¶ Rewrite the WPS output with required location using result path from CWL execution.
Configures the parameters such that PyWPS will either auto-resolve the local paths to match with URL defined by
weaver.wps_output_url
or upload it to S3 bucket fromweaver.wps_output_s3_bucket
and provide reference directly.See also
-
make_tool
(self: ToolPathObjectType, toolpath_object: LoadingContext, loading_context) → ProcessCWL[source]¶
-
get_application_requirement
(self) → Dict[AnyStr, Any][source]¶ Obtains the first item in CWL package
requirements
orhints
that corresponds to a Weaver-specific application type as defined inCWL_REQUIREMENT_APP_TYPES
.- Returns
dictionary that minimally has
class
field, and optionally other parameters from that requirement.
-
get_job_process_definition
(self: AnyStr, jobname: JSON, joborder: CWL, tool) → WpsPackage[source]¶ This function is called before running an ADES job (either from a workflow step or a simple EMS dispatch). It must return a WpsProcess instance configured with the proper package, ADES target and cookies.
- Parameters
jobname – The workflow step or the package id that must be launch on an ADES
string
joborder – The params for the job
dict {input_name: input_value}
input_value is one of input_object or array [input_object] input_object is one of string or dict {class: File, location: string} in our case input are expected to be File objecttool – Whole CWL config including hints requirement
weaver.processes.wps_process_base
¶-
class
weaver.processes.wps_process_base.
WpsProcessInterface
(: WPSRequest, request)[source]¶ Common interface for WpsProcess to be used is cwl jobs
Initialize self. See help(type(self)) for accurate signature.
-
abstract
execute
(self, workflow_inputs: CWL, out_dir: AnyStr, expected_outputs: Dict[AnyStr, AnyStr])[source]¶ Execute a remote process using the given inputs. The function is expected to monitor the process and update the status. Retrieve the expected outputs and store them in the
out_dir
.- Parameters
workflow_inputs – CWL job dict
out_dir – directory where the outputs must be written
expected_outputs – expected value outputs as {‘id’: ‘value’}
-
abstract
weaver.processes.wps_testing
¶-
class
weaver.processes.wps_testing.
WpsTestProcess
(**kw)[source]¶ - Parameters
handler – A callable that gets invoked for each incoming request. It should accept a single
pywps.app.WPSRequest
argument and return apywps.app.WPSResponse
object.identifier (string) – Name of this process.
title (string) – Human readable title of process.
abstract (string) – Brief narrative description of the process.
keywords (list) – Keywords that characterize a process.
inputs – List of inputs accepted by this process. They should be
LiteralInput
andComplexInput
andBoundingBoxInput
objects.outputs – List of outputs returned by this process. They should be
LiteralOutput
andComplexOutput
andBoundingBoxOutput
objects.metadata – List of metadata advertised by this process. They should be
pywps.app.Common.Metadata
objects.translations (dict[str,dict[str,str]]) – The first key is the RFC 4646 language code, and the nested mapping contains translated strings accessible by a string property. e.g. {“fr-CA”: {“title”: “Mon titre”, “abstract”: “Une description”}}
Initialize self. See help(type(self)) for accurate signature.
weaver.processes.wps_workflow
¶-
weaver.processes.wps_workflow.
default_make_tool
(toolpath_object: ToolPathObjectType, loading_context: LoadingContext, get_job_process_definition: GetJobProcessDefinitionFunction) → ProcessCWL[source]¶
-
class
weaver.processes.wps_workflow.
CallbackJob
(: WpsWorkflow, job: Callable[[Any, Any], Any], output_callback: Builder, cachebuilder: Text, jobcache)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.processes.wps_workflow.
WpsWorkflow
(: Dict[Text, Any], toolpath_object: LoadingContext, loading_context: GetJobProcessDefinitionFunction, get_job_process_definition)[source]¶ Build a Process object from the provided dictionary.
-
job
(self, joborder: Dict[Text, AnyValue], output_callbacks: Callable[[Any, Any], Any], runtime_context: RuntimeContext) → Generator[Union[JobBase, CallbackJob], None, None][source]¶ Workflow job generator.
- Parameters
joborder – inputs of the job submission
output_callbacks – method to fetch step outputs and corresponding step details
runtime_context – configs about execution environment
- Returns
-
-
class
weaver.processes.wps_workflow.
WpsWorkflowJob
(builder: Builder, joborder: Dict[Text, Union[Dict[Text, Any], List, Text, None]], requirements: List[Dict[Text, Text]], hints: List[Dict[Text, Text]], name: Text, wps_process: WpsProcessInterface, expected_outputs: List[ExpectedOutputType])[source]¶ Initialize the job object.
-
run
(self, runtimeContext: RuntimeContext, tmpdir_lock: Optional[threading.Lock] = None) → None[source]¶
-
weaver.store
¶
weaver.store.base
¶-
class
weaver.store.base.
StoreInterface
[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.store.base.
StoreServices
[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.store.base.
StoreProcesses
[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.store.base.
StoreJobs
[source]¶ Initialize self. See help(type(self)) for accurate signature.
weaver.store.mongodb
¶Stores to read/write data to from/to MongoDB using pymongo.
-
class
weaver.store.mongodb.
MongodbStore
(: Collection, collection: Optional[Dict[AnyStr, Any]], sane_name_config=None)[source]¶ Base class extended by all concrete store implementations.
Initialize self. See help(type(self)) for accurate signature.
-
classmethod
get_args_kwargs
(cls: Any, *args: Any, **kwargs) → Tuple[Tuple, Dict][source]¶ Filters
MongodbStore
-specific arguments to safely pass them down its__init__
.
-
classmethod
-
class
weaver.store.mongodb.
MongodbServiceStore
(*args, **kwargs)[source]¶ Registry for OWS services. Uses MongoDB to store service url and attributes.
Initialize self. See help(type(self)) for accurate signature.
-
save_service
(self: Service, service: bool, overwrite: Optional[Request] = True, request=None) → Service[source]¶ Stores an OWS service in mongodb.
-
delete_service
(self: AnyStr, name: Optional[Request], request=None) → bool[source]¶ Removes service from MongoDB storage.
-
list_services
(self: Optional[Request], request=None) → List[Service][source]¶ Lists all services in MongoDB storage.
-
fetch_by_name
(self: AnyStr, name: Optional[AnyStr], visibility: Optional[Request] = None, request=None) → Service[source]¶ Gets service for given
name
from MongoDB storage.
-
-
class
weaver.store.mongodb.
MongodbProcessStore
(*args, **kwargs)[source]¶ Registry for processes. Uses MongoDB to store processes and attributes.
Initialize self. See help(type(self)) for accurate signature.
-
static
_get_process_field
(process: AnyProcess, function_dict: Union[Dict[AnyProcessType, Callable[], Any]], Callable[], Any]]) → Any[source]¶ Takes a lambda expression or a dict of process-specific lambda expressions to retrieve a field. Validates that the passed process object is one of the supported types.
- Parameters
process – process to retrieve the field from.
function_dict – lambda or dict of lambda of process type
- Returns
retrieved field if the type was supported
- Raises
ProcessInstanceError – invalid process type
-
save_process
(self: Union[Process, ProcessWPS], process: bool, overwrite: Optional[Request] = True, request=None) → Process[source]¶ Stores a process in storage.
- Parameters
process – An instance of
weaver.datatype.Process
.overwrite – Overwrite the matching process instance by name if conflicting.
request – <unused>
-
delete_process
(self: AnyStr, process_id: Optional[AnyStr], visibility: Optional[Request] = None, request=None) → bool[source]¶ Removes process from database, optionally filtered by visibility. If
visibility=None
, the process is deleted (if existing) regardless of its visibility value.
-
list_processes
(self: Optional[AnyStr], visibility: Optional[Request] = None, request=None) → List[Process][source]¶ Lists all processes in database, optionally filtered by visibility.
- Parameters
visibility – One value amongst weaver.visibility.
request – <unused>
-
fetch_by_id
(self: AnyStr, process_id: Optional[AnyStr], visibility: Optional[Request] = None, request=None) → Process[source]¶ Get process for given process_id from storage, optionally filtered by visibility. If
visibility=None
, the process is retrieved (if existing) regardless of its visibility value.- Parameters
process_id – process identifier
visibility – one value amongst weaver.visibility.
request – <unused>
- Returns
An instance of
weaver.datatype.Process
.
-
get_visibility
(self: AnyStr, process_id: Optional[Request], request=None) → AnyStr[source]¶ Get visibility of a process.
- Returns
One value amongst weaver.visibility.
-
set_visibility
(self: AnyStr, process_id: AnyStr, visibility: Optional[Request], request=None) → None[source]¶ Set visibility of a process.
- Parameters
visibility – One value amongst weaver.visibility.
process_id –
request – <unused>
- Raises
ValueError – when
visibility
is not one ofweaver.visibility.VISIBILITY_VALUES
.
-
static
-
class
weaver.store.mongodb.
MongodbJobStore
(*args, **kwargs)[source]¶ Registry for process jobs tracking. Uses MongoDB to store job attributes.
Initialize self. See help(type(self)) for accurate signature.
-
save_job
(self, task_id: AnyStr, process: AnyStr, service: Optional[AnyStr] = None, inputs: Optional[List[Any]] = None, is_workflow: bool = False, user_id: Optional[int] = None, execute_async: bool = True, custom_tags: Optional[List[AnyStr]] = None, access: Optional[AnyStr] = None, notification_email: Optional[AnyStr] = None, accept_language: Optional[AnyStr] = None) → Job[source]¶ Stores a job in mongodb.
-
update_job
(self: Job, job) → Job[source]¶ Updates a job parameters in MongoDB storage. :param job: instance of
weaver.datatype.Job
.
-
delete_job
(self: AnyStr, job_id: Optional[Request], request=None) → bool[source]¶ Removes job from MongoDB storage.
-
fetch_by_id
(self: AnyStr, job_id: Optional[Request], request=None) → Job[source]¶ Gets job for given
job_id
from MongoDB storage.
-
list_jobs
(self: Optional[Request], request=None) → List[Job][source]¶ Lists all jobs in MongoDB storage. For user-specific access to available jobs, use
MongodbJobStore.find_jobs()
instead.
-
find_jobs
(self, request: Request, process: Optional[AnyStr] = None, service: Optional[AnyStr] = None, tags: Optional[List[AnyStr]] = None, access: Optional[AnyStr] = None, notification_email: Optional[AnyStr] = None, status: Optional[AnyStr] = None, sort: Optional[AnyStr] = None, page: int = 0, limit: int = 10, group_by: Optional[Union[AnyStr, List[AnyStr]]] = None) → Union[JobListAndCount, JobCategoriesAndCount][source]¶ Finds all jobs in MongoDB storage matching search filters to obtain results with requested paging or grouping.
- Parameters
request – request that lead to this call to obtain permissions and user id.
process – process name to filter matching jobs.
service – service name to filter matching jobs.
tags – list of tags to filter matching jobs.
access – access visibility to filter matching jobs (default: PUBLIC).
notification_email – notification email to filter matching jobs.
status – status to filter matching jobs.
sort – field which is used for sorting results (default: creation date, descending).
page – page number to return when using result paging (only when not using
group_by
).limit – number of jobs per page when using result paging (only when not using
group_by
).group_by – one or many fields specifying categories to form matching groups of jobs (paging disabled).
- Returns
(list of jobs matching paging OR list of {categories, list of jobs, count}) AND total of matched job
Example:
Using paging (default), result will be in the form:
( [Job(1), Job(2), Job(3), ...], <total> )
Where
<total>
will indicate the complete count of matched jobs with filters, but the list of jobs will be limited only topage
index andlimit
specified.Using grouping with a list of field specified with
group_by
, results will be in the form:( [{category: {field1: valueA, field2: valueB, ...}, [Job(1), Job(2), ...], count: <count>}, {category: {field1: valueC, field2: valueD, ...}, [Job(x), Job(y), ...], count: <count>}, ... ], <total> )
Where
<total>
will again indicate all matched jobs by every category combined, and<count>
will indicate the amount of jobs matched for each individual category. Also,category
will indicate values of specified fields (fromgroup_by
) that compose corresponding jobs with matching values.
-
-
class
weaver.store.mongodb.
MongodbQuoteStore
(*args, **kwargs)[source]¶ Registry for quotes. Uses MongoDB to store quote attributes.
Initialize self. See help(type(self)) for accurate signature.
-
fetch_by_id
(self: AnyStr, quote_id) → Quote[source]¶ Gets quote for given
quote_id
from MongoDB storage.
-
find_quotes
(self: Optional[AnyStr], process_id: int = None, page: int = 0, limit: Optional[AnyStr] = 10, sort=None) → Tuple[List[Quote], int][source]¶ Finds all quotes in MongoDB storage matching search filters.
Returns a tuple of filtered
items
and theircount
, whereitems
can have paging and be limited to a maximum per page, butcount
always indicate the total number of matches.
-
-
class
weaver.store.mongodb.
MongodbBillStore
(*args, **kwargs)[source]¶ Registry for bills. Uses MongoDB to store bill attributes.
Initialize self. See help(type(self)) for accurate signature.
-
fetch_by_id
(self: AnyStr, bill_id) → Bill[source]¶ Gets bill for given
bill_id
from MongoDB storage.
-
find_bills
(self: Optional[AnyStr], quote_id: int = None, page: int = 0, limit: Optional[AnyStr] = 10, sort=None) → Tuple[List[Bill], int][source]¶ Finds all bills in MongoDB storage matching search filters.
Returns a tuple of filtered
items
and theircount
, whereitems
can have paging and be limited to a maximum per page, butcount
always indicate the total number of matches.
-
weaver.wps_restapi
¶
weaver.wps_restapi.jobs
¶weaver.wps_restapi.jobs.jobs
¶-
weaver.wps_restapi.jobs.jobs.
get_job
(request: Request) → Job[source]¶ Obtain a job from request parameters.
- Returns
Job information if found.
- Raises
HTTPNotFound – with JSON body details on missing/non-matching job, process, provider IDs.
-
weaver.wps_restapi.jobs.jobs.
validate_service_process
(request: Request) → Tuple[Union[None, AnyStr], Union[None, AnyStr]][source]¶ Verifies that service or process specified by path or query will raise the appropriate error if applicable.
-
weaver.wps_restapi.jobs.jobs.
get_queried_jobs
(request)[source]¶ Retrieve the list of jobs which can be filtered, sorted, paged and categorized using query parameters.
-
weaver.wps_restapi.jobs.jobs.
cancel_job
(request)[source]¶ Dismiss a job.
Note: Will only stop tracking this particular process (WPS 1.0 doesn’t allow to stop a process)
weaver.wps_restapi.processes
¶weaver.wps_restapi.processes.processes
¶-
weaver.wps_restapi.processes.processes.
execute_process
(self, job_id, url, headers=None, notification_email=None)[source]¶
-
weaver.wps_restapi.processes.processes.
make_results_relative
(results: List[JSON], settings: SettingsType) → List[JSON][source]¶ Redefines job results to be saved in database as relative paths to output directory configured in PyWPS (i.e.: relative to
weaver.wps_output_dir
).This allows us to easily adjust the exposed result HTTP path according to server configuration (i.e.: relative to
weaver.wps_output_path
and/orweaver.wps_output_url
) and it also avoid rewriting the whole database job results if the setting is changed later on.
-
weaver.wps_restapi.processes.processes.
map_locations
(job: Job, settings: SettingsType) → None[source]¶ Generates symlink references from the Job UUID to PyWPS UUID results (outputs directory, status and log locations). Update the Job’s WPS ID if applicable (job executed locally). Assumes that all results are located under the same reference UUID.
-
weaver.wps_restapi.processes.processes.
set_wps_language
(wps: WebProcessingService, accept_language: Optional[str] = None, request: Optional[Request] = None) → None[source]¶ Set the
language
property on theWebProcessingService
object.Given the Accept-Language header value, match the best language to the supported languages.
By default, and if no match is found, the
WebProcessingService.language
property is set to None.https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language (q-factor weighting is ignored, only order is considered)
- Parameters
wps – process for which to set the language header if it is accepted
accept_language (str) – the value of the Accept-Language header
request – request from which to extract Accept-Language header if not provided directly
-
weaver.wps_restapi.processes.processes.
validate_supported_submit_job_handler_parameters
(json_body)[source]¶ Tests supported parameters not automatically validated by colander deserialize.
-
weaver.wps_restapi.processes.processes.
submit_job_handler
(request: Request, service_url: AnyStr, is_workflow: bool = False, visibility: Optional[AnyStr] = None) → HTTPSuccessful[source]¶
-
weaver.wps_restapi.processes.processes.
submit_provider_job
(request)[source]¶ Execute a provider process.
-
weaver.wps_restapi.processes.processes.
list_remote_processes
(service: Service, request: Request) → List[ProcessDB][source]¶ Obtains a list of remote service processes in a compatible
weaver.datatype.Process
format.Note: remote processes won’t be stored to the local process storage.
-
weaver.wps_restapi.processes.processes.
get_provider_processes
(request)[source]¶ Retrieve available provider processes (GetCapabilities).
-
weaver.wps_restapi.processes.processes.
describe_provider_process
(request: Request) → ProcessDB[source]¶ Obtains a remote service process description in a compatible local process format.
Note: this processes won’t be stored to the local process storage.
-
weaver.wps_restapi.processes.processes.
get_provider_process
(request)[source]¶ Retrieve a process description (DescribeProcess).
-
weaver.wps_restapi.processes.processes.
get_processes_filtered_by_valid_schemas
(request: Request) → Tuple[List[JSON], List[AnyStr]][source]¶ Validates the processes summary schemas and returns them into valid/invalid lists. :returns: list of valid process summaries and invalid processes IDs for manual cleanup.
-
weaver.wps_restapi.processes.processes.
get_processes
(request)[source]¶ List registered processes (GetCapabilities). Optionally list both local and provider processes.
-
weaver.wps_restapi.processes.processes.
add_local_process
(request)[source]¶ Register a local process.
-
weaver.wps_restapi.processes.processes.
get_local_process
(request)[source]¶ Get a registered local process information (DescribeProcess).
-
weaver.wps_restapi.processes.processes.
get_local_process_package
(request)[source]¶ Get a registered local process package definition.
-
weaver.wps_restapi.processes.processes.
get_local_process_payload
(request)[source]¶ Get a registered local process payload definition.
-
weaver.wps_restapi.processes.processes.
get_process_visibility
(request)[source]¶ Get the visibility of a registered local process.
-
weaver.wps_restapi.processes.processes.
set_process_visibility
(request)[source]¶ Set the visibility of a registered local process.
weaver.wps_restapi.providers
¶weaver.wps_restapi.providers.providers
¶weaver.wps_restapi.quotation
¶weaver.wps_restapi.quotation.quotes
¶-
weaver.wps_restapi.quotation.quotes.
process_quote_estimator
(process)[source]¶ - Parameters
process – instance of
weaver.datatype.Process
for which to evaluate the quote.- Returns
dict of {price, currency, estimatedTime} values for the process quote.
weaver.wps_restapi.api
¶-
weaver.wps_restapi.api.
api_versions
(request: Request) → HTTPException[source]¶ Weaver versions information.
-
weaver.wps_restapi.api.
api_conformance
(request: Request) → HTTPException[source]¶ Weaver specification conformance information.
-
weaver.wps_restapi.api.
get_swagger_json
(http_scheme: AnyStr = 'http', http_host: AnyStr = 'localhost', base_url: Optional[AnyStr] = None, use_docstring_summary: bool = True) → dict[source]¶ Obtains the JSON schema of weaver API from request and response views schemas.
- Parameters
http_scheme – Protocol scheme to use for building the API base if not provided by base URL parameter.
http_host – Hostname to use for building the API base if not provided by base URL parameter.
base_url – Explicit base URL to employ of as API base instead of HTTP scheme/host parameters.
use_docstring_summary – Setting that controls if function docstring should be used to auto-generate the summary field of responses.
-
weaver.wps_restapi.api.
api_swagger_json
(request: Request) → dict[source]¶ weaver REST API schema generation in JSON format.
-
weaver.wps_restapi.api.
api_swagger_ui
(request)[source]¶ weaver REST API swagger-ui schema documentation (this page).
-
weaver.wps_restapi.api.
get_request_info
(request: Request, detail: Optional[AnyStr] = None) → JSON[source]¶ Provided additional response details based on the request and execution stack on failure.
-
weaver.wps_restapi.api.
ows_json_format
(function)[source]¶ Decorator that adds additional detail in the response’s JSON body if this is the returned content-type.
-
weaver.wps_restapi.api.
not_found_or_method_not_allowed
(request)[source]¶ Overrides the default is HTTPNotFound [404] by appropriate HTTPMethodNotAllowed [405] when applicable.
Not found response can correspond to underlying process operation not finding a required item, or a completely unknown route (path did not match any existing API definition). Method not allowed is more specific to the case where the path matches an existing API route, but the specific request method (GET, POST, etc.) is not allowed on this path.
Without this fix, both situations return [404] regardless.
Overrides the default is HTTPForbidden [403] by appropriate HTTPUnauthorized [401] when applicable.
Unauthorized response is for restricted user access according to credentials and/or authorization headers. Forbidden response is for operation refused by the underlying process operations.
Without this fix, both situations return [403] regardless.
weaver.wps_restapi.colander_extras
¶-
class
weaver.wps_restapi.colander_extras.
DropableNoneSchema
(*arg, **kw)[source]¶ Drops the underlying schema node if
missing=drop
was specified and that the value representing it isNone
.Original behaviour of schema classes that can have children nodes such as
colander.MappingSchema
andcolander.SequenceSchema
are to drop the sub-node only if its value is resolved ascolander.null
orcolander.drop
. This results in “missing” definitions replaced byNone
in many implementations to raisecolander.Invalid
during deserialization. Inheriting this class in a schema definition will handle this situation automatically.Required schemas (without
missing=drop
, i.e.:colander.required
) will still raise for undefined nodes.The following snippet shows the result that can be achieved using this schema class:
class SchemaA(DropableNoneSchema, MappingSchema): field = SchemaNode(String()) class SchemaB(MappingSchema): s1 = SchemaA(missing=drop) # optional s2 = SchemaA() # required SchemaB().deserialize({"s1": None, "s2": {"field": "ok"}}) # >> {'s2': {'field': 'ok'}}
Initialize self. See help(type(self)) for accurate signature.
-
deserialize
(self, cstruct)[source]¶ Deserialize the cstruct into an appstruct based on the schema, run this appstruct through the preparer, if one is present, then validate the prepared appstruct. The
cstruct
value is deserialized into anappstruct
unconditionally.If
appstruct
returned by type deserialization and preparation is the valuecolander.null
, do something special before attempting validation:If the
missing
attribute of this node has been set explicitly, return its value. No validation of this value is performed; it is simply returned.If the
missing
attribute of this node has not been set explicitly, raise acolander.Invalid
exception error.
If the appstruct is not
colander.null
and cannot be validated , acolander.Invalid
exception will be raised.If a
cstruct
argument is not explicitly provided, it defaults tocolander.null
.
-
-
class
weaver.wps_restapi.colander_extras.
VariableMappingSchema
(unknown='ignore')[source]¶ Mapping schema that will allow any unknown field to remain present in the resulting deserialization.
This definition is useful for defining a dictionary where some field names are not known in advance. Other fields that are explicitly specified with sub-schema nodes will be validated as per usual behaviour.
Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.colander_extras.
SchemaNodeDefault
(*arg, **kw)[source]¶ If
default
keyword is provided duringcolander.SchemaNode
creation, overrides the returned value by this default if missing from the structure duringdeserialize()
call.Original behaviour was to drop the missing value instead of replacing by the default. Executes all other
colander.SchemaNode
operations normally.Initialize self. See help(type(self)) for accurate signature.
-
deserialize
(self, cstruct)[source]¶ Deserialize the cstruct into an appstruct based on the schema, run this appstruct through the preparer, if one is present, then validate the prepared appstruct. The
cstruct
value is deserialized into anappstruct
unconditionally.If
appstruct
returned by type deserialization and preparation is the valuecolander.null
, do something special before attempting validation:If the
missing
attribute of this node has been set explicitly, return its value. No validation of this value is performed; it is simply returned.If the
missing
attribute of this node has not been set explicitly, raise acolander.Invalid
exception error.
If the appstruct is not
colander.null
and cannot be validated , acolander.Invalid
exception will be raised.If a
cstruct
argument is not explicitly provided, it defaults tocolander.null
.
-
-
class
weaver.wps_restapi.colander_extras.
OneOfMappingSchema
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
deserialize
(self, cstruct)[source]¶ Deserialize the cstruct into an appstruct based on the schema, run this appstruct through the preparer, if one is present, then validate the prepared appstruct. The
cstruct
value is deserialized into anappstruct
unconditionally.If
appstruct
returned by type deserialization and preparation is the valuecolander.null
, do something special before attempting validation:If the
missing
attribute of this node has been set explicitly, return its value. No validation of this value is performed; it is simply returned.If the
missing
attribute of this node has not been set explicitly, raise acolander.Invalid
exception error.
If the appstruct is not
colander.null
and cannot be validated , acolander.Invalid
exception will be raised.If a
cstruct
argument is not explicitly provided, it defaults tocolander.null
.
-
-
class
weaver.wps_restapi.colander_extras.
CustomTypeConversionDispatcher
(custom_converters=None, default_converter=None)[source]¶ Initialize self. See help(type(self)) for accurate signature.
weaver.wps_restapi.swagger_definitions
¶This module should contain any and every definitions in use to build the swagger UI, so that one can update the swagger without touching any other files after the initial integration
-
class
weaver.wps_restapi.swagger_definitions.
SchemaNode
(*arg, **kw)[source]¶ Override the default
colander.SchemaNode
to auto-handledefault
value substitution if an actual value was omitted during deserialization for a field defined with this schema and adefault
parameter.See also
Implementation in
SchemaNodeDefault
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
SequenceSchema
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
MappingSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ExplicitMappingSchema
(*arg, **kw)[source]¶ Original behaviour of
colander.MappingSchema
implementation, where fields referencing toNone
values are kept as an explicit indication of an undefined or missing value for this field.Initialize self. See help(type(self)) for accurate signature.
-
weaver.wps_restapi.swagger_definitions.
process_package_uri
= /processes/{process_id}/package[source]¶
-
weaver.wps_restapi.swagger_definitions.
process_payload_uri
= /processes/{process_id}/payload[source]¶
-
weaver.wps_restapi.swagger_definitions.
process_visibility_uri
= /processes/{process_id}/visibility[source]¶
-
weaver.wps_restapi.swagger_definitions.
process_job_uri
= /processes/{process_id}/jobs/{job_id}[source]¶
-
weaver.wps_restapi.swagger_definitions.
process_quotes_uri
= /processes/{process_id}/quotations[source]¶
-
weaver.wps_restapi.swagger_definitions.
process_quote_uri
= /processes/{process_id}/quotations/{quote_id}[source]¶
-
weaver.wps_restapi.swagger_definitions.
process_results_uri
= /processes/{process_id}/jobs/{job_id}/result[source]¶
-
weaver.wps_restapi.swagger_definitions.
process_exceptions_uri
= /processes/{process_id}/jobs/{job_id}/exceptions[source]¶
-
weaver.wps_restapi.swagger_definitions.
process_logs_uri
= /processes/{process_id}/jobs/{job_id}/logs[source]¶
-
weaver.wps_restapi.swagger_definitions.
provider_processes_uri
= /providers/{provider_id}/processes[source]¶
-
weaver.wps_restapi.swagger_definitions.
provider_process_uri
= /providers/{provider_id}/processes/{process_id}[source]¶
-
weaver.wps_restapi.swagger_definitions.
jobs_full_uri
= /providers/{provider_id}/processes/{process_id}/jobs[source]¶
-
weaver.wps_restapi.swagger_definitions.
job_full_uri
= /providers/{provider_id}/processes/{process_id}/jobs/{job_id}[source]¶
-
weaver.wps_restapi.swagger_definitions.
job_exceptions_uri
= /providers/{provider_id}/processes/{process_id}/jobs/{job_id}/exceptions[source]¶
-
weaver.wps_restapi.swagger_definitions.
results_full_uri
= /providers/{provider_id}/processes/{process_id}/jobs/{job_id}/result[source]¶
-
weaver.wps_restapi.swagger_definitions.
result_full_uri
= /providers/{provider_id}/processes/{process_id}/jobs/{job_id}/result/{result_id}[source]¶
-
weaver.wps_restapi.swagger_definitions.
result_short_uri
= /jobs/{job_id}/result/{result_id}[source]¶
-
weaver.wps_restapi.swagger_definitions.
exceptions_full_uri
= /providers/{provider_id}/processes/{process_id}/jobs/{job_id}/exceptions[source]¶
-
weaver.wps_restapi.swagger_definitions.
logs_full_uri
= /providers/{provider_id}/processes/{process_id}/jobs/{job_id}/logs[source]¶
-
class
weaver.wps_restapi.swagger_definitions.
ProcessPath
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProviderPath
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobPath
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
BillPath
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
QuotePath
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ResultPath
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JsonHeader
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
HtmlHeader
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
XmlHeader
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AcceptHeader
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AcceptLanguageHeader
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
KeywordList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JsonLink
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
MetadataBase
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
MetadataLink
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
MetadataValue
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Metadata
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
MetadataList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JsonLinkList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
LandingPage
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Format
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FormatDescription
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FormatDescriptionList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AdditionalParameterValuesList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AdditionalParameter
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AdditionalParameterList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AdditionalParameters
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AdditionalParametersList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Content
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Offering
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OWSContext
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DescriptionType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
MinMaxOccursInt
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
MinMaxOccursStr
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
WithMinMaxOccurs
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ComplexInputType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
SupportedCrs
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
SupportedCrsList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
BoundingBoxInputType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DataTypeSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
UomSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AllowedValuesList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AllowedValues
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AllowedRange
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AllowedRangesList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AllowedRanges
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AnyValue
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ValuesReference
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
LiteralDataDomainType
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
LiteralDataDomainTypeList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
LiteralInputType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InputType
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InputTypeList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
LiteralOutputType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
BoundingBoxOutputType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ComplexOutputType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OutputDataDescriptionType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OutputType
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OutputDescriptionList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobExecuteModeEnum
(*args, **kwargs)[source]¶ Override the default
colander.SchemaNode
to auto-handledefault
value substitution if an actual value was omitted during deserialization for a field defined with this schema and adefault
parameter.See also
Implementation in
SchemaNodeDefault
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobControlOptionsEnum
(*args, **kwargs)[source]¶ Override the default
colander.SchemaNode
to auto-handledefault
value substitution if an actual value was omitted during deserialization for a field defined with this schema and adefault
parameter.See also
Implementation in
SchemaNodeDefault
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobResponseOptionsEnum
(*args, **kwargs)[source]¶ Override the default
colander.SchemaNode
to auto-handledefault
value substitution if an actual value was omitted during deserialization for a field defined with this schema and adefault
parameter.See also
Implementation in
SchemaNodeDefault
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
TransmissionModeEnum
(*args, **kwargs)[source]¶ Override the default
colander.SchemaNode
to auto-handledefault
value substitution if an actual value was omitted during deserialization for a field defined with this schema and adefault
parameter.See also
Implementation in
SchemaNodeDefault
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobStatusEnum
(*args, **kwargs)[source]¶ Override the default
colander.SchemaNode
to auto-handledefault
value substitution if an actual value was omitted during deserialization for a field defined with this schema and adefault
parameter.See also
Implementation in
SchemaNodeDefault
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobSortEnum
(*args, **kwargs)[source]¶ Override the default
colander.SchemaNode
to auto-handledefault
value substitution if an actual value was omitted during deserialization for a field defined with this schema and adefault
parameter.See also
Implementation in
SchemaNodeDefault
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
QuoteSortEnum
(*args, **kwargs)[source]¶ Override the default
colander.SchemaNode
to auto-handledefault
value substitution if an actual value was omitted during deserialization for a field defined with this schema and adefault
parameter.See also
Implementation in
SchemaNodeDefault
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
LaunchJobQuerystring
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
VisibilityValue
(*arg, **kw)[source]¶ Override the default
colander.SchemaNode
to auto-handledefault
value substitution if an actual value was omitted during deserialization for a field defined with this schema and adefault
parameter.See also
Implementation in
SchemaNodeDefault
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Visibility
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FrontpageEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
VersionsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ConformanceEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
SwaggerJSONEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
SwaggerUIEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProviderEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProviderProcessEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessPackageEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessPayloadEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessVisibilityGetEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessVisibilityPutEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FullJobEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ShortJobEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessResultsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FullResultsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ShortResultsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FullExceptionsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ShortExceptionsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessExceptionsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FullLogsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ShortLogsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessLogsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
CreateProviderRequestBody
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InputDataType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OutputDataType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Output
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OutputList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProviderSummarySchema
(*arg, **kw)[source]¶ WPS provider summary definition.
Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProviderCapabilitiesSchema
(*arg, **kw)[source]¶ WPS provider capabilities.
Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
TransmissionModeList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobControlOptionsList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ExceptionReportType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessSummary
(*arg, **kw)[source]¶ WPS process definition.
Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessSummaryList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessCollection
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Process
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessOutputDescriptionSchema
(*arg, **kw)[source]¶ WPS process output definition.
Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobStatusInfo
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobEntrySchema
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobCollection
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
CreatedJobStatusSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
CreatedQuotedJobStatusSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetPagingJobsSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GroupedJobsCategorySchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GroupedCategoryJobsSchema
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetGroupedJobsSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetQueriedJobsSchema
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DismissedJobSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
QuoteProcessParametersSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AlternateQuotation
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
AlternateQuotationList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Reference
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DataEncodingAttributes
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DataFloat
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DataInteger
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DataString
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DataBoolean
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ValueType
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Input
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InputList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Execute
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Quotation
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
QuoteProcessListSchema
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
QuoteSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
QuotationList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
QuotationListSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
BillSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
BillList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
BillListSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
SupportedValues
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DefaultValues
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Unit
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
UnitType
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessInputDescriptionSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessDescriptionSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
UndeploymentResult
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DeploymentResult
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessDescriptionBodySchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProvidersSchema
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobOutputSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
JobOutputsSchema
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OutputInfo
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OutputInfoList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ExceptionTextList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ExceptionSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ExceptionsOutputSchema
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
LogsOutputSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FrontpageParameterSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FrontpageParameters
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
FrontpageSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
SwaggerJSONSpecSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
SwaggerUISpecSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
VersionsSpecSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
VersionsList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
VersionsSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ConformanceList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ConformanceSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
PackageBody
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ExecutionUnit
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ExecutionUnitList
(*arg, **kw)[source]¶ Override the default
colander.SequenceSchema
to auto-handle dropping missing entry definitions when its value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessOffering
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessDescriptionChoiceType
(*args, **kwargs)[source]¶ Allows specifying multiple supported mapping schemas variants for an underlying schema definition. Corresponds to the
oneOf
specifier of OpenAPI specification.Example:
class Variant1(MappingSchema): [...fields of Variant1...] class Variant2(MappingSchema): [...fields of Variant2...] class RequiredByBoth(MappingSchema): [...fields required by both Variant1 and Variant2...] class LiteralDataDomainType(OneOfMappingSchema, RequiredByBoth): _one_of = (Variant1, Variant2) [...alternatively, field required by all variants here...]
In the above example, the validation (ie:
deserialize
) process will succeed if any of the_one_of
variants’ validator completely succeed, and will fail if every variant fails validation execution.Warning
Because the validation process requires only at least one of the variants to succeed, it is important to insert more permissive validators later in the
_one_of
iterator. For example, having a variant with all fields defined as optional (ie: withmissing=drop
) inserted as first item in_one_of
will make it always succeed regardless of following variants. This would have as side effect to never validate the other variants explicitly for specific field types and formats since the first option would always consist as a valid input fulfilling the specified definition (ie: an empty{}
schema with all fields missing).Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Deploy
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
PostProcessesEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
PostProcessJobsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetJobsQueries
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetJobsRequest
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetJobsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetProcessJobsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetProviderJobsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetProcessJobEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
DeleteProcessJobEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
BillsEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
BillEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessQuotesEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessQuoteEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetQuotesQueries
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
QuotesEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
QuoteEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
PostProcessQuote
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
PostQuote
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
PostProcessQuoteRequestEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetProviders
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
PostProvider
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetProviderProcesses
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetProviderProcess
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
PostProviderProcessJobRequest
(*arg, **kw)[source]¶ Launching a new process request definition.
Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ErrorJsonResponseBodySchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetFrontpageResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetSwaggerJSONResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetSwaggerUIResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetVersionsResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetConformanceResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProvidersListResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetProvidersListResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProviderCapabilitiesSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetProviderCapabilitiesResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
NoContentDeleteProviderSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorDeleteProviderResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
NotImplementedDeleteProviderResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProviderProcessesSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetProviderProcessesListResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetProcessesQuery
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
GetProcessesEndpoint
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProcessesListResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetProcessesListResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkPostProcessDeployBodySchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkPostProcessesResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorPostProcessesResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProcessInfoResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetProcessResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProcessPackageSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetProcessPackageResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProcessPayloadSchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetProcessPayloadResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
ProcessVisibilityResponseBodySchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProcessVisibilitySchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetProcessVisibilityResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkPutProcessVisibilitySchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorPutProcessVisibilityResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkDeleteProcessUndeployBodySchema
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkDeleteProcessResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorDeleteProcessResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProviderProcessDescriptionResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetProviderProcessResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
CreatedPostProvider
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorPostProviderResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
NotImplementedPostProviderResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
CreatedLaunchJobResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorPostProcessJobResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorPostProviderProcessJobResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetProcessJobResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkDeleteProcessJobResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetQueriedJobsResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetJobsResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkDismissJobResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorDeleteJobResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetJobStatusResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetJobStatusResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
Result
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetJobResultsResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetJobResultsResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetOutputResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetJobOutputResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
CreatedQuoteExecuteResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorPostQuoteExecuteResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
CreatedQuoteRequestResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorPostQuoteRequestResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetQuoteInfoResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetQuoteInfoResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetQuoteListResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetQuoteListResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetBillDetailResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetBillInfoResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetBillListResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetBillListResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetJobExceptionsResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetJobExceptionsResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
OkGetJobLogsResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.wps_restapi.swagger_definitions.
InternalServerErrorGetJobLogsResponse
(*arg, **kw)[source]¶ Override the default
colander.MappingSchema
to auto-handle dropping missing field definitions when the corresponding value is eitherNone
,colander.null
orcolander.drop
.Initialize self. See help(type(self)) for accurate signature.
weaver.wps_restapi.utils
¶-
weaver.wps_restapi.utils.
get_wps_restapi_base_url
(container: AnySettingsContainer) → AnyStr[source]¶
-
weaver.wps_restapi.utils.
get_wps_output_format
(request: Request, service_url: AnyStr = None) → AnyStr[source]¶ - Get the preferred output format from WPS after checking various hints:
‘version’ in query string
Content-Type in accept headers
GetCapabilities of the service
- Parameters
request – request for which a response of WPS version-specific format must be generated.
service_url – endpoint URL of the service to request ‘GetCapabilities’ if version not found by previous hints.
- Returns
one of
OUTPUT_FORMAT
(default: 1.0.0 => ‘xml’ if no direct hint matched)
Submodules¶
weaver.app
¶
Weaver Web Application (weaver-manager
).
weaver.config
¶
-
weaver.config.
get_weaver_configuration
(container: AnySettingsContainer) → AnyStr[source]¶ Obtains the defined operation configuration mode.
- Returns
one value amongst
weaver.config.WEAVER_CONFIGURATIONS
.
-
weaver.config.
get_weaver_config_file
(file_path: AnyStr, default_config_file: AnyStr, generate_default_from_example: bool = True) → AnyStr[source]¶ Validates that the specified configuration file can be found, or falls back to the default one.
Handles ‘relative’ paths for settings in
WEAVER_DEFAULT_INI_CONFIG
referring to other configuration files. Default file must be one ofWEAVER_DEFAULT_CONFIGS
.If both the specified file and the default file cannot be found, default file under
WEAVER_DEFAULT_INI_CONFIG
is auto-generated from the corresponding.example
file ifgenerate_default_from_example
isTrue
. If it isFalse
, an empty string is returned instead without generation since no existing file can be guaranteed, and it is up to the caller to handle this situation as it explicitly disabled generation.- Parameters
file_path – path to a configuration file (can be relative if resolvable or matching a default file name)
default_config_file – one of
WEAVER_DEFAULT_CONFIGS
.generate_default_from_example – enable fallback copy of default configuration file from corresponding example.
- Returns
absolue path of the resolved file.
weaver.datatype
¶
-
class
weaver.datatype.
Base
[source]¶ Dictionary with extended attributes auto-
getter
/setter
for convenience. Explicitly overriddengetter
/setter
attributes are called instead ofdict
-keyget
/set
-item to ensure corresponding checks and/or value adjustments are executed before applying it to the sub-dict
.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.datatype.
Service
(*args, **kwargs)[source]¶ Dictionary that contains OWS services. It always has
url
key.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.datatype.
Job
(*args, **kwargs)[source]¶ Dictionary that contains OWS service jobs. It always has
id
andtask_id
keys.Initialize self. See help(type(self)) for accurate signature.
-
save_log
(self, errors: Optional[Union[AnyStr, List[WPSException]]] = None, logger: Optional[Logger] = None, message: Optional[AnyStr] = None, level: int = INFO) → None[source]¶ Logs the specified error and/or message, and adds the log entry to the complete job log.
For each new log entry, additional
Job
properties are added according toJob._get_log_msg()
and the format defined byget_job_log_msg()
.- Parameters
errors – An error message or a list of WPS exceptions from which to log and save generated message stack.
logger – An additional
Logger
for which to propagate logged messages on top saving them to the job.message – Explicit string to be logged, otherwise use the current
Job.status_message
is used.level – Logging level to apply to the logged
message
. This parameter is ignored iferrors
are logged.
Note
The job object is updated with the log but still requires to be pushed to database to actually persist it.
-
property
task_id
(self) → Optional[AnyStr][source]¶ Reference Task UUID attributed by the
Celery
worker that monitors and executes this job.
-
property
wps_id
(self) → Optional[AnyStr][source]¶ Reference WPS Request/Response UUID attributed by the executed
PyWPS
process.This UUID matches the status-location, log and output directory of the WPS process. This parameter is only available when the process is executed on this local instance.
See also
-
property
service
(self) → Optional[AnyStr][source]¶ Service identifier of the corresponding remote process.
See also
-
property
process
(self) → Optional[AnyStr][source]¶ Process identifier of the corresponding remote process.
See also
-
property
request
(self) → Optional[AnyStr][source]¶ XML request for WPS execution submission as string.
-
property
response
(self) → Optional[AnyStr][source]¶ XML status response from WPS execution submission as string.
-
json
(self: Optional[AnySettingsContainer], container=None) → JSON[source]¶ Obtain the JSON data representation for response body.
Note
Settings are required to update API shortcut URLs to job additional information. Without them, paths will not include the API host, which will not resolve to full URI.
-
-
class
weaver.datatype.
Process
(*args, **kwargs)[source]¶ Dictionary that contains a process description for db storage. It always has
identifier
andprocessEndpointWPS1
keys.Initialize self. See help(type(self)) for accurate signature.
-
params
(self) → Dict[AnyStr, Any][source]¶ Obtain the internal data representation for storage.
Note
This method implementation should provide a JSON-serializable definition of all fields representing the object to store.
-
json
(self) → JSON[source]¶ Obtain the JSON data representation for response body.
Note
This method implementation should validate the JSON schema against the API definition whenever applicable to ensure integrity between the represented data type and the expected API response.
-
static
from_wps
(wps_process: ProcessWPS, **extra_params: Any) → Process[source]¶ Converts a PyWPS Process into a
weaver.datatype.Process
using provided parameters.
-
-
class
weaver.datatype.
Quote
(*args, **kwargs)[source]¶ Dictionary that contains quote information. It always has
id
andprocess
keys.Initialize self. See help(type(self)) for accurate signature.
-
class
weaver.datatype.
Bill
(*args, **kwargs)[source]¶ Dictionary that contains bill information. It always has
id
,user
,quote
andjob
keys.Initialize self. See help(type(self)) for accurate signature.
weaver.exceptions
¶
-
exception
weaver.exceptions.
WeaverException
[source]¶ Base class of exceptions defined by
weaver
package.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
InvalidIdentifierValue
[source]¶ Error indicating that an id to be employed for following operations is not considered as valid to allow further processed or usage.
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
ServiceException
[source]¶ Base exception related to a
weaver.datatype.Service
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
ServiceNotAccessible
[source]¶ Error indicating that a WPS service exists but is not visible to retrieve from the storage backend of an instance of
weaver.store.ServiceStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
ServiceNotFound
[source]¶ Error indicating that an OWS service could not be read from the storage backend by an instance of
weaver.store.ServiceStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
ServiceRegistrationError
[source]¶ Error indicating that an OWS service could not be registered in the storage backend by an instance of
weaver.store.ServiceStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
ProcessException
[source]¶ Base exception related to a
weaver.datatype.Process
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
ProcessNotAccessible
[source]¶ Error indicating that a local WPS process exists but is not visible to retrieve from the storage backend of an instance of
weaver.store.ProcessStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
ProcessNotFound
[source]¶ Error indicating that a local WPS process could not be read from the storage backend by an instance of
weaver.store.ProcessStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
ProcessRegistrationError
[source]¶ Error indicating that a WPS process could not be registered in the storage backend by an instance of
weaver.store.ProcessStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
ProcessInstanceError
[source]¶ Error indicating that the process instance passed is not supported with storage backend by an instance of
weaver.store.ProcessStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
JobException
[source]¶ Base exception related to a
weaver.datatype.Job
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
JobNotFound
[source]¶ Error indicating that a job could not be read from the storage backend by an instance of
weaver.store.JobStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
JobRegistrationError
[source]¶ Error indicating that a job could not be registered in the storage backend by an instance of
weaver.store.JobStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
JobUpdateError
[source]¶ Error indicating that a job could not be updated in the storage backend by an instance of
weaver.store.JobStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
PackageException
[source]¶ Base exception related to a
weaver.processes.wps_package.Package
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
PackageTypeError
[source]¶ Error indicating that an instance of
weaver.processes.wps_package.WpsPackage
could not properly parse input/output type(s) for package deployment or execution.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
PackageRegistrationError
[source]¶ Error indicating that an instance of
weaver.processes.wps_package.WpsPackage
could not properly be registered for package deployment because of invalid prerequisite.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
PackageExecutionError
[source]¶ Error indicating that an instance of
weaver.processes.wps_package.WpsPackage
could not properly execute the package using provided inputs and package definition.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
PackageNotFound
[source]¶ Error indicating that an instance of
weaver.processes.wps_package.WpsPackage
could not properly retrieve the package definition using provided references.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
PayloadNotFound
[source]¶ Error indicating that an instance of
weaver.processes.wps_package.WpsPackage
could not properly retrieve the package deploy payload using provided references.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
QuoteException
[source]¶ Base exception related to a
weaver.datatype.Quote
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
QuoteNotFound
[source]¶ Error indicating that a quote could not be read from the storage backend by an instance of
weaver.store.QuoteStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
QuoteRegistrationError
[source]¶ Error indicating that a quote could not be registered in the storage backend by an instance of
weaver.store.QuoteStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
QuoteInstanceError
[source]¶ Error indicating that a given object doesn’t correspond to an expected instance of
weaver.datatype.Quote
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
BillException
[source]¶ Base exception related to a
weaver.datatype.Bill
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
BillNotFound
[source]¶ Error indicating that a bill could not be read from the storage backend by an instance of
weaver.store.BillStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
BillRegistrationError
[source]¶ Error indicating that a bill could not be registered in the storage backend by an instance of
weaver.store.BillStore
.Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.exceptions.
BillInstanceError
[source]¶ Error indicating that a given object doesn’t correspond to an expected instance of
weaver.datatype.Bill
.Initialize self. See help(type(self)) for accurate signature.
-
weaver.exceptions.
log_unhandled_exceptions
(logger: logging.Logger = LOGGER, message: AnyStr = 'Unhandled exception occurred.', exception: Type[Exception] = Exception, force: bool = False, require_http: bool = True, is_request: bool = True) → Callable[source]¶ Decorator that will raise
exception
with specifiedmessage
if an exception is caught while execution the wrapped function, after logging relevant details about the caught exception withlogger
.- Parameters
logger – logger to use for logging (default: use
weaver.exception.LOGGER
).message – message that will be logged with details and then raised with
exception
.exception – exception type to be raised instead of the caught exception.
force – force handling of any raised exception (default: only known unhandled exceptions are logged).
require_http – consider non HTTP-like exceptions as unknown and raise one instead (default:
True
and raisesHTTPInternalServerError
, unlessexception
is HTTP-like).is_request – specifies if the decorator is applied onto a registered request function to handle its inputs.
- Raises
exception – if an unknown exception was caught (or forced) during the decorated function’s execution.
Exception – original exception if it is known.
weaver.execute
¶
weaver.formats
¶
-
weaver.formats.
get_format
(mime_type: AnyStr) → Format[source]¶ Obtains a
Format
with predefined extension and encoding details from known MIME-types.
-
weaver.formats.
get_extension
(mime_type: AnyStr) → AnyStr[source]¶ Retrieves the extension corresponding to
mime_type
if explicitly defined, or by parsing it.
-
weaver.formats.
get_cwl_file_format
(mime_type: AnyStr, make_reference: bool = False, must_exist: bool = True, allow_synonym: bool = True) → Union[Tuple[Optional[JSON], Optional[AnyStr]], Optional[AnyStr]][source]¶ Obtains the corresponding IANA/EDAM
format
value to be applied under a CWL I/OFile
from themime_type
(Content-Type header) using the first matched one.Lookup procedure is as follows:
- If
make_reference=False
: - If there is a match, returns
tuple({<namespace-name: namespace-url>}, <format>)
with: corresponding namespace mapping to be applied under
$namespaces
in the CWL.value of
format
adjusted according to the namespace to be applied toFile
in the CWL.
- If there is a match, returns
If there is no match but
must_exist=False
, returns a literal and non-existing definition astuple({"iana": <iana-url>}, <format>)
.If there is no match but
must_exist=True
ANDallow_synonym=True
, retry the call with the synonym if available, or move to next step. Skip this step ifallow_synonym=False
.Otherwise, returns
(None, None)
- If
- If
make_reference=True
: If there is a match, returns the explicit format reference as
<namespace-url>/<format>
.If there is no match but
must_exist=False
, returns the literal reference as<iana-url>/<format>
(N.B.: literal non-official MIME-type reference will be returned even if an official synonym exists).If there is no match but
must_exist=True
ANDallow_synonym=True
, retry the call with the synonym if available, or move to next step. Skip this step ifallow_synonym=False
.Returns a single
None
as there is not match (directly or synonym).
- If
- Note:
In situations where
must_exist=False
is used and that the namespace and/or full format URL cannot be resolved to an existing reference, CWL will raise a validation error as it cannot confirm theformat
. You must therefore make sure that the returned reference (or a synonym format) really exists when usingmust_exist=False
before providing it to the CWL I/O definition. Settingmust_exist=False
should be used only for literal string comparison or pre-processing steps to evaluate formats.
- Parameters
mime_type – Some reference, namespace’d or literal (possibly extended) MIME-type string.
make_reference – Construct the full URL reference to the resolved MIME-type. Otherwise return tuple details.
must_exist – Return result only if it can be resolved to an official MIME-type (or synonym if enabled), otherwise
None
. Non-official MIME-type can be enforced if disabled, in which case IANA namespace/URL is used as it preserves the original<type>/<subtype>
format.allow_synonym – Allow resolution of non-official MIME-type to an official MIME-type synonym if available. Types defined as synonym have semantically the same format validation/resolution for CWL. Requires
must_exist=True
, otherwise the non-official MIME-type is employed directly as result.
- Returns
Resolved MIME-type format for CWL usage, accordingly to specified arguments (see description details).
-
weaver.formats.
clean_mime_type_format
(mime_type: AnyStr, suffix_subtype: bool = False, strip_parameters: bool = False) → AnyStr[source]¶ Removes any additional namespace key or URL from
mime_type
so that it corresponds to the generic representation (e.g.:application/json
) instead of the<namespace-name>:<format>
mapping variant used in CWL->inputs/outputs->File->format or the complete URL reference.According to provided arguments, it also cleans up additional parameters or extracts sub-type suffixes.
- Parameters
mime_type – MIME-type, full URL to MIME-type or namespace-formatted string that must be cleaned up.
suffix_subtype – Remove additional sub-type specializations details separated by
+
symbol such that an explicit format likeapplication/vnd.api+json
returns only its most basic suffix format defined as``application/json``.strip_parameters – Removes additional MIME-type parameters such that only the leading part defining the
type/subtype
are returned. For example, this will get rid of; charset=UTF-8
or; version=4.0
parameters.
Note
Parameters
suffix_subtype
andstrip_parameters
are not necessarily exclusive.
weaver.owsexceptions
¶
See also: https://github.com/geopython/pywps/blob/master/pywps/exceptions.py
-
exception
weaver.owsexceptions.
OWSException
(detail=None, value=None, **kw)[source]¶ Represents a WSGI response.
If no arguments are passed, creates a
Response
that uses a variety of defaults. The defaults may be changed by sub-classing theResponse
. See the sub-classing notes.- Variables
body (bytes or text_type) – If
body
is atext_type
, then it will be encoded using eithercharset
when provided ordefault_encoding
whencharset
is not provided if thecontent_type
allows for acharset
. This argument is mutually exclusive withapp_iter
.status (int or str) – Either an
int
or a string that is an integer followed by the status text. If it is an integer, it will be converted to a proper status that also includes the status text. Any existing status text will be kept. Non-standard values are allowed.headerlist (list) – A list of HTTP headers for the response.
app_iter (iterable) – An iterator that is used as the body of the response. Should conform to the WSGI requirements and should provide bytes. This argument is mutually exclusive with
body
.content_type (str or None) – Sets the
Content-Type
header. If nocontent_type
is provided, and there is noheaderlist
, thedefault_content_type
will be automatically set. Ifheaderlist
is provided then this value is ignored.conditional_response (bool) – Used to change the behavior of the
Response
to check the original request for conditional response headers. Seeconditional_response_app()
for more information.charset (str or None) – Adds a
charset
Content-Type
parameter. If nocharset
is provided and theContent-Type
is text, then thedefault_charset
will automatically be added. Currently the onlyContent-Type
’s that allow for acharset
are defined to betext/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added. If aheaderlist
is provided this value is ignored.
All other response attributes may be set on the response by providing them as keyword arguments. A
TypeError
will be raised for any unexpected keywords.Sub-classing notes:
The
default_content_type
is used as the default for theContent-Type
header that is returned on the response. It istext/html
.The
default_charset
is used as the default character set to return on theContent-Type
header, if theContent-Type
allows for acharset
parameter. Currently the onlyContent-Type
’s that allow for acharset
are defined to be:text/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added.The
unicode_errors
is set tostrict
, and access on atext
will raise an error if it fails to decode thebody
.default_conditional_response
is set toFalse
. This flag may be set toTrue
so that allResponse
objects will attempt to check the original request for conditional response headers. Seeconditional_response_app()
for more information.default_body_encoding
is set to ‘UTF-8’ by default. It exists to allow users to get/set theResponse
object using.text
, even if nocharset
has been set for theContent-Type
.
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.owsexceptions.
OWSAccessForbidden
(*args, **kwargs)[source]¶ Represents a WSGI response.
If no arguments are passed, creates a
Response
that uses a variety of defaults. The defaults may be changed by sub-classing theResponse
. See the sub-classing notes.- Variables
body (bytes or text_type) – If
body
is atext_type
, then it will be encoded using eithercharset
when provided ordefault_encoding
whencharset
is not provided if thecontent_type
allows for acharset
. This argument is mutually exclusive withapp_iter
.status (int or str) – Either an
int
or a string that is an integer followed by the status text. If it is an integer, it will be converted to a proper status that also includes the status text. Any existing status text will be kept. Non-standard values are allowed.headerlist (list) – A list of HTTP headers for the response.
app_iter (iterable) – An iterator that is used as the body of the response. Should conform to the WSGI requirements and should provide bytes. This argument is mutually exclusive with
body
.content_type (str or None) – Sets the
Content-Type
header. If nocontent_type
is provided, and there is noheaderlist
, thedefault_content_type
will be automatically set. Ifheaderlist
is provided then this value is ignored.conditional_response (bool) – Used to change the behavior of the
Response
to check the original request for conditional response headers. Seeconditional_response_app()
for more information.charset (str or None) – Adds a
charset
Content-Type
parameter. If nocharset
is provided and theContent-Type
is text, then thedefault_charset
will automatically be added. Currently the onlyContent-Type
’s that allow for acharset
are defined to betext/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added. If aheaderlist
is provided this value is ignored.
All other response attributes may be set on the response by providing them as keyword arguments. A
TypeError
will be raised for any unexpected keywords.Sub-classing notes:
The
default_content_type
is used as the default for theContent-Type
header that is returned on the response. It istext/html
.The
default_charset
is used as the default character set to return on theContent-Type
header, if theContent-Type
allows for acharset
parameter. Currently the onlyContent-Type
’s that allow for acharset
are defined to be:text/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added.The
unicode_errors
is set tostrict
, and access on atext
will raise an error if it fails to decode thebody
.default_conditional_response
is set toFalse
. This flag may be set toTrue
so that allResponse
objects will attempt to check the original request for conditional response headers. Seeconditional_response_app()
for more information.default_body_encoding
is set to ‘UTF-8’ by default. It exists to allow users to get/set theResponse
object using.text
, even if nocharset
has been set for theContent-Type
.
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.owsexceptions.
OWSNotFound
(*args, **kwargs)[source]¶ Represents a WSGI response.
If no arguments are passed, creates a
Response
that uses a variety of defaults. The defaults may be changed by sub-classing theResponse
. See the sub-classing notes.- Variables
body (bytes or text_type) – If
body
is atext_type
, then it will be encoded using eithercharset
when provided ordefault_encoding
whencharset
is not provided if thecontent_type
allows for acharset
. This argument is mutually exclusive withapp_iter
.status (int or str) – Either an
int
or a string that is an integer followed by the status text. If it is an integer, it will be converted to a proper status that also includes the status text. Any existing status text will be kept. Non-standard values are allowed.headerlist (list) – A list of HTTP headers for the response.
app_iter (iterable) – An iterator that is used as the body of the response. Should conform to the WSGI requirements and should provide bytes. This argument is mutually exclusive with
body
.content_type (str or None) – Sets the
Content-Type
header. If nocontent_type
is provided, and there is noheaderlist
, thedefault_content_type
will be automatically set. Ifheaderlist
is provided then this value is ignored.conditional_response (bool) – Used to change the behavior of the
Response
to check the original request for conditional response headers. Seeconditional_response_app()
for more information.charset (str or None) – Adds a
charset
Content-Type
parameter. If nocharset
is provided and theContent-Type
is text, then thedefault_charset
will automatically be added. Currently the onlyContent-Type
’s that allow for acharset
are defined to betext/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added. If aheaderlist
is provided this value is ignored.
All other response attributes may be set on the response by providing them as keyword arguments. A
TypeError
will be raised for any unexpected keywords.Sub-classing notes:
The
default_content_type
is used as the default for theContent-Type
header that is returned on the response. It istext/html
.The
default_charset
is used as the default character set to return on theContent-Type
header, if theContent-Type
allows for acharset
parameter. Currently the onlyContent-Type
’s that allow for acharset
are defined to be:text/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added.The
unicode_errors
is set tostrict
, and access on atext
will raise an error if it fails to decode thebody
.default_conditional_response
is set toFalse
. This flag may be set toTrue
so that allResponse
objects will attempt to check the original request for conditional response headers. Seeconditional_response_app()
for more information.default_body_encoding
is set to ‘UTF-8’ by default. It exists to allow users to get/set theResponse
object using.text
, even if nocharset
has been set for theContent-Type
.
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.owsexceptions.
OWSNotAcceptable
(*args, **kwargs)[source]¶ Represents a WSGI response.
If no arguments are passed, creates a
Response
that uses a variety of defaults. The defaults may be changed by sub-classing theResponse
. See the sub-classing notes.- Variables
body (bytes or text_type) – If
body
is atext_type
, then it will be encoded using eithercharset
when provided ordefault_encoding
whencharset
is not provided if thecontent_type
allows for acharset
. This argument is mutually exclusive withapp_iter
.status (int or str) – Either an
int
or a string that is an integer followed by the status text. If it is an integer, it will be converted to a proper status that also includes the status text. Any existing status text will be kept. Non-standard values are allowed.headerlist (list) – A list of HTTP headers for the response.
app_iter (iterable) – An iterator that is used as the body of the response. Should conform to the WSGI requirements and should provide bytes. This argument is mutually exclusive with
body
.content_type (str or None) – Sets the
Content-Type
header. If nocontent_type
is provided, and there is noheaderlist
, thedefault_content_type
will be automatically set. Ifheaderlist
is provided then this value is ignored.conditional_response (bool) – Used to change the behavior of the
Response
to check the original request for conditional response headers. Seeconditional_response_app()
for more information.charset (str or None) – Adds a
charset
Content-Type
parameter. If nocharset
is provided and theContent-Type
is text, then thedefault_charset
will automatically be added. Currently the onlyContent-Type
’s that allow for acharset
are defined to betext/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added. If aheaderlist
is provided this value is ignored.
All other response attributes may be set on the response by providing them as keyword arguments. A
TypeError
will be raised for any unexpected keywords.Sub-classing notes:
The
default_content_type
is used as the default for theContent-Type
header that is returned on the response. It istext/html
.The
default_charset
is used as the default character set to return on theContent-Type
header, if theContent-Type
allows for acharset
parameter. Currently the onlyContent-Type
’s that allow for acharset
are defined to be:text/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added.The
unicode_errors
is set tostrict
, and access on atext
will raise an error if it fails to decode thebody
.default_conditional_response
is set toFalse
. This flag may be set toTrue
so that allResponse
objects will attempt to check the original request for conditional response headers. Seeconditional_response_app()
for more information.default_body_encoding
is set to ‘UTF-8’ by default. It exists to allow users to get/set theResponse
object using.text
, even if nocharset
has been set for theContent-Type
.
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.owsexceptions.
OWSNoApplicableCode
(*args, **kwargs)[source]¶ WPS Bad Request Exception
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.owsexceptions.
OWSMissingParameterValue
(*args, **kwargs)[source]¶ MissingParameterValue WPS Exception
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.owsexceptions.
OWSInvalidParameterValue
(*args, **kwargs)[source]¶ InvalidParameterValue WPS Exception
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.owsexceptions.
OWSNotImplemented
(*args, **kwargs)[source]¶ Represents a WSGI response.
If no arguments are passed, creates a
Response
that uses a variety of defaults. The defaults may be changed by sub-classing theResponse
. See the sub-classing notes.- Variables
body (bytes or text_type) – If
body
is atext_type
, then it will be encoded using eithercharset
when provided ordefault_encoding
whencharset
is not provided if thecontent_type
allows for acharset
. This argument is mutually exclusive withapp_iter
.status (int or str) – Either an
int
or a string that is an integer followed by the status text. If it is an integer, it will be converted to a proper status that also includes the status text. Any existing status text will be kept. Non-standard values are allowed.headerlist (list) – A list of HTTP headers for the response.
app_iter (iterable) – An iterator that is used as the body of the response. Should conform to the WSGI requirements and should provide bytes. This argument is mutually exclusive with
body
.content_type (str or None) – Sets the
Content-Type
header. If nocontent_type
is provided, and there is noheaderlist
, thedefault_content_type
will be automatically set. Ifheaderlist
is provided then this value is ignored.conditional_response (bool) – Used to change the behavior of the
Response
to check the original request for conditional response headers. Seeconditional_response_app()
for more information.charset (str or None) – Adds a
charset
Content-Type
parameter. If nocharset
is provided and theContent-Type
is text, then thedefault_charset
will automatically be added. Currently the onlyContent-Type
’s that allow for acharset
are defined to betext/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added. If aheaderlist
is provided this value is ignored.
All other response attributes may be set on the response by providing them as keyword arguments. A
TypeError
will be raised for any unexpected keywords.Sub-classing notes:
The
default_content_type
is used as the default for theContent-Type
header that is returned on the response. It istext/html
.The
default_charset
is used as the default character set to return on theContent-Type
header, if theContent-Type
allows for acharset
parameter. Currently the onlyContent-Type
’s that allow for acharset
are defined to be:text/*
,application/xml
, and*/*+xml
. Any otherContent-Type
’s will not have acharset
added.The
unicode_errors
is set tostrict
, and access on atext
will raise an error if it fails to decode thebody
.default_conditional_response
is set toFalse
. This flag may be set toTrue
so that allResponse
objects will attempt to check the original request for conditional response headers. Seeconditional_response_app()
for more information.default_body_encoding
is set to ‘UTF-8’ by default. It exists to allow users to get/set theResponse
object using.text
, even if nocharset
has been set for theContent-Type
.
Initialize self. See help(type(self)) for accurate signature.
weaver.sort
¶
weaver.status
¶
-
weaver.status.
map_status
(wps_status: AnyStatusType, compliant: AnyStr = STATUS_COMPLIANT_OGC) → AnyStr[source]¶ Maps WPS statuses (weaver.status, OWSLib or PyWPS) to OWSLib/PyWPS compatible values. For each compliant combination, unsupported statuses are changed to corresponding ones (with closest logical match). Statuses are returned with weaver.status.JOB_STATUS_VALUES format (lowercase and not preceded by ‘Process’).
- Parameters
wps_status – one of weaver.status.JOB_STATUS_VALUES to map to compliant standard or PyWPS int status.
compliant – one of STATUS_COMPLIANT_[…] values.
- Returns
mapped status complying to the requested compliant category, or STATUS_UNKNOWN if no match found.
weaver.tweens
¶
-
weaver.tweens.
ows_response_tween
(request, handler)[source]¶ Tween that wraps any API request with appropriate dispatch of error conversion to handle formatting.
-
weaver.tweens.
ows_response_tween_factory_excview
(handler, registry)[source]¶ A tween factory which produces a tween which transforms common exceptions into OWS specific exceptions.
weaver.warning
¶
-
exception
weaver.warning.
TimeZoneInfoAlreadySetWarning
[source]¶ Warn when trying to obtain a localized time with already defined time-zone info.
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.warning.
DisabledSSLCertificateVerificationWarning
[source]¶ Warn when an option to disable SSL certificate verification is employed for some operations.
Initialize self. See help(type(self)) for accurate signature.
-
exception
weaver.warning.
UnsupportedOperationWarning
[source]¶ Warn about an operation not yet implemented or unsupported according to context.
Initialize self. See help(type(self)) for accurate signature.
weaver.wps
¶
-
weaver.wps.
_get_settings_or_wps_config
(container: AnySettingsContainer, weaver_setting_name: AnyStr, config_setting_section: AnyStr, config_setting_name: AnyStr, default_not_found: AnyStr, message_not_found: AnyStr) → AnyStr[source]¶
-
weaver.wps.
get_wps_path
(container: AnySettingsContainer) → AnyStr[source]¶ Retrieves the WPS path (without hostname). Searches directly in settings, then weaver.wps_cfg file, or finally, uses the default values if not found.
-
weaver.wps.
get_wps_url
(container: AnySettingsContainer) → AnyStr[source]¶ Retrieves the full WPS URL (hostname + WPS path). Searches directly in settings, then weaver.wps_cfg file, or finally, uses the default values if not found.
-
weaver.wps.
get_wps_output_dir
(container: AnySettingsContainer) → AnyStr[source]¶ Retrieves the WPS output directory path where to write XML and result files. Searches directly in settings, then weaver.wps_cfg file, or finally, uses the default values if not found.
-
weaver.wps.
get_wps_output_path
(container: AnySettingsContainer) → AnyStr[source]¶ Retrieves the WPS output path (without hostname) for staging XML status, logs and process outputs. Searches directly in settings, then weaver.wps_cfg file, or finally, uses the default values if not found.
-
weaver.wps.
get_wps_output_url
(container: AnySettingsContainer) → AnyStr[source]¶ Retrieves the WPS output URL that maps to WPS output directory path. Searches directly in settings, then weaver.wps_cfg file, or finally, uses the default values if not found.
-
weaver.wps.
get_wps_local_status_location
(url_status_location: AnyStr, container: AnySettingsContainer, must_exist: bool = True) → Optional[AnyStr][source]¶ Attempts to retrieve the local file path corresponding to the WPS status location as URL.
- Parameters
url_status_location – URL reference pointing to some WPS status location XML.
container – any settings container to map configured local paths.
must_exist – return only existing path if enabled, otherwise return the parsed value without validation.
- Returns
found local file path if it exists,
None
otherwise.
-
weaver.wps.
check_wps_status
(url: Optional[AnyStr] = None, response: Optional[etree.ElementBase] = None, sleep_secs: int = 2, verify: bool = True, settings: Optional[AnySettingsContainer] = None) → WPSExecution[source]¶ Run
owslib.wps.WPSExecution.checkStatus()
with additional exception handling.- Parameters
url – job URL where to look for job status.
response – WPS response document of job status.
sleep_secs – number of seconds to sleep before returning control to the caller.
verify – Flag to enable SSL verification.
settings – Application settings to retrieve any additional request parameters as applicable.
- Returns
OWSLib.wps.WPSExecution object.
- 1
Created with sphinx-autoapi
Changes¶
1.14.0 (2021-01-11)¶
Changes:¶
Add
data
input support for CWL Workflow step referring to WPS-3 Process.Add documentation example references to Application Package and Process
Deploy
/Execute
repositories.Add parsing of
providers
inwps_processes.yml
to directly register remote WPS providers that will dynamically fetch underlying WPS processes, instead of static per-service processes stored locally.Add field
visible
towps_processes.yml
entries to allow directly defining the registered processes visibility.Adjust response of remote provider processes to return the same format as local processes.
Fixes:¶
Fix
stdout
/stderr
log file not permitted directly within CWL Workflow (must be inside intermediate steps).Fix missing S3 bucket location constraint within unittests.
1.13.1 (2020-07-17)¶
Changes:¶
No change.
Fixes:¶
Create an
stdout.log
orstderr.log
file in casecwltool
hasn’t created it.
1.13.0 (2020-07-15)¶
Changes:¶
Add AWS S3 bucket support for process input reference files.
Add
weaver.wps_output_s3_bucket
setting to upload results to AWS S3 bucket instead of local directory.Add
weaver.wps_output_s3_region
setting to allow override parameter extracted from AWS profile otherwise.Add more documentation about supported file reference schemes.
Add documentation references to ESGF-CWT Compute API.
Add conditional input file reference fetching (depending on ADES/EMS, process type from CWL
hints
) to take advantage of request-options and all supported scheme formats by Weaver, instead of relying on PyWPS and/or CWL wherever how far downstream the URL reference was reaching.
Fixes:¶
Adjust some docstrings to better indicate raised errors.
Adjust
weaver.processes.wps_package.WpsPackage
to use its internal logger when running the process in order to preserve log entries under its job execution. They were otherwise lost over time across all process executions.
1.12.0 (2020-07-03)¶
Changes:¶
Add multiple CWL ESGF processes and workflows, namely
SubsetNASAESGF
,SubsetNASAESGF
and many more.Add tests for ESGF processes and workflows.
Add documentation for
ESGF-CWTRequirement
processes.Add
file2string_array
andmetalink2netcdf
builtins.Add
esgf_process
Wps1Process
extension, to handleESGF-CWTRequirement
processes and workflows.
Fixes:¶
Reset
MongoDatabase
connection when we are in a forked process.
1.11.0 (2020-07-02)¶
Changes:¶
Generate Weaver OpenAPI specification for readthedocs publication.
Add some sections for documentation (#61).
Add support of documentation RST file redirection to generated HTML for reference resolution in both Github source and Readthedocs served pages.
Improve documentation links, ReadTheDocs format and TOC references.
Avoid logging
stdout/stderr
in workflows.Add tests to make sure processes
stdout/stderr
are logged.Remove Python 2.7 version as not officially supported.
Move and update WPS status location and status check functions into
weaver.wps
module.
Fixes:¶
Fix reported WPS status location to handle when starting with
/
although not representing an absolute path.
1.10.1 (2020-06-03)¶
Changes:¶
No change.
Fixes:¶
Pin
celery==4.4.2
to avoid import error on missingfutures.utils
called internally in following versions.
1.10.0 (2020-06-03)¶
Changes:¶
Add support of value-typed metadata fields for process description.
Enforce
rel
field when specifying anhref
JSON link to match corresponding XML requirement.
1.9.0 (2020-06-01)¶
Changes:¶
Add
weaver.wps_workdir
configuration setting to define the location where the underlyingcwltool
application should be executed under. This can allow more control over the scope of the mounted volumes for Application Package running a docker image.Add mapping of WPS results from the
Job
’s UUID to generated PyWPS UUID for outputs, status and log locations.Add experimental configuration settings
weaver.cwl_euid
andweaver.cwl_egid
to provide effective user/group identifiers to employ when running the CWL Application Package. Using these require good control of the directory and process I/O locations as invalid permissions could break a previously working job execution.Add more logging configuration and apply them to
cwltool
before execution of Application Package.Enforce
no_match_user=False
andno_read_only=False
ofcwltool
’sRuntimeContext
to ensure that docker application is executed with same user asweaver
and that process input files are not modified inplace (readonly) where potentially inaccessible (according to settings). Definition of CWL package will need to add InitialWorkDirRequirement as per defined by reference specification to stage those files if they need to be accessed with write permissions (see: example). Addresses some issues listed in #155.Enforce removal of some invalid CWL hints/requirements that would break the behaviour offered by
Weaver
.Use
weaver.request_options
for WPS GetCapabilities and WPS Check Status requests under the running job.Change default
DOCKER_REPO
value defined inMakefile
to point to reference mentioned inREADME.md
and considered as official deployment location.Add
application/x-cwl
MIME-type supported with updatedEDAM 1.24
ontology.Add
application/x-yaml
MIME-type to known formats.Add
application/x-tar
andapplication/tar+gzip
MIME-type (not official) but resolved as synonymapplication/gzip
(official) to preserve compressed file support during CWL format validation.
Fixes:¶
Set
get_cwl_file_format
default argumentmust_exist=True
instead ofFalse
to retrieve original default behaviour of the function. Since CWL usually doesn’t need to addFile.format
field when no corresponding reference actually exists, this default also makes more sense.
1.8.1 (2020-05-22)¶
Changes:¶
Add Travis-CI smoke test of built docker images for early detection of invalid setup or breaking code to boot them.
Add Travis-CI checks for imports. This check was not validated previously although available.
Adjust
weaver.ini.example
to reflect working demo server configuration (employed by smoke test).Move
weaver
web application toweaver.app
to reduce chances of breakingsetup.py
installation from import errors due toweaver
dependencies not yet installed. Redirect to new location makes this change transparent when loaded with the usualweaver.ini
configuration.
Fixes:¶
Fix base docker image to install Python 3 development dependencies in order to compile requirements with expected environment Python version. Package
python-dev
for Python 2 was being installed instead.Fix failing docker image boot due to incorrectly placed
yaml
import during setup installation.Fix imports according to
Makefile
targetscheck-imports
andfix-imports
.Fix parsing of
PyWPS
metadata to correctly employ values provided byweaver.ini
.
1.8.0 (2020-05-21)¶
Changes:¶
Modify
weaver.utils.request_retry
toweaver.utils.request_extra
to include more requests functionality and reuse it across the whole code base.Add
requests_extra
SSL verification option using specific URL regex(es) matches from configuration settings.Add
file://
transport scheme support directly to utilityrequests_extra
to handle local file paths.Add file
weaver.request_options
INI configuration setting to specify per-request method/URL options.Add
requests_extra
support ofRetry-After
response header (if any available on429
status) which indicates how long to wait until next request to avoid automatically defined response right after.Add
weaver.wps_workdir
configuration setting with allow setting correspondingpywps.workdir
directory.
Fixes:¶
Modify
Dockerfile-manager
to run web application usingpserve
asgunicorn
doesn’t correctly handles worker options anymore when loaded formweaver.ini
with--paste
argument. Also simplifies the command which already required multiple patches such as reapplying the host/port binding from INI file.Fix handling of Literal Data I/O
type
when retrieved fromOWSLib.wps
object with remote WPS XML body.Adjust
make start
target to use newmake install-run
target which installs the dependencies and package in edition mode so that configuration files present locally can be employed for running the application. Previously, one would have to move their configurations to thesite-package
install location of the active Python.Fix
celery>4.2
not found because of application path modification.Fix invalid handling of
wps_processes.yml
reference inweaver.ini
when specified as relative path to configuration directory.Fix handling of
WPS<->CWL
I/O merge ofdata_format
field againstsupported_formats
withpywps>=4.2.4
.Fix installation of
yaml
-related packages for Python 2 backward compatibility.
1.7.0 (2020-05-15)¶
Changes:¶
Add additional status log for
EOImage
input modification with OpenSearch during process execution.Add captured
stderr/stdout
logging of underlying CWL application being executed to resultingJob
logs (addresses first step of #131).Use
weaver.utils.request_retry
in even more places and extend convenience arguments offered by it to adapt it to specific use cases.
Fixes:¶
Fix handling of WPS-REST output matching a JSON file for multiple-output format specified with a relative local path as specified by job output location. Only remote HTTP references where correctly parsed. Also avoid failing the job if the reference JSON parsing fails. It will simply return the original reference URL in this case without expanded data (relates to #25).
Fix CWL job logs to be timezone aware, just like most other logs that will report UTC time.
Fix JSON response parsing of remote provider processes.
Fix parsing of CWL ordered parsing when I/O is specified as shorthand
"<id>":"<type>" directly under the ``inputs
oroutputs
dictionary instead of extended sub-dictionary variant (fixes #137).
1.6.0 (2020-05-07)¶
Changes:¶
Reuse
weaver.utils.request_retry
function across a few locations that where essentially reimplementing the core functionality.Add even more failure-permissive request attempts when validating a MIME-type against IANA website.
Add auto-resolution of common extensions known under PyWPS as well as employing their specific encoding.
Add
geotiff
format type support via PyWPS (#100).Make WPS status check more resilient to failing WPS outputs location not found in case the directory path can be resolved to a valid local file representing the XML status (i.e.: don’t depend as much on the HTTP WPS output route).
Ensure backward support of generic/default
text/plain
I/O when extracted from a referenced WPS-1/2 XML remote process which provides insufficient format details. For CWL output generated from it, replace the glob pattern to match anything (<id>.*
) instead of<id>.txt
extracted fromtext/plain
to simulate MIME-type as*/*
. Issue log warning message for future use cases.
Fixes:¶
Fix invalid
AllowedValue
parsing when usingLiteralData
inputs that resulted inAnyValue
being parsed as a"None"
string. This was transparent in case of string inputs and breaking for other types like integer when they attempted conversion.Fix erroneous
Metadata
keywords passed down toowslib.wps.Metadata
objects in case of more verbose detailed not allowed by this implementation.Fix parsing of explicitly-typed optional array CWL I/O notation that was not considered (i.e.: using
type
as list with additional"null"
instead oftype: "<type>?"
shorthand).Fix parsing of MIME-type from
format
field to exclude additional parameters (e.g.:; charset=UTF-8
for remote IANA validation.
1.5.1 (2020-03-26)¶
Changes:¶
Add unittest of utility function
fetch_file
.Split some unittest utility functions to allow more reuse.
Fixes:¶
Fix invalid
retry
parameter not handled automatically by request.
1.5.0 (2020-03-25)¶
Changes:¶
Adjust incorrectly parsed href file reference as WPS complex input which resulted in failing location retrieval.
Partially address unnecessary fetch of file that has to be passed down to CWL, which will in turn request the file as required. Need update from PyWPS to resolve completely (#91, geopython/pywps#526).
Adjust WPS output results to use relative HTTP path in order to recompose the output URL if server settings change.
Support WPS output results as value (WPS literal data). Everything was considered an href file beforehand.
Add additional
timeout
andretry
during fetching of remote file for processjsonarray2netcdf
to avoid unnecessary failures during edge case connexion problems.Add support of
title
andversion
field ofbuiltin
processes.
Fixes:¶
Patch
builtin
process execution failing sincecwltool 2.x
update.Avoid long fetch operation using streamed request that defaulted to chuck size of 1. Now, we use an appropriate size according to available memory.
1.3.0 (2020-03-10)¶
Changes:¶
Provide a way to override the external URL reported by WPS-1/2 and WPS-REST via configuration settings allowing for more advanced server-side results in response bodies.
1.2.0 (2020-03-06)¶
Changes:¶
Add wps languages for other wps requests types:
DescribeProcess
andGetCapabilities
.
Fixes:¶
Fix a bug where the validation of
OneOf
items was casting the value to the first valid possibility. Now, it doesn’t change the value if it’s valid without casting it (and still casts it if it’s necessary to make it valid).
1.1.0 (2020-02-17)¶
Changes:¶
Simplify docker image generation and make base/manager/worker variants all available under the same docker repo docker-registry.crim.ca/ogc/weaver with different tags (#5).
Add planned future support of
Accept-Language
header for WPS-1/2 (geopython/OWSLib 0.20.0
) (#74).Improved job logs update with message and progress to allow better tracking of internal operations and/or problems.
Allow WPS builtin process
jsonarray2netcdf
to fetch a remote file.Change doc to point to DockerHub pavics/weaver images.
Adjust CI rule long-lasting failures until it gets patched by original reference (gitleaks-actions#3).
Fixes:¶
Fix readthedocs documentation generation.
Fix
.travis
docker image build condition.Fix
geopython/OWSLib>=0.19.1
requirement for Python 3.8 support (#62).Fix job update filling due to status location incorrectly resolved according to configured PyWPS output path.
1.0.0 (2020-01-28)¶
New Features:¶
Add
notification_email
field toJob
datatype that stores an encrypted email (according to settings) when provided in the job submission body (#44).Add ability to filter jobs with
notification_email
query parameter (#44).Add jobs statistics grouping by specific fields using comma-separated list
groups
query parameter (#46).Add some tests to evaluate new job search methods / grouping results and responses (#44, #46).
Add handling of multiple CWL field
format
forFile
type.Add missing ontology reference support for CWL field
format
by defaulting to IANA namespace.Add support for I/O
array
ofenum
(ie: multiple values ofAllowedValues
for a given input) (#30).Add support of
label
synonym astitle
for inputs and process description (CWL specifying alabel
will set it in WPS process) (#31)Add support of input
minOccurs
andmaxOccurs
asint
while maintainingstr
support (#14).Add conformance route with implementation links (#53).
Add additional landing page link details (#54).
Add
weaver.wps_restapi.colander_extras.DropableNoneSchema
to auto-handle some schema JSON deserialization.Add
weaver.wps_restapi.colander_extras.VariableMappingSchema
to auto-handle some schema JSON deserialization.
Changes:¶
Use
bump2version
and move all config undersetup.cfg
.Remove enforced
text/plain
for CWLFile
when missingformat
field.Replace bubbling up of too verbose unhandled exceptions (500 Internal Server Error) by summary message and additional internal logging for debugging the cause using an utility exception log decorator.
Use the same exception log decorator to simplify function definitions when HTTP exceptions are already handled.
Make
null
reference a singleton so that multiple instantiation calls all refer to the same instance and produce the expected behaviour of<x> is null
instead of hard-to-identify errors because of english syntax.Remove unused function
weaver.utils.replace_caps_url
and corresponding tests.Remove
weaver.processes.utils.jsonify_value
duplicated byweaver.processes.wps_package.complex2json
.Use more JSON body schema validation using API schema definitions deserialization defined by
weaver.datatype
.Enforce
builtin
processes registration on startup to receive applicable updates.Provide 2 separate docker images for Weaver manager and worker, corresponding to the EMS/ADES API and the
celery
job runner respectively.Update Apache license.
Fixes:¶
Adjust some typing definitions incorrectly specified.
Fix I/O field ordering preserved as specified in payload or loaded reference file.
Fix setting
minOccurs=0
when adefault
is specified in the corresponding CWL I/O (#17, #25).Fix incorrectly overridden
maxOccurs="unbounded"
bymaxOccurs="1"
when a partial array input definition is specified without explicitmaxOccurs
in WPS payload (#17, #25).Fix case where omitted
format[s]
in both CWL and WPS deploy bodies generated a process description with complex I/O (file) without requiredformats
field. Defaulttext/plain
format is now automatically added.Fix case where
format[s]
lists between CWL and WPS where incorrectly merged.Fix
metadata
field within a WPS I/O incorrectly parsed when provided by a WPS-1/2 XML process definition.Fix invalid JSON response formatting on failing schema validation of process deployment body.
Fix docker images to support
pserve
when usinggunicorn>=20.x
dropping support of--paste
config feature.Fix multiple Python 2/3 compatibility issues.
0.2.0 (2019-03-26)¶
Fixes to handle invalid key characters
"$"
and"."
during CWL package read/write operations to database.Fixes some invalid CWL package generation from WPS-1 references.
More cases handled for WPS-1 to CWL
WPS1Requirement
conversion (AllowedValues
,Default
,SupportedFormats
,minOccurs
,maxOccurs
).Add file format validation to generated CWL package from WPS-1 MIME-types.
Allow auto-deployment of WPS-REST processes from WPS-1 references specified by configuration.
Add many deployment and execution validation tests for
WPS1Requirement
.Add
builtin
application packages support for common operations.
0.1.3 (2019-03-07)¶
Add useful Makefile targets for deployment.
Add badges indications in
README.rst
for tracking from repo landing page.Fix security issue of PyYAML requirement.
Fix some execution issues for
Wps1Process
.Fix some API schema erroneous definitions.
Additional logging of unhandled errors.
Improve some typing definitions.
0.1.2 (2019-03-05)¶
Introduce
WPS1Requirement
and correspondingWps1Process
to run a WPS-1 process under CWL.Remove mongodb requirement, assume it is running on an external service or docker image.
Add some typing definitions.
Fix some problematic imports.
Fix some PEP8 issues and PyCharm warnings.
0.1.1 (2019-03-04)¶
Modify Dockerfile to use lighter
debian:latest
instead ofbirdhouse/bird-base:latest
.Modify Dockerfile to reduce build time by reusing built image layers (requirements installation mostly).
Make some buildout dependencies optional to also reduce build time and image size.
Some additional striping of deprecated or invalid items from Twitcher.
FAQ¶
This section present some commonly encountered use-cases and basic solutions regarding ADES/EMS operation or more specifically related to CWL specification.
How to specify the Docker image reference?¶
In most situations, the CommandLineTool
process will need to run a Docker image. Doing so is as simple as
adding the DockerRequirement
(reference) as follows to the Application Package definition:
{
"cwlVersion": "v1.0",
"requirements": {
"DockerRequirement": {
"dockerPull": "<docker-url>"
}
},
"inputs": ["<...>"],
"outputs": ["<...>"],
}
Note
The Docker image reference must be publicly accessible to allow CWL to pull it. Alternatively, a private Docker reference can be used if the image is locally available. The process will fail to execute if it cannot resolve the reference.
The Application Package can be provided during process deployment. Please refer to below references for more details.
See also
Fixing permission error on input files¶
Some processes expect their inputs to be writable (e.g.: ZIP files). When running an Application Package based on a Docker image, Weaver mounts the input files as volumes in read-only mode for security reasons. This causes these processes to immediately fail as the running user cannot override nor write temporary files in the same directory (where the volume was mounted to), as it is marked with read permissions.
To resolve this issue, the application developer should add the InitialWorkDirRequirement
(reference, example) to his CWL package definition. This tells CWL to stage the files into the
Docker container into the running directory where the user will be allowed to generate outputs, and therefore,
also allow edition of the inputs or generation of temporary files as when unpacking a compressed file.
As example, the CWL definition could be similar to the following:
{
"cwlVersion": "v1.0",
"class": "CommandLineTool",
"requirements": {
"DockerRequirement": {
"dockerPull": "<docker-url>"
},
"InitialWorkDirRequirement": {
"listing": [{
"entry": "$(inputs.input_file)",
"writable": true
}
]
}
},
"arguments": ["$(runtime.outdir)"],
"inputs": {
"input_file": {
"type": "File"
}
}
Note that $(inputs.input_file)
within InitialWorkDirRequirement
tells which input to resolve for staging using
the "writable": True
parameter. All files listed there will be mounted with write permissions into working runtime
directory of the executed Docker container.
Where can I find references to CWL specification and examples?¶
There exist multiple sources, but official ones provided below have a create amount of examples and are being continuously improved by the developers (including being updated according to changes).
See also below.
Where can I find examples of Process Deployment, Execution and Application Package definitions?¶
Many examples are reported in detail in the Examples chapter.
Glossary¶
- ADES
- Application Package
General term that refers to “what and how the :term:`Process` will execute”. Application Packages provide the core details about the execution methodology of the underlying operation the Process provides, and are therefore always contained within a Process definition. This is more specifically represented by a CWL specification in the case of Weaver implementation, but could technically be defined by another similar approach. See Application Package section for all relevant details.
- AWS
Amazon Web Services
- CWL
- Representation of the internal Application Package of the Process to provide execution methodology of the referenced Docker image or other supported definitions. See application-package section for further details.
- Docker
Containerized and isolated environment platform that allows all required dependencies of an application or software to be packaged in a single image in order to correctly execute the virtualized application.
- EDAM
Ontology that regroups multiple definitions, amongst which Weaver looks up some of its known and supported MIME-types (EDAM media types) when resolving file formats. It is used as extension to IANA media types by providing additional formats that are more specifics to some data domains.
- EMS
- Execution Management ServiceSee Processes section for details. Alternative mode is ADES, which can be selected as defined by the appropriate Configuration parameter.
- ESGF
- ESGF-CWT
- HREF
- Hyperlink ReferenceOften shortened to simply reference. Represents either a locally or remotely accessible item, such as a file or a Process depending on context, that uses explicit
<protocol>://<host/path>
representation to define its location. See also File Reference Types for typical examples. - I/O
Inputs and/or Outputs of CWL and/or WPS depending on context.
- IANA
Ontology that regroups multiple definitions, amongst which Weaver looks up most of its known and supported MIME-types (IANA media types) when resolving file formats.
- Job
Definition of a Process execution state with applicable operation metadata.
- JSON
- JavaScript Object NotationDefault data representation of all objects contained in the application or for their creation.
- MIME-types
- OGC
- OGC API - Processes
The new API that defines JSON REST-binding representation of WPS Process collection.
- OpenSearch
Protocol of lookup and retrieval of remotely stored files. Please refer to OpenSearch Data Source for details.
- Process
Entity that describes the required inputs, produced outputs, and any applicable metadata for the execution of the defined script, calculation, or operation.
- S3
Simple Storage Service (AWS S3), bucket file storage.
- WKT
Well-Known Text geometry representation.
- WPS
- Web Processing Service.From a formal standpoint, this is the previous OGC standard iteration that was employed prior to OGC API - Processes to represent a server that host one or more Process for execution. When compared against CWL context or generally across Weaver documentation and code, this term refers to attributes that are specific to typical Process description, in contrast to specialized attributes introduced by other concepts, such as for example CWL-specific implementation details.
- XML
- Extensible Markup LanguageAlternative representation of some data object provided by the application. Requires appropriate
Accept
header to return this format. See OpenAPI Specification for details.