weaver.processes.execution

Module Contents

weaver.processes.execution.LOGGER[source]
weaver.processes.execution.JOB_PROGRESS_SETUP = 1[source]
weaver.processes.execution.JOB_PROGRESS_DESCRIBE = 2[source]
weaver.processes.execution.JOB_PROGRESS_GET_INPUTS = 3[source]
weaver.processes.execution.JOB_PROGRESS_GET_OUTPUTS = 4[source]
weaver.processes.execution.JOB_PROGRESS_EXECUTE_REQUEST = 5[source]
weaver.processes.execution.JOB_PROGRESS_EXECUTE_STATUS_LOCATION = 6[source]
weaver.processes.execution.JOB_PROGRESS_EXECUTE_MONITOR_START = 7[source]
weaver.processes.execution.JOB_PROGRESS_EXECUTE_MONITOR_LOOP = 8[source]
weaver.processes.execution.JOB_PROGRESS_EXECUTE_MONITOR_DONE = 96[source]
weaver.processes.execution.JOB_PROGRESS_EXECUTE_MONITOR_END = 98[source]
weaver.processes.execution.JOB_PROGRESS_NOTIFY = 99[source]
weaver.processes.execution.JOB_PROGRESS_DONE = 100[source]
weaver.processes.execution.execute_process(self, job_id, wps_url, headers=None)[source]

Celery task that executes the WPS process job monitoring as status updates (local and remote).

weaver.processes.execution.fetch_wps_process(job, wps_url, headers, settings)[source]

Retrieves the WPS process description from the local or remote WPS reference URL.

weaver.processes.execution.parse_wps_inputs(wps_process, job)[source]

Parses expected WPS process inputs against submitted job input values considering supported process definitions.

weaver.processes.execution.send_job_complete_notification_email(job, task_logger, settings)[source]

Sends the notification email of completed execution if it was requested during job submission.

weaver.processes.execution.make_results_relative(results: List[weaver.typedefs.JSON], settings: weaver.typedefs.SettingsType)List[weaver.typedefs.JSON][source]

Converts file references to a pseudo-relative location to allow the application to dynamically generate paths.

Redefines job results to be saved in database as pseudo-relative paths to configured WPS output directory. This allows the application to easily adjust the exposed result HTTP path according to the service configuration (i.e.: relative to weaver.wps_output_dir and/or weaver.wps_output_url) and it also avoids rewriting the database job results entry if those settings are changed later on following reboot of the web application.

Only references prefixed with weaver.wps_output_dir, weaver.wps_output_url or a corresponding resolution from weaver.wps_output_path with weaver.url will be modified to pseudo-relative paths. Other references (file/URL endpoints that do not correspond to Weaver) will be left untouched for literal remote reference. Results that do not correspond to a reference are also unmodified.

Note

The references are not real relative paths (i.e.: starting with ./), as those could also be specified as input, and there would be no way to guarantee proper differentiation from paths already handled and stored in the database. Instead, pseudo-relative paths employ an explicit absolute-like path (i.e.: starting with /) and are assumed to always require to be prefixed by the configured WPS locations (i.e.: weaver.wps_output_dir or weaver.wps_output_url based on local or HTTP response context).

With this approach, data persistence with mapped volumes into the dockerized Weaver service can be placed anywhere at convenience. This is important because sibling docker execution require exact mappings such that volume mount /data/path:/data/path resolve correctly on both sides (host and image path must be identical). If volumes get remapped differently, ensuring that weaver.wps_output_dir setting follows the same remapping update will automatically resolve to the proper location for both local references and exposed URL endpoints.

Parameters
  • results – JSON mapping of data results as {"<id>": <definition>} entries where a reference can be found.

  • settings – container to retrieve current application settings.

weaver.processes.execution.map_locations(job: weaver.datatype.Job, settings: weaver.typedefs.SettingsType)None[source]

Maps directory locations between pywps process execution and produced jobs storage.

Generates symlink references from the Job UUID to PyWPS UUID results (outputs directory, status and log locations). Update the Job’s WPS ID if applicable (job executed locally). Assumes that all results are located under the same reference UUID.

weaver.processes.execution.submit_job(request: pyramid.request.Request, reference: Union[weaver.datatype.Service, weaver.datatype.Process], tags: Optional[List[str]] = None)weaver.typedefs.JSON[source]

Generates the job submission from details retrieved in the request.

See also

submit_job_handler() to provide elements pre-extracted from requests or from other parsing.

weaver.processes.execution._validate_job_parameters(json_body)[source]

Tests supported parameters not automatically validated by colander deserialize.

weaver.processes.execution.submit_job_handler(payload: Ellipsis, settings: weaver.typedefs.SettingsType, service_url: str, provider_id: Optional[str] = None, process_id: str = None, is_workflow: bool = False, is_local: bool = True, visibility: Optional[str] = None, language: Optional[str] = None, auth: Optional[weaver.typedefs.HeaderCookiesType] = None, tags: Optional[List[str]] = None, user: Optional[int] = None, context: Optional[str] = None)weaver.typedefs.JSON[source]

Submits the job to the Celery worker with provided parameters.

Assumes that parameters have been pre-fetched and validated, except for the input payload.