weaver.processes.utils
Module Contents
- weaver.processes.utils.get_process(process_id: Optional[str] = None, request: Optional[weaver.typedefs.PyramidRequest] = None, settings: Optional[weaver.typedefs.SettingsType] = None, revision: bool = True) weaver.datatype.Process [source]
Obtain the specified process and validate information, returning appropriate HTTP error if invalid.
Process identifier must be provided from either the request path definition or literal ID. Database must be retrievable from either the request, underlying settings, or direct store reference.
Changed in version 4.20: Process identifier can also be an ‘id:version’ tag. Also, the request query parameter ‘version’ can be used. If using the
process_id
explicitly instead of the request, a versioned Process reference MUST employ the tagged representation to resolve the appropriate Process revision.Different parameter combinations are intended to be used as needed or more appropriate, such that redundant operations can be reduced where some objects are already fetched from previous operations.
- Parameters
process_id – Explicit Process identifier to employ for lookup.
request – When no explicit ID specified, try to find information from the request.
settings – Application settings for database connection. Can be guessed from local thread or request object if not given.
revision – When parsing the Process ID (either explicit or from request), indicate if any tagged revision specifier should be used or dropped.
- weaver.processes.utils.map_progress(progress: weaver.typedefs.Number, range_min: weaver.typedefs.Number, range_max: weaver.typedefs.Number) weaver.typedefs.Number [source]
Calculates the relative progression of the percentage process within min/max values.
- weaver.processes.utils.get_process_information(process_description: weaver.typedefs.JSON) weaver.typedefs.JSON [source]
Obtain the details for the process within its description considering various supported formats.
- weaver.processes.utils._check_deploy(payload: weaver.typedefs.JSON) Union[weaver.typedefs.ProcessDeployment, weaver.typedefs.CWL] [source]
Validate minimum deploy payload field requirements with exception handling.
- weaver.processes.utils._validate_deploy_process_info(process_info: weaver.typedefs.JSON, reference: Optional[str], package: Optional[weaver.typedefs.CWL], settings: weaver.typedefs.SettingsType, headers: Optional[weaver.typedefs.AnyHeadersContainer]) weaver.typedefs.JSON [source]
Obtain the process definition from deploy payload with exception handling.
- weaver.processes.utils.resolve_cwl_graph(package: weaver.typedefs.CWL) weaver.typedefs.CWL [source]
- weaver.processes.utils.deploy_process_from_payload(payload: Union[weaver.typedefs.JSON, str], container: Union[weaver.typedefs.AnySettingsContainer, weaver.typedefs.AnyRequestType], overwrite: Union[bool, weaver.datatype.Process] = False) pyramid.httpexceptions.HTTPException [source]
Deploy the process after resolution of all references and validation of the parameters from payload definition.
Adds a
weaver.datatype.Process
instance to storage using the provided JSONpayload
matchingweaver.wps_restapi.swagger_definitions.ProcessDescription
.- Parameters
payload – JSON payload that was specified during the process deployment request.
container – Container to retrieve application settings. If it is a
request
-like object, additional parameters may be used to identify the payload schema.overwrite – In case of a pure deployment (from scratch), indicates (using
bool
) whether to allow override of an existing process definition if conflict occurs. No versioning is applied in this case (full replacement). In case of an update deployment (from previous), indicates which process to be replaced with updated version. The new version should not conflict with another existing process version. If payload doesn’t provide a new version, the following MAJOR version from the specified overwrite process is used to define the new revision.
- Returns
HTTPOk if the process registration was successful.
- Raises
HTTPException – for any invalid process deployment step.
- weaver.processes.utils._save_deploy_process(process: weaver.datatype.Process, override: bool, container: weaver.typedefs.AnySettingsContainer) weaver.typedefs.JSON [source]
Store the
Process
to database with error handling and appropriate message reporting the problem.
- weaver.processes.utils._update_deploy_process_version(process: weaver.datatype.Process, process_overwrite: weaver.datatype.Process, update_level: weaver.utils.VersionLevel, container: Optional[weaver.typedefs.AnySettingsContainer] = None) weaver.typedefs.JSON [source]
Handle all necessary update operations of a Process definition.
Validate that any specified version for Process deployment is valid against any other existing versions. Perform any necessary database adjustments to replace the old Process references for the creation of the updated Process to ensure all versions and links remain valid against their original references.
- Parameters
process – Desired new process definition.
process_overwrite – Old process from which update of the definition in database could be required.
update_level – Minimum semantic version level required for this update operation. If the new Process definition did not provide a version explicitly, this level will be used to automatically infer the following revision number based on the old Process reference.
container – Any container to retrieve a database connection.
- Returns
Process summary with definition retrieved from storage (saved) after all operations were applied.
- Raises
HTTPException – Relevant error is raised in the even of any erroneous process definition (old and new).
- weaver.processes.utils._bump_process_version(version: weaver.typedefs.AnyVersion, update_level: weaver.utils.VersionLevel) weaver.typedefs.AnyVersion [source]
Obtain the relevant version with specified level incremented by one.
- weaver.processes.utils._apply_process_metadata(process: weaver.datatype.Process, update_data: weaver.typedefs.JSON) weaver.utils.VersionLevel [source]
Apply requested changes for update of the Process.
Assumes that update data was pre-validated with appropriate schema validation to guarantee relevant typings and formats are applied for expected fields. Validation of fields metadata with their specific conditions is accomplished when attempting to apply changes.
See also
Schema
sd.PatchProcessBodySchema
describes specific field handling based on unspecified value, null or empty-list. Corresponding update levels required for fields are also provided in this schema definition.- Parameters
process – Process to modify. Can be the latest or a previously tagged version.
update_data – Fields with updated data to apply to the process.
- Returns
Applicable update level based on updates to be applied.
- weaver.processes.utils.update_process_metadata(request: weaver.typedefs.AnyRequestType) pyramid.httpexceptions.HTTPException [source]
Update only MINOR or PATCH level Process metadata.
Desired new version can be eiter specified explicitly in request payload, or will be guessed accordingly to detected changes to be applied.
- weaver.processes.utils.parse_wps_process_config(config_entry: Union[weaver.typedefs.JSON, str]) Tuple[str, str, List[str], bool] [source]
Parses the available WPS provider or process entry to retrieve its relevant information.
- Returns
WPS provider name, WPS service URL, and list of process identifier(s).
- Raises
ValueError – if the entry cannot be parsed correctly.
- weaver.processes.utils.register_wps_processes_static(service_url: str, service_name: str, service_visibility: bool, service_processes: List[str], container: weaver.typedefs.AnyRegistryContainer) None [source]
Register WPS-1 Process under a service Provider as static references.
For a given WPS provider endpoint, either iterates over all available processes under it to register them one by one, or limit itself only to those of the reduced set specified by
service_processes
.The registered WPS-1 processes generate a static reference, meaning that metadata of each process as well as any other modifications to the real remote reference will not be tracked, including validation of even their actual existence, or modifications to inputs/outputs. The Application Package will only point to it assuming it remains valid.
Each of the deployed processes using static reference will be accessible directly under Weaver endpoints:
/processes/<service-name>_<process-id>
The service is NOT deployed as Provider since the processes are registered directly.
See also
- Parameters
service_url – WPS-1 service location (where
GetCapabilities
andDescribeProcess
requests can be made).service_name – Identifier to employ for generating the full process identifier.
service_visibility – Visibility flag of the provider.
service_processes – process IDs under the service to be registered, or all if empty.
container – settings to retrieve required configuration settings.
- weaver.processes.utils.register_wps_processes_dynamic(service_name: str, service_url: str, service_visibility: bool, container: weaver.typedefs.AnyRegistryContainer) None [source]
Register a WPS service
provider
such thatprocesses
under it are dynamically accessible on demand.The registered WPS-1 provider generates a dynamic reference to processes under it. Only the Provider reference itself is actually registered. No Process are directly registered following this operation.
When information about the offered processes, descriptions of those processes or their execution are requested, Weaver will query the referenced Provider for details and convert the corresponding Process dynamically. This means that latest metadata of the Process, and any modification to it on the remote service will be immediately reflected on Weaver without any need to re-deploy processes.
Each of the deployed processes using dynamic reference will be accessible under Weaver endpoints:
/providers/<service-name>/processes/<process-id>
The processes are NOT deployed locally since the processes are retrieved from the Provider itself.
See also
- Parameters
service_url – WPS-1 service location (where
GetCapabilities
andDescribeProcess
requests can be made).service_name – Identifier to employ for registering the provider identifier.
service_visibility – Visibility flag of the provider.
container – settings to retrieve required configuration settings.
- weaver.processes.utils.register_wps_processes_from_config(container: weaver.typedefs.AnySettingsContainer, wps_processes_file_path: Optional[weaver.typedefs.FileSystemPathType] = None) None [source]
Registers remote WPS providers and/or processes as specified from the configuration file.
Loads a
wps_processes.yml
file and registers processes under `WPS-1/2`_ providers to the current Weaver instance as equivalent OGC API - Processes instances.References listed under
processes
are registered statically (by themselves, unchanging snapshot). References listed underproviders
, the WPS themselves are registered, making each Process listed in theirGetCapabilities
available. In this case, registered processes are defined dynamically, meaning they will be fetched on the provider each time a request refers to them, keeping their definition up-to-date with the remote server.New in version 1.14: When references are specified using
providers
section instead ofprocesses
, the registration only saves the remote WPS provider endpoint to dynamically populate WPS processes on demand. Previous behavior was to register each WPS process individually with ID[service]_[process]
.Changed in version 4.19: Parameter position are inverted. If
wps_processes_file_path
is explicitly provided, it is used directly without considering settings. Otherwise, automatically employ the definition in setting:weaver.wps_processes_file
.See also
weaver.wps_processes.yml.example for additional file format details.
Note
Settings with an explicit empty
weaver.wps_processes_file
entry will be considered as nothing to load. If the entry is omitted, default locationWEAVER_DEFAULT_WPS_PROCESSES_CONFIG
is attempted instead.- Parameters
container – Registry container to obtain database reference as well as application settings.
wps_processes_file_path – Override file path to employ instead of default settings definition.
- weaver.processes.utils._check_package_file(cwl_file_path_or_url: str) str [source]
Validates that the specified CWL file path or URL points to an existing and allowed file format.
- Parameters
cwl_file_path_or_url – one of allowed file types path on disk, or an URL pointing to one served somewhere.
- Returns
validated absolute path or URL of the file reference.
- Raises
PackageRegistrationError – in case of missing file, invalid format or invalid HTTP status code.
- weaver.processes.utils.is_cwl_package(package: Any) bool [source]
Perform minimal validation of a CWL package definition.
- weaver.processes.utils.load_package_file(file_path: str) weaver.typedefs.CWL [source]
Loads the package in YAML/JSON format specified by the file path.
- weaver.processes.utils.register_cwl_processes_from_config(container: weaver.typedefs.AnySettingsContainer) int [source]
Load multiple CWL definitions from a directory to register corresponding Process.
New in version 4.19.
Each individual CWL definition must fully describe a Process by itself. Therefore, an
id
must be available in the file to indicate the target deployment reference. In case of conflict, the existing database Process will be overridden to ensure file updates are applied.Files are loaded in alphabetical order. If a Workflow needs to refer to other processes, they should be named in way that dependencies will be resolvable prior to the registration of the Workflow Process. The resolved directory to search for CWL will be traversed recursively. This, along with the name of CWL files themselves, can be used to resolve order-dependent loading cases. Only
.cwl
extensions are considered to avoid invalid parsing of other files that could be defined in the shared configuration directory.Note
Settings with an explicit empty
weaver.cwl_processes_dir
entry will be considered as nothing to load. If the entry is omitted, default locationWEAVER_CONFIG_DIR
is used to search for CWL files.- Parameters
container – Registry container to obtain database reference as well as application settings.
- Returns
Number of successfully registered processes from found CWL files.