weaver.store.mongodb

Stores to read/write data to from/to MongoDB using pymongo.

Module Contents

weaver.store.mongodb.LOGGER[source]
class weaver.store.mongodb.MongodbStore(collection: Optional[Dict[str, Any]], sane_name_config=None)[source]

Base class extended by all concrete store implementations.

classmethod get_args_kwargs(cls: Any, *args: Any, **kwargs)Tuple[Tuple, Dict][source]

Filters MongodbStore-specific arguments to safely pass them down its __init__.

class weaver.store.mongodb.MongodbServiceStore(*args, **kwargs)[source]

Registry for OWS services.

Uses MongoDB to store service url and attributes.

save_service(self: weaver.datatype.Service, service: bool, overwrite=True)weaver.datatype.Service[source]

Stores an OWS service in mongodb.

delete_service(self: str, name)bool[source]

Removes service from MongoDB storage.

list_services(self)List[weaver.datatype.Service][source]

Lists all services in MongoDB storage.

fetch_by_name(self: str, name: Optional[str], visibility=None)weaver.datatype.Service[source]

Gets service for given name from MongoDB storage.

fetch_by_url(self: str, url)weaver.datatype.Service[source]

Gets service for given url from MongoDB storage.

clear_services(self)bool[source]

Removes all OWS services from MongoDB storage.

class weaver.store.mongodb.MongodbProcessStore(*args, **kwargs)[source]

Registry for processes.

Uses MongoDB to store processes and attributes.

_register_defaults(self: List[weaver.datatype.Process], processes)None[source]

Default process registration to apply definition updates with duplicate entry handling.

_add_process(self: weaver.typedefs.AnyProcess, process: bool, upsert=False)None[source]

Stores the specified process to the database.

The operation assumes that any conflicting or duplicate process definition was pre-validated. Parameter upsert=True can be employed to allow exact replacement and ignoring duplicate errors. When using upsert=True, it is assumed that whichever the result (insert, update, duplicate error) arises, the final result of the stored process should be identical in each case.

Note

Parameter upsert=True is useful for initialization-time of the storage with default processes that can sporadically generate clashing-inserts between multi-threaded/workers applications that all try adding builtin processes around the same moment.

static _get_process_field(process: weaver.typedefs.AnyProcess, function_dict: Union[Dict[weaver.typedefs.AnyProcessType, Callable[], Any]], Callable[], Any]])Any[source]

Obtain a field from a process instance after validation and using mapping of process implementation functions.

Takes a lambda expression or a dict of process-specific lambda expressions to retrieve a field. Validates that the passed process object is one of the supported types.

Parameters
  • process – process to retrieve the field from.

  • function_dict – lambda or dict of lambda of process type.

Returns

retrieved field if the type was supported.

Raises

ProcessInstanceError – invalid process type.

_get_process_id(self: weaver.typedefs.AnyProcess, process)str[source]
_get_process_type(self: weaver.typedefs.AnyProcess, process)str[source]
_get_process_endpoint_wps1(self: weaver.typedefs.AnyProcess, process)str[source]
save_process(self: Union[weaver.datatype.Process, pywps.Process], process: bool, overwrite=True)weaver.datatype.Process[source]

Stores a process in storage.

Parameters
  • process – An instance of weaver.datatype.Process.

  • overwrite – Overwrite the matching process instance by name if conflicting.

delete_process(self: str, process_id: Optional[str], visibility=None)bool[source]

Removes process from database, optionally filtered by visibility.

If visibility=None, the process is deleted (if existing) regardless of its visibility value.

list_processes(self: Optional[str], visibility=None)List[weaver.datatype.Process][source]

Lists all processes in database, optionally filtered by visibility.

Parameters

visibility – One value amongst weaver.visibility.

fetch_by_id(self: str, process_id: Optional[str], visibility=None)weaver.datatype.Process[source]

Get process for given process_id from storage, optionally filtered by visibility.

If visibility=None, the process is retrieved (if existing) regardless of its visibility value.

Parameters
  • process_id – process identifier

  • visibility – one value amongst weaver.visibility.

Returns

An instance of weaver.datatype.Process.

get_visibility(self: str, process_id)str[source]

Get visibility of a process.

Returns

One value amongst weaver.visibility.

set_visibility(self: str, process_id: str, visibility)None[source]

Set visibility of a process.

Parameters
  • visibility – One value amongst weaver.visibility.

  • process_id

Raises
clear_processes(self)bool[source]

Clears all processes from the store.

class weaver.store.mongodb.MongodbJobStore(*args, **kwargs)[source]

Registry for process jobs tracking.

Uses MongoDB to store job attributes.

save_job(self: Ellipsis, task_id: str, process: str, service: Optional[str] = None, inputs: Optional[List[Any]] = None, is_workflow: bool = False, is_local: bool = False, user_id: Optional[int] = None, execute_async: bool = True, custom_tags: Optional[List[str]] = None, access: Optional[str] = None, notification_email: Optional[str] = None, accept_language: Optional[str] = None, created: Optional[datetime.datetime] = None)weaver.datatype.Job[source]

Creates a new Job and stores it in mongodb.

update_job(self: weaver.datatype.Job, job)weaver.datatype.Job[source]

Updates a job parameters in MongoDB storage.

Parameters

job – instance of weaver.datatype.Job.

delete_job(self: str, job_id)bool[source]

Removes job from MongoDB storage.

fetch_by_id(self: str, job_id)weaver.datatype.Job[source]

Gets job for given job_id from MongoDB storage.

list_jobs(self)List[weaver.datatype.Job][source]

Lists all jobs in MongoDB storage.

For user-specific access to available jobs, use MongodbJobStore.find_jobs() instead.

find_jobs(self: Ellipsis, process: Optional[str] = None, service: Optional[str] = None, tags: Optional[List[str]] = None, access: Optional[str] = None, notification_email: Optional[str] = None, status: Optional[str] = None, sort: Optional[str] = None, page: int = 0, limit: int = 10, datetime: Optional[weaver.store.base.DatetimeIntervalType] = None, group_by: Optional[Union[str, List[str]]] = None, request: Optional[pyramid.request.Request] = None)Union[weaver.store.base.JobListAndCount, weaver.store.base.JobCategoriesAndCount][source]

Finds all jobs in MongoDB storage matching search filters to obtain results with requested paging or grouping.

Using paging (default), result will be in the form:

(
    [Job(1), Job(2), Job(3), ...],
    <total>
)

Where <total> will indicate the complete count of matched jobs with filters, but the list of jobs will be limited only to page index and limit specified.

Using grouping with a list of field specified with group_by, results will be in the form:

(
    [{category: {field1: valueA, field2: valueB, ...}, [Job(1), Job(2), ...], count: <count>},
     {category: {field1: valueC, field2: valueD, ...}, [Job(x), Job(y), ...], count: <count>},
     ...
    ],
    <total>
)

Where <total> will again indicate all matched jobs by every category combined, and <count> will indicate the amount of jobs matched for each individual category. Also, category will indicate values of specified fields (from group_by) that compose corresponding jobs with matching values.

Parameters
  • request – request that lead to this call to obtain permissions and user id.

  • process – process name to filter matching jobs.

  • service – service name to filter matching jobs.

  • tags – list of tags to filter matching jobs.

  • access – access visibility to filter matching jobs (default: VISIBILITY_PUBLIC).

  • notification_email – notification email to filter matching jobs.

  • status – status to filter matching jobs.

  • sort – field which is used for sorting results (default: creation date, descending).

  • page – page number to return when using result paging (only when not using group_by).

  • limit – number of jobs per page when using result paging (only when not using group_by).

  • datetime – field used for filtering data by creation date with a given date or interval of date.

  • group_by – one or many fields specifying categories to form matching groups of jobs (paging disabled).

Returns

(list of jobs matching paging OR list of {categories, list of jobs, count}) AND total of matched job.

clear_jobs(self)bool[source]

Removes all jobs from MongoDB storage.

class weaver.store.mongodb.MongodbQuoteStore(*args, **kwargs)[source]

Registry for quotes.

Uses MongoDB to store quote attributes.

save_quote(self: weaver.datatype.Quote, quote)weaver.datatype.Quote[source]

Stores a quote in mongodb.

fetch_by_id(self: str, quote_id)weaver.datatype.Quote[source]

Gets quote for given quote_id from MongoDB storage.

list_quotes(self: Ellipsis)List[weaver.datatype.Quote][source]

Lists all quotes in MongoDB storage.

find_quotes(self: Optional[str], process_id: int = None, page: int = 0, limit: Optional[str] = 10, sort=None)Tuple[List[weaver.datatype.Quote], int][source]

Finds all quotes in MongoDB storage matching search filters.

Returns a tuple of filtered items and their count, where items can have paging and be limited to a maximum per page, but count always indicate the total number of matches.

class weaver.store.mongodb.MongodbBillStore(*args, **kwargs)[source]

Registry for bills.

Uses MongoDB to store bill attributes.

save_bill(self: weaver.datatype.Bill, bill)weaver.datatype.Bill[source]

Stores a bill in mongodb.

fetch_by_id(self: str, bill_id)weaver.datatype.Bill[source]

Gets bill for given bill_id from MongoDB storage.

list_bills(self: Ellipsis)List[weaver.datatype.Bill][source]

Lists all bills in MongoDB storage.

find_bills(self: Optional[str], quote_id: int = None, page: int = 0, limit: Optional[str] = 10, sort=None)Tuple[List[weaver.datatype.Bill], int][source]

Finds all bills in MongoDB storage matching search filters.

Returns a tuple of filtered items and their count, where items can have paging and be limited to a maximum per page, but count always indicate the total number of matches.