Richard Law
60202129ec
* add process manager * process manager updating * control connectivity from inside manager * missing method on execute_process function signature * adds tinydb depenedency for requirements-provider * adds some processes tests * process manager under server config, not a process * additional html rendering for jobs and processes * fixes typo in docstring * complete html templating for jobs and processes * considers timezone when formatting processing start/end datetimes * expanded locales; locale and tz environment variables * makes environment TZ a global from pygeoapi/__init__.py * don't redefine built-in format * adds placeholder values in process form if given as examples * placeholder in form handles null example * changes to process progress update * updates and changes to processing web UI * adds python3-distutils dep, and env changes for TZs and locales * adds tinydb dependency for processes * Adds libpq-dev and related deps for processing - should eventually be reconciled * minor margin change * fix bug in postJob.js that prevented results from displaying * move process elements around to economise on space * adds second, complex process * post data may be bytes * make external process example * adds redis process manager (may be moved to third-party plugin eventually) * reintroduce default async processing; refactoring managers; improving sync/async processing ui * throw pygeoapi ManagerExecuteError, not redis exception * adds test for async-execute * restores original test host * separates async execute handler function * adds support for file uploads as processing inputs * adds endpoint for file download as process output * naming consistency * Fix job submission form in case server url is relative (#549) `new URL()` fails if the first argument isn't a complete url * Implement job deletion in UI, api and tinydb * Also allow deletion from job list The UI could be fancier by reloading the page or just removing the deleted job right away, but I don't want to start building that now. * Delete output file when deleting a job * typo in test case * move delete button in job view * remove Dockerfile changes (cf #470) * removes some docs/source*.rst absent in master * fixes whitespace * fixes pip installs * removes more locale-specific content, cf 70 * fixes duplicate app following merge * removes redis as default plugin * removes ENV_TZ, cf #407 * fix typo * fixes syntax error * process execution test cleanup * Fix format detection in execute_process (#589) Before, html was always selected. * remove db files in code directories, fix flake8, specific functionality * remove interactive capabilities in lieu of Swagger * Add basic openapi documentation for job detail endpoint (#590) It's somewhat superficial still, e.g. jobId is missing as proper parameter in the UI. * Show elapsed time for currently running jobs (#591) * flake8, jobs workflow * update docs, DB path in test config * more code cleanup * manager workflow cleanup * add default dummy manager * update docs * fix routes * fixed per comments * fix per comments * fix per comments * fix travis * simply describe process logic * simplify describe process logic Co-authored-by: Tom Kralidis <tomkralidis@gmail.com> Co-authored-by: Jorge de Jesus <jorge.jesus@gmail.com> Co-authored-by: totycro <bernhard.mallinger@eox.at> Co-authored-by: Richard Law <richard.law@cartoncloud.com.au>
79 lines
3.3 KiB
ReStructuredText
79 lines
3.3 KiB
ReStructuredText
.. _ogcapi-processes:
|
|
|
|
Publishing processes via OGC API - Processes
|
|
============================================
|
|
|
|
`OGC API - Processes`_ provides geospatial data processing functionality in a standards-based
|
|
fashion (inputs, outputs).
|
|
|
|
pygeoapi implements OGC API - Processes functionality by providing a plugin architecture, thereby
|
|
allowing developers to implement custom processing workflows in Python.
|
|
|
|
A `sample`_ ``hello-world`` process is provided with the pygeoapi default configuration.
|
|
|
|
Configuration
|
|
-------------
|
|
|
|
.. code-block:: yaml
|
|
|
|
processes:
|
|
hello-world:
|
|
processor:
|
|
name: HelloWorld
|
|
|
|
Asynchronous support
|
|
--------------------
|
|
|
|
By default, pygeoapi implements process execution (jobs) as synchronous mode. That is, when
|
|
jobs are submitted, the process is executed and returned in real-time. Certain processes
|
|
that may take time to execute, or be delegated to a scheduler/queue, are better suited to
|
|
an asynchronous design pattern. This means that when a job is submitted in asynchronous
|
|
mode, the server responds immediately with a reference to the job, which allows the client
|
|
to periodically poll the server for the processing status of a given job.
|
|
|
|
pygeoapi provides asynchronous support by providing a 'manager' concept which, well,
|
|
manages job execution. The manager concept is implemented as part of the pygeoapi
|
|
:ref:`plugins` architecture. pygeoapi provides a default manager implementation
|
|
based on `TinyDB`_ for simplicity. Custom manager plugins can be developed for more
|
|
advanced job management capabilities (e.g. Kubernetes, databases, etc.).
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
server:
|
|
manager:
|
|
name: TinyDB
|
|
connection: /tmp/pygeoapi-process-manager.db
|
|
output_dir: /tmp/
|
|
|
|
|
|
Putting it all together
|
|
-----------------------
|
|
|
|
To summarize how pygeoapi processes and managers work together::
|
|
|
|
- process plugins implement the core processing / workflow functionality
|
|
- manager plugins control and manage how processes are executed
|
|
|
|
Processing examples
|
|
-------------------
|
|
|
|
- list all processes
|
|
- http://localhost:5000/processes
|
|
- describe the ``hello-world`` process
|
|
- http://localhost:5000/processes/hello-world
|
|
- show all jobs for the ``hello-world`` process
|
|
- http://localhost:5000/processes/hello-world/jobs
|
|
- execute a job for the ``hello-world`` process
|
|
- ``curl -X POST "http://localhost:5000/processes/hello-world/jobs" -H "Content-Type: application/json" -d "{\"inputs\":[{\"id\":\"name\",\"type\":\"text/plain\",\"value\":\"hi there2\"}]}"``
|
|
- execute a job for the ``hello-world`` process with a raw response
|
|
- ``curl -X POST "http://localhost:5000/processes/hello-world/jobs?response=raw" -H "Content-Type: application/json" -d "{\"inputs\":[{\"id\":\"name\",\"type\":\"text/plain\",\"value\":\"hi there2\"}]}"``
|
|
- execute a job for the ``hello-world`` process in asynchronous mode
|
|
- ``curl -X POST "http://localhost:5000/processes/hello-world/jobs" -H "Content-Type: application/json" -d "{\"mode\": \"async\", \"inputs\":[{\"id\":\"name\",\"type\":\"text/plain\",\"value\":\"hi there2\"}]}"``
|
|
|
|
.. todo:: add more examples once OAProc implementation is complete
|
|
|
|
.. _`OGC API - Processes`: https://github.com/opengeospatial/wps-rest-binding
|
|
.. _`sample`: https://github.com/geopython/pygeoapi/blob/master/pygeoapi/process/hello_world.py
|
|
.. _`TinyDB`: https://tinydb.readthedocs.io
|