Commit Graph

179 Commits

Author SHA1 Message Date
Tom Kralidis 70edc3c0f2 update TinyDB q= support for multiple terms (#765) 2021-08-26 16:30:46 -04:00
Tom Kralidis 6dcd10b10a test for item prev and next links on item queries (#752) (#757) 2021-08-25 06:34:39 -04:00
Tom Kralidis bc97ed480a add geometry to queryables (#755) 2021-08-22 08:28:56 -04:00
Tom Kralidis 9d920901b5 update OAProc execute handling (#754)
* fix OAProc default output to raw

* control outputs list from api machinery

* fix tests
2021-08-18 15:28:55 -04:00
Francesco Bartoli c5942f7a13 Fix time format in cite configuration (#748) 2021-08-02 07:27:48 -04:00
Tom Kralidis 5359789f13 fix OARec test data (#745)
* fix OARec test data

* fix TinyDB bbox handling

* add missing OACov conformance test
2021-07-23 14:50:34 -04:00
Tom Kralidis 914756666c OARec bbox should be two arrays, not three (#744) 2021-07-23 13:39:39 -04:00
Benjamin Webb 3efa041c54 Patch SensorThings API Tests (#737) 2021-07-22 19:11:25 -04:00
Jo cf4517c1c2 - update the script that loads data to ES, to support data encoded with utf8 (#742) 2021-07-22 19:10:38 -04:00
Tom Kralidis 00a08e05ba update OAProc content models (#735) 2021-07-22 15:13:09 +02:00
Francesco Bartoli cdc4f98090 Fix OAProc conformance classes (#736)
* Fix OAProc conformance classes

* Fix conformance test
2021-07-22 09:06:47 -04:00
Francesco Bartoli bb4cd0bf69 Add cql-json support for ES (#723)
Fix starlette event loop


Fix starlette event loop


Fix starlette event loop


Fix starlette event loop


Fix provider regression


Make method public


Make method public


Move function to the helpers utility


Add the CQL lifecycle for development


Add CQL docs


Fix flake8


Isolate import for starlette codepath
2021-07-21 21:00:14 -04:00
Tom Kralidis f01dfd38a7 fix response for non-existent collections (#722) (#730) 2021-07-20 09:41:59 +02:00
Ari Pollak d4f2b74505 Make FileSystem tests a bit more detailed (#727) 2021-07-19 22:08:44 -04:00
Benjamin Webb 8f8d19c5f8 Sensorthings API Provider (#721)
* Create sensorthings.py

* Update plugin.py

* Add files via upload

* Update sensorthings.py

* Add files via upload

* Update sensorthings.py

* Update sensorthings.py

* Update sensorthings.py

* Update sensorthings.py

* Update sensorthings BRGM Endpoint

* Clean Examples

* Cleaned sensorthings config

* add bbox and datetime query params

Added optional intralinking between STA entities

* add pytest for sta provider

* add sta pytest to workflow

* improve versatility of intralink and uri

* Update Datastreams geometry provider

* Update iow.sta.pygeoapi.config.yml

* faster hits using sta $count

* fix flake8

* added sta provider for github actions

* type in main.yml

* Update main.yml

* Update main.yml

* rename tests/data/datastreams.json

* Update main.yml

* Update load_sta_data.py

* Update load_sta_data.py

* Update load_sta_data.py

* Update load_sta_data.py

* Update load_sta_data.py

* Fix sta pytest

* fix flake8

* Update docs

* Update test_sensorthings__provider.py

* Merge various fixes from sta-patch-1

* force sorting

* Move STA data to remote host

* Clarify use of the intralink field

* rename test_sensorthings_provider.py

* Add Readme.md and switch to docker-compose

* Sensorthings -> SensorThings
2021-07-06 17:04:45 -04:00
Tom Kralidis 89eff49ce0 add support for configuration schema and validation (#553) (#719)
* add support for configuration schema and validation (#553)

* rename types

* minor doc fix

* update i18n regexes

* make openapi click options required arguments
2021-07-06 17:01:05 -04:00
Tom Kralidis 167281ee83 add support for OpenAPI rendering via ReDoc (#414) (#720)
* add support for OpenAPI rendering via ReDoc (#414)

* add support for OpenAPI rendering via ReDoc (#414)
2021-06-30 11:11:57 -04:00
Tom Kralidis 7ecac02693 add support for OpenAPI validation (#717) (#718)
* add support for OpenAPI validation (#717)

* update copyright years

* fix ES ref

* update CLI
2021-06-29 08:48:22 -04:00
Kyle Onda 37b1e9553b add geometry representations for json-ld output format (#689)
* Update linked_data.py

* Update linked_data.py

Added schema:geoShape to geojsonld = false

* Update linked_data.py

* Update linked_data.py for rebasing

* Update linked_data.py

* Update configuration.rst

* Update linked_data.py for geosparql geometry

* Add files via upload

* Update test_api.py

* Update test_api.py

* Update linked_data.py logic

* Add pytest for more geometry types

* Update test_util.py

* rename geojson2geojsonld, separated geom2schemageo

* Add pytest for polygon and linestring

* Update expanation of uri_field

Co-authored-by: Benjamin Webb <40066515+webb-ben@users.noreply.github.com>
Co-authored-by: Benjamin Webb <bmwebb20@colby.edu>
2021-06-17 19:00:35 -04:00
Tom Kralidis 989dfb6d30 fix Coverage output and format exception MIME type (#712)
* fix Coverage output and format exception MIME type

* Minor tweaks, added test to assert that a text/html header (without 'f' query param) returns JSON for get_collection_coverage()

Co-authored-by: Sander Schaminee <sander.schaminee@geocat.net>
2021-06-10 19:47:21 -04:00
Sander Schaminee 8f3e5c2ad7 Simplify API tests (#707) 2021-06-09 06:10:10 -04:00
Sander Schaminee 023f24d26b Multilingual support (alternative) (#664)
* Created localization (l10n) module + tests. Added l10n support to API and plugins (wip).

* Big refactor:

* All routed API methods are now decorated by @pre_process (consistency) and no longer have a headers+format argument but a request argument (**kwargs also removed)
* The pre_process decorator turns an incoming Flask/Starlette request into a generic APIRequest instance
* The new APIRequest class extracts all relevant info (params, data, locale, etc.) from the request and exposes them as properties
* Removed a lot of boilerplate (i.e. format checking) and wrapped that into methods
* Updated server-specific API calls in each route method (pass entire request object, not headers and query params)

* Several improvements and fixes:

* Updated OpenAPI page with "l" query param
* Added example translations (metadata)
* Changed plugin signature: added explicit locale attribute (instead of **kwargs)
* Moved locale processing to get_plugin_locale() function in l10n module
* API should pass raw requested locale to plugins, locale should always be set
* Fixed API tests and added APIRequest tests
* Prepared utils.py for Jinja2 i18n extension
* Rebased on commit b40297a8 and fixed compatibility with #661 and #662

* Updated documentation for language support

* Rebased and fixed compatibility with PR #658:

* Fixed EDR provider signature (added locale)
* Fixed EDR API routes and query function (and improved parameter-name handling)
* Fixed EDR tests

* Translate entire config in render_j2_template for requested locale:

* Added new translate_dict function to l10n module (+ tests)
* Updated all render_j2_template calls with locale parameter
* Updated pygeoapi-test-config.yml with some language structs

* Minor improvements

* support both 'language' and 'languages' property in server config and provider definitions
* renamed and modified translate_dict() to more generic translate_struct() function (l10n module)
* remove Content-Language header from provider responses if provider has no language support and format is json(ld)
* updated tests

* Leave provider locale handling to API

* Moved code to determine locale from providers to API class (and remove for formatters and processes)
* Removed locale parameter from plugin __init__ signatures
* Removed locale parameter from load_plugin()
* Added **kwargs to provider implementations for get, query, get_metadata, get_coverage_domainset and get_coverage_rangetype method signatures
* Added language=<locale> to all API calls to provider get, query, get_metadata, get_coverage_domainset and get_coverage_rangetype methods

* Use 'lang' instead of 'l' as language query parameter

* Updated Open API
* Updated documentation
* Fixed tests

* Implemented requested PR changes:

* Added usage examples to the APIRequest docstring
* Removed language support from coverage functions
* Updated plugins.rst and language.rst to match new behavior
* Removed language struct from resource links in pygeoapi-config.yml
* Rebased on latest master (fixed test_api.py)

* Rebased and applied fixes:

* Data property in APIRequest now is an awaitable attribute (fixed for Starlette compatibility)
* Named references to 'l' parameter to 'lang'

* Final changes/improvements:

* Make sure that Content-Language is always set;
* Added more tests to ensure that the default language returned is the first configured language (if no language was requested by the user);
* Updated docs;
* Replaced re-occuring strings with constants in api.py;
* Fixed Flake8 checks.

* add missing async to starlette routes (#704)

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2021-06-08 18:46:35 -04:00
Tom Kralidis d92f8012d1 update OARec metadata models per spec changes (#701) 2021-05-31 18:27:53 -04:00
Tom Kralidis ac7098aab5 update OATiles conformance classes (#692) 2021-05-27 08:38:06 -04:00
Tom Kralidis a4bf35d310 implement STAC 1.0.0-rc.2 improvements (#679) 2021-04-27 19:52:28 -04:00
Tom Kralidis 3f248a5c42 fix datetime window validation (#666) (#671) 2021-03-30 06:21:11 -04:00
Tom Kralidis 43b6165fa9 EDR: s/parameters/parameter-names/ (#667) 2021-03-26 09:58:46 -04:00
Tom Kralidis b40297a864 handle datetime errors in config (#659) (#661) 2021-03-18 06:25:39 -04:00
Tom Kralidis 951a1fb486 implement OGC EDR API (#658)
* implement OGC EDR API
* add docs/tests
* fix tests
2021-03-15 17:37:40 +02:00
Tom Kralidis 2f99f19fb0 refactor exceptions (#650)
* refactor exceptions

* add test
2021-02-26 14:07:50 -05:00
Tom Kralidis 4242083330 allow for enumerated values in providers 2021-02-25 22:26:25 +01:00
Tom Kralidis e0003b066b update queryables response 2021-02-25 22:26:25 +01:00
Tom Kralidis 9da53b0b76 update sortby parameter handling 2021-02-21 19:43:26 -05:00
paul van genuchten 1bd1ad509b add contact and url's to record (#636) 2021-02-21 16:18:35 -05:00
Tom Kralidis 803fea81b4 add q to queryables endpoint (#638) (#641) 2021-02-21 14:51:41 -05:00
Tom Kralidis cc6dc2e33b update config, fix ref (#634) 2021-02-18 21:54:48 -05:00
Tom Kralidis 0c1d98e868 implement OGC API - Records (#404) (#631) 2021-02-18 17:56:23 -05:00
Tom Kralidis 10f9a6a9fd update OpenAPI repo names 2021-02-12 09:41:49 -05:00
totycro 0aab0c9bd2 Date now handling (#613)
* Drop support for "now" in dategetter

This value already leads to an error in queries, so it's not really
supported at this point anyway.

* Use None instead of ".." for open temporal intervals

See https://github.com/opengeospatial/ogcapi-features/blob/master/core/openapi/schemas/extent.yaml#L86

* Allow empty string in dateranges to mean open interval

See http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#_parameter_datetime
2021-01-27 22:21:09 -05:00
Tom Kralidis 6915efbcad add links, refactor job handling, HTML updates (#599)
* add links, refactor job handling, HTML updates

* fix test

* misc fixes
2020-12-30 09:30:12 -05:00
Richard Law 60202129ec implement processing manager (#365)
* add process manager

* process manager updating

* control connectivity from inside manager

* missing method on execute_process function signature

* adds tinydb depenedency for requirements-provider

* adds some processes tests

* process manager under server config, not a process

* additional html rendering for jobs and processes

* fixes typo in docstring

* complete html templating for jobs and processes

* considers timezone when formatting processing start/end datetimes

* expanded locales; locale and tz environment variables

* makes environment TZ a global from pygeoapi/__init__.py

* don't redefine built-in format

* adds placeholder values in process form if given as examples

* placeholder in form handles null example

* changes to process progress update

* updates and changes to processing web UI

* adds python3-distutils dep, and env changes for TZs and locales

* adds tinydb dependency for processes

* Adds libpq-dev and related deps for processing - should eventually be reconciled

* minor margin change

* fix bug in postJob.js that prevented results from displaying

* move process elements around to economise on space

* adds second, complex process

* post data may be bytes

* make external process example

* adds redis process manager (may be moved to third-party plugin eventually)

* reintroduce default async processing; refactoring managers; improving sync/async processing ui

* throw pygeoapi ManagerExecuteError, not redis exception

* adds test for async-execute

* restores original test host

* separates async execute handler function

* adds support for file uploads as processing inputs

* adds endpoint for file download as process output

* naming consistency

* Fix job submission form in case server url is relative (#549)

`new URL()` fails if the first argument isn't a complete url

* Implement job deletion in UI, api and tinydb

* Also allow deletion from job list

The UI could be fancier by reloading the page or just removing the
deleted job right away, but I don't want to start building that now.

* Delete output file when deleting a job

* typo in test case

* move delete button in job view

* remove Dockerfile changes (cf #470)

* removes some docs/source*.rst absent in master

* fixes whitespace

* fixes pip installs

* removes more locale-specific content, cf 70

* fixes duplicate app following merge

* removes redis as default plugin

* removes ENV_TZ, cf #407

* fix typo

* fixes syntax error

* process execution test cleanup

* Fix format detection in execute_process (#589)

Before, html was always selected.

* remove db files in code directories, fix flake8, specific functionality

* remove interactive capabilities in lieu of Swagger

* Add basic openapi documentation for job detail endpoint (#590)

It's somewhat superficial still, e.g. jobId is missing as
proper parameter in the UI.

* Show elapsed time for currently running jobs (#591)

* flake8, jobs workflow

* update docs, DB path in test config

* more code cleanup

* manager workflow cleanup

* add default dummy manager

* update docs

* fix routes

* fixed per comments

* fix per comments

* fix per comments

* fix travis

* simply describe process logic

* simplify describe process logic

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
Co-authored-by: Jorge de Jesus <jorge.jesus@gmail.com>
Co-authored-by: totycro <bernhard.mallinger@eox.at>
Co-authored-by: Richard Law <richard.law@cartoncloud.com.au>
2020-12-22 06:53:31 -05:00
Richard Law c111140f0b Bug/datetime validation (#570)
* fixes #569

* additional test case for test_validate_datetime

* additional test case for test_validate_datetime

* linting
2020-11-13 06:41:16 -05:00
Tom Kralidis c043405508 add support for propertyname and skipGeometry parameters (#51) (#565)
* add support for propertyname and skipGeometry parameters (#51)

* fix tests

* fix tests

* fix tests

* fix vars

* add tests
2020-11-02 10:07:37 +01:00
Tom Kralidis 42a3424ae8 add bbox and datetime support to OACov providers (#560) 2020-10-29 16:48:58 -04:00
Tom Kralidis 2c567d25f7 fix conformance links (#558) 2020-10-22 09:54:29 -04:00
Tom Kralidis 1fa275b832 fix OGR WFS tests using MapServer demo (#552) 2020-10-14 06:57:54 -04:00
Tom Kralidis 8a65087eef [WIP] refactor bbox and datetime for reuse by other APIs (#551)
* refactor bbox and datetime for reuse by other APIs

* update plugin query arguments

* test min/max and temporal
2020-10-10 16:02:23 -04:00
Gregory Petrochenkov 44992caab3 Add zarr to xarray provider (#546)
* Progress

* Add temp directory to zarr zip

* Update readme with zarr data info

* Remove numpy from util.py

* Review revisions

* Fix file system test
2020-09-24 18:02:15 -04:00
Tom Kralidis c579b6ddb2 fix failing test (#541) 2020-09-24 15:03:45 -04:00
Tom Kralidis f36e16d7e9 various fixes (#538)
* fix version output

* sort directory names

* unify tile tests

* fix typo in docs

* set OpenAPI info.version to pygeoapi version, add tiles output format

* align request/response headers, update not found tiles to return 404
2020-09-21 12:37:04 +02:00