Commit Graph

168 Commits

Author SHA1 Message Date
Francesco Bartoli bb4cd0bf69 Add cql-json support for ES (#723)
Fix starlette event loop


Fix starlette event loop


Fix starlette event loop


Fix starlette event loop


Fix provider regression


Make method public


Make method public


Move function to the helpers utility


Add the CQL lifecycle for development


Add CQL docs


Fix flake8


Isolate import for starlette codepath
2021-07-21 21:00:14 -04:00
Tom Kralidis f01dfd38a7 fix response for non-existent collections (#722) (#730) 2021-07-20 09:41:59 +02:00
Ari Pollak d4f2b74505 Make FileSystem tests a bit more detailed (#727) 2021-07-19 22:08:44 -04:00
Benjamin Webb 8f8d19c5f8 Sensorthings API Provider (#721)
* Create sensorthings.py

* Update plugin.py

* Add files via upload

* Update sensorthings.py

* Add files via upload

* Update sensorthings.py

* Update sensorthings.py

* Update sensorthings.py

* Update sensorthings.py

* Update sensorthings BRGM Endpoint

* Clean Examples

* Cleaned sensorthings config

* add bbox and datetime query params

Added optional intralinking between STA entities

* add pytest for sta provider

* add sta pytest to workflow

* improve versatility of intralink and uri

* Update Datastreams geometry provider

* Update iow.sta.pygeoapi.config.yml

* faster hits using sta $count

* fix flake8

* added sta provider for github actions

* type in main.yml

* Update main.yml

* Update main.yml

* rename tests/data/datastreams.json

* Update main.yml

* Update load_sta_data.py

* Update load_sta_data.py

* Update load_sta_data.py

* Update load_sta_data.py

* Update load_sta_data.py

* Fix sta pytest

* fix flake8

* Update docs

* Update test_sensorthings__provider.py

* Merge various fixes from sta-patch-1

* force sorting

* Move STA data to remote host

* Clarify use of the intralink field

* rename test_sensorthings_provider.py

* Add Readme.md and switch to docker-compose

* Sensorthings -> SensorThings
2021-07-06 17:04:45 -04:00
Tom Kralidis 89eff49ce0 add support for configuration schema and validation (#553) (#719)
* add support for configuration schema and validation (#553)

* rename types

* minor doc fix

* update i18n regexes

* make openapi click options required arguments
2021-07-06 17:01:05 -04:00
Tom Kralidis 167281ee83 add support for OpenAPI rendering via ReDoc (#414) (#720)
* add support for OpenAPI rendering via ReDoc (#414)

* add support for OpenAPI rendering via ReDoc (#414)
2021-06-30 11:11:57 -04:00
Tom Kralidis 7ecac02693 add support for OpenAPI validation (#717) (#718)
* add support for OpenAPI validation (#717)

* update copyright years

* fix ES ref

* update CLI
2021-06-29 08:48:22 -04:00
Kyle Onda 37b1e9553b add geometry representations for json-ld output format (#689)
* Update linked_data.py

* Update linked_data.py

Added schema:geoShape to geojsonld = false

* Update linked_data.py

* Update linked_data.py for rebasing

* Update linked_data.py

* Update configuration.rst

* Update linked_data.py for geosparql geometry

* Add files via upload

* Update test_api.py

* Update test_api.py

* Update linked_data.py logic

* Add pytest for more geometry types

* Update test_util.py

* rename geojson2geojsonld, separated geom2schemageo

* Add pytest for polygon and linestring

* Update expanation of uri_field

Co-authored-by: Benjamin Webb <40066515+webb-ben@users.noreply.github.com>
Co-authored-by: Benjamin Webb <bmwebb20@colby.edu>
2021-06-17 19:00:35 -04:00
Tom Kralidis 989dfb6d30 fix Coverage output and format exception MIME type (#712)
* fix Coverage output and format exception MIME type

* Minor tweaks, added test to assert that a text/html header (without 'f' query param) returns JSON for get_collection_coverage()

Co-authored-by: Sander Schaminee <sander.schaminee@geocat.net>
2021-06-10 19:47:21 -04:00
Sander Schaminee 8f3e5c2ad7 Simplify API tests (#707) 2021-06-09 06:10:10 -04:00
Sander Schaminee 023f24d26b Multilingual support (alternative) (#664)
* Created localization (l10n) module + tests. Added l10n support to API and plugins (wip).

* Big refactor:

* All routed API methods are now decorated by @pre_process (consistency) and no longer have a headers+format argument but a request argument (**kwargs also removed)
* The pre_process decorator turns an incoming Flask/Starlette request into a generic APIRequest instance
* The new APIRequest class extracts all relevant info (params, data, locale, etc.) from the request and exposes them as properties
* Removed a lot of boilerplate (i.e. format checking) and wrapped that into methods
* Updated server-specific API calls in each route method (pass entire request object, not headers and query params)

* Several improvements and fixes:

* Updated OpenAPI page with "l" query param
* Added example translations (metadata)
* Changed plugin signature: added explicit locale attribute (instead of **kwargs)
* Moved locale processing to get_plugin_locale() function in l10n module
* API should pass raw requested locale to plugins, locale should always be set
* Fixed API tests and added APIRequest tests
* Prepared utils.py for Jinja2 i18n extension
* Rebased on commit b40297a8 and fixed compatibility with #661 and #662

* Updated documentation for language support

* Rebased and fixed compatibility with PR #658:

* Fixed EDR provider signature (added locale)
* Fixed EDR API routes and query function (and improved parameter-name handling)
* Fixed EDR tests

* Translate entire config in render_j2_template for requested locale:

* Added new translate_dict function to l10n module (+ tests)
* Updated all render_j2_template calls with locale parameter
* Updated pygeoapi-test-config.yml with some language structs

* Minor improvements

* support both 'language' and 'languages' property in server config and provider definitions
* renamed and modified translate_dict() to more generic translate_struct() function (l10n module)
* remove Content-Language header from provider responses if provider has no language support and format is json(ld)
* updated tests

* Leave provider locale handling to API

* Moved code to determine locale from providers to API class (and remove for formatters and processes)
* Removed locale parameter from plugin __init__ signatures
* Removed locale parameter from load_plugin()
* Added **kwargs to provider implementations for get, query, get_metadata, get_coverage_domainset and get_coverage_rangetype method signatures
* Added language=<locale> to all API calls to provider get, query, get_metadata, get_coverage_domainset and get_coverage_rangetype methods

* Use 'lang' instead of 'l' as language query parameter

* Updated Open API
* Updated documentation
* Fixed tests

* Implemented requested PR changes:

* Added usage examples to the APIRequest docstring
* Removed language support from coverage functions
* Updated plugins.rst and language.rst to match new behavior
* Removed language struct from resource links in pygeoapi-config.yml
* Rebased on latest master (fixed test_api.py)

* Rebased and applied fixes:

* Data property in APIRequest now is an awaitable attribute (fixed for Starlette compatibility)
* Named references to 'l' parameter to 'lang'

* Final changes/improvements:

* Make sure that Content-Language is always set;
* Added more tests to ensure that the default language returned is the first configured language (if no language was requested by the user);
* Updated docs;
* Replaced re-occuring strings with constants in api.py;
* Fixed Flake8 checks.

* add missing async to starlette routes (#704)

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2021-06-08 18:46:35 -04:00
Tom Kralidis d92f8012d1 update OARec metadata models per spec changes (#701) 2021-05-31 18:27:53 -04:00
Tom Kralidis ac7098aab5 update OATiles conformance classes (#692) 2021-05-27 08:38:06 -04:00
Tom Kralidis a4bf35d310 implement STAC 1.0.0-rc.2 improvements (#679) 2021-04-27 19:52:28 -04:00
Tom Kralidis 3f248a5c42 fix datetime window validation (#666) (#671) 2021-03-30 06:21:11 -04:00
Tom Kralidis 43b6165fa9 EDR: s/parameters/parameter-names/ (#667) 2021-03-26 09:58:46 -04:00
Tom Kralidis b40297a864 handle datetime errors in config (#659) (#661) 2021-03-18 06:25:39 -04:00
Tom Kralidis 951a1fb486 implement OGC EDR API (#658)
* implement OGC EDR API
* add docs/tests
* fix tests
2021-03-15 17:37:40 +02:00
Tom Kralidis 2f99f19fb0 refactor exceptions (#650)
* refactor exceptions

* add test
2021-02-26 14:07:50 -05:00
Tom Kralidis 4242083330 allow for enumerated values in providers 2021-02-25 22:26:25 +01:00
Tom Kralidis e0003b066b update queryables response 2021-02-25 22:26:25 +01:00
Tom Kralidis 9da53b0b76 update sortby parameter handling 2021-02-21 19:43:26 -05:00
paul van genuchten 1bd1ad509b add contact and url's to record (#636) 2021-02-21 16:18:35 -05:00
Tom Kralidis 803fea81b4 add q to queryables endpoint (#638) (#641) 2021-02-21 14:51:41 -05:00
Tom Kralidis cc6dc2e33b update config, fix ref (#634) 2021-02-18 21:54:48 -05:00
Tom Kralidis 0c1d98e868 implement OGC API - Records (#404) (#631) 2021-02-18 17:56:23 -05:00
Tom Kralidis 10f9a6a9fd update OpenAPI repo names 2021-02-12 09:41:49 -05:00
totycro 0aab0c9bd2 Date now handling (#613)
* Drop support for "now" in dategetter

This value already leads to an error in queries, so it's not really
supported at this point anyway.

* Use None instead of ".." for open temporal intervals

See https://github.com/opengeospatial/ogcapi-features/blob/master/core/openapi/schemas/extent.yaml#L86

* Allow empty string in dateranges to mean open interval

See http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#_parameter_datetime
2021-01-27 22:21:09 -05:00
Tom Kralidis 6915efbcad add links, refactor job handling, HTML updates (#599)
* add links, refactor job handling, HTML updates

* fix test

* misc fixes
2020-12-30 09:30:12 -05:00
Richard Law 60202129ec implement processing manager (#365)
* add process manager

* process manager updating

* control connectivity from inside manager

* missing method on execute_process function signature

* adds tinydb depenedency for requirements-provider

* adds some processes tests

* process manager under server config, not a process

* additional html rendering for jobs and processes

* fixes typo in docstring

* complete html templating for jobs and processes

* considers timezone when formatting processing start/end datetimes

* expanded locales; locale and tz environment variables

* makes environment TZ a global from pygeoapi/__init__.py

* don't redefine built-in format

* adds placeholder values in process form if given as examples

* placeholder in form handles null example

* changes to process progress update

* updates and changes to processing web UI

* adds python3-distutils dep, and env changes for TZs and locales

* adds tinydb dependency for processes

* Adds libpq-dev and related deps for processing - should eventually be reconciled

* minor margin change

* fix bug in postJob.js that prevented results from displaying

* move process elements around to economise on space

* adds second, complex process

* post data may be bytes

* make external process example

* adds redis process manager (may be moved to third-party plugin eventually)

* reintroduce default async processing; refactoring managers; improving sync/async processing ui

* throw pygeoapi ManagerExecuteError, not redis exception

* adds test for async-execute

* restores original test host

* separates async execute handler function

* adds support for file uploads as processing inputs

* adds endpoint for file download as process output

* naming consistency

* Fix job submission form in case server url is relative (#549)

`new URL()` fails if the first argument isn't a complete url

* Implement job deletion in UI, api and tinydb

* Also allow deletion from job list

The UI could be fancier by reloading the page or just removing the
deleted job right away, but I don't want to start building that now.

* Delete output file when deleting a job

* typo in test case

* move delete button in job view

* remove Dockerfile changes (cf #470)

* removes some docs/source*.rst absent in master

* fixes whitespace

* fixes pip installs

* removes more locale-specific content, cf 70

* fixes duplicate app following merge

* removes redis as default plugin

* removes ENV_TZ, cf #407

* fix typo

* fixes syntax error

* process execution test cleanup

* Fix format detection in execute_process (#589)

Before, html was always selected.

* remove db files in code directories, fix flake8, specific functionality

* remove interactive capabilities in lieu of Swagger

* Add basic openapi documentation for job detail endpoint (#590)

It's somewhat superficial still, e.g. jobId is missing as
proper parameter in the UI.

* Show elapsed time for currently running jobs (#591)

* flake8, jobs workflow

* update docs, DB path in test config

* more code cleanup

* manager workflow cleanup

* add default dummy manager

* update docs

* fix routes

* fixed per comments

* fix per comments

* fix per comments

* fix travis

* simply describe process logic

* simplify describe process logic

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
Co-authored-by: Jorge de Jesus <jorge.jesus@gmail.com>
Co-authored-by: totycro <bernhard.mallinger@eox.at>
Co-authored-by: Richard Law <richard.law@cartoncloud.com.au>
2020-12-22 06:53:31 -05:00
Richard Law c111140f0b Bug/datetime validation (#570)
* fixes #569

* additional test case for test_validate_datetime

* additional test case for test_validate_datetime

* linting
2020-11-13 06:41:16 -05:00
Tom Kralidis c043405508 add support for propertyname and skipGeometry parameters (#51) (#565)
* add support for propertyname and skipGeometry parameters (#51)

* fix tests

* fix tests

* fix tests

* fix vars

* add tests
2020-11-02 10:07:37 +01:00
Tom Kralidis 42a3424ae8 add bbox and datetime support to OACov providers (#560) 2020-10-29 16:48:58 -04:00
Tom Kralidis 2c567d25f7 fix conformance links (#558) 2020-10-22 09:54:29 -04:00
Tom Kralidis 1fa275b832 fix OGR WFS tests using MapServer demo (#552) 2020-10-14 06:57:54 -04:00
Tom Kralidis 8a65087eef [WIP] refactor bbox and datetime for reuse by other APIs (#551)
* refactor bbox and datetime for reuse by other APIs

* update plugin query arguments

* test min/max and temporal
2020-10-10 16:02:23 -04:00
Gregory Petrochenkov 44992caab3 Add zarr to xarray provider (#546)
* Progress

* Add temp directory to zarr zip

* Update readme with zarr data info

* Remove numpy from util.py

* Review revisions

* Fix file system test
2020-09-24 18:02:15 -04:00
Tom Kralidis c579b6ddb2 fix failing test (#541) 2020-09-24 15:03:45 -04:00
Tom Kralidis f36e16d7e9 various fixes (#538)
* fix version output

* sort directory names

* unify tile tests

* fix typo in docs

* set OpenAPI info.version to pygeoapi version, add tiles output format

* align request/response headers, update not found tiles to return 404
2020-09-21 12:37:04 +02:00
Francesco Bartoli 4f52536051 Add tiles ogc api (#419)
* Add ogc api tiles implementation

* Add json response to tiles endpoint

* Return correct json payload

* Fix url generation

* Add vector tile layer with clickable items

* Fix typo

* Fix oafeat collection tiles link

* Add format to metadata link

* Fix almost all flake8 errors

* Fix metadata link

* Add routes to serving tiles

* Refactor tile implementation with an own module

* Add http client for mvt

* Fix visibility of tiles template

* Fix tiles in collection item page

* Change source to data coherently with stac provider

* Fix flake8 errors

* Fix test for conformance classes

* Fix flake8 errors


Fix flake8 errors

* Refactor to align with new configuration structure

* Fix MVT plugin name

* Add openapi tiles path and schemas

* Add ogc api tiles implementation

* Add json response to tiles endpoint

* Return correct json payload

* Fix url generation

* Add vector tile layer with clickable items

* Fix typo

* Fix oafeat collection tiles link

* Add format to metadata link

* Fix almost all flake8 errors

* Fix metadata link

* Add routes to serving tiles

* Refactor tile implementation with an own module

* Add http client for mvt

* Fix visibility of tiles template

* Fix tiles in collection item page

* Change source to data coherently with stac provider

* Fix flake8 errors

* Fix test for conformance classes

* Fix flake8 errors


Fix flake8 errors

* Refactor to align with new configuration structure

* Fix MVT plugin name

* Add openapi tiles path and schemas

* Fix flake errors

* Fix merge and tests

* Add test configuration and data for tiles

* Change mapbox vector tile provider name

* Fix provider inspection and filter

* Fix tiles rendering in collection page

* Add mimetype to the mvt provider

* Fix mimetype

* Rename functions accordingly to singular and plural forms

* Fix tiling schemes in tiles template

* Serve mvt through the pygeoapi route

* Refactor and add test for collection tiles

* Fix test for not being tiles

* Remove httpx dependency

* Fix missing comma

* Fix tests

* Fix flake8

* Add initial metadata implementation

* Fix route with preprocess

* Fix tiles template


Fix tiles template

* Complete tileset api and template

* Fix flake8

* Complete metadata with tilejson response

* Add support for local vector tiles

* Fix flake8

* Fix plugin tile object and provider name

* Add a todo comment

* testing alternate plugin setup, fix some HTML, add OpenAPI endpoint

* remove URI template

* add docs

* Fix failing test

* Fix flake8 error

Co-authored-by: Francesco Bartoli <francesco.bartoli@wfp.org>
Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2020-09-20 09:50:40 -04:00
Tom Kralidis 2fb20c59f6 fix coverage query with no parameters (#528)
* fix file dump, update docs

* fix file dump, update docs

* fix file dump, update docs

* support data read by file or network
2020-09-11 15:19:11 -04:00
Tom Kralidis 4d2f229c88 fix rasterio driver output (#527)
* fix rasterio driver output

* fix rasterio driver output

* rename driver
2020-09-10 16:24:29 -04:00
Tom Kralidis daf9b19195 [WIP] various xarray updates (#526)
* various xarray updates

* fix ref

* add support non-numeric axis subsetting, HTTP 204 on no content

* add info on COADS test data
2020-09-10 10:36:51 -04:00
Gregory Petrochenkov 8cb34a587a implement xarray Coverage Provider (#525)
* add support for OGC API - Coverages

* fix coverage CRS ref

* fix ref to OACov schemas for testing

* Halfway progress

* Adjust rangeSubset

* Fixes regarding pr comments

* Version zarr

* Changing dependency order

* Remove zarr

* Add gpkg for tests

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2020-09-09 09:51:34 -04:00
alex-mathew 7ddcc9963c Issue 499 - Move id field back to properties and adjust logic to check id field in both root and properties (#515)
* moved id field to root in sample geojson provider files (geopython#499)

* modified geojson data provider to get id from root (geopython#499)

* updated geojson provider tests (geopython#499)

* updated position of id field (geopython#499)

* refactored code (geopython#499)

* refactored geojson provider (geopython#499)
2020-08-25 18:02:29 -04:00
Tom Kralidis 85939109b1 minor docs updates and test cleanups 2020-08-22 10:49:18 -04:00
Tom Kralidis da824fba8f add support for OGC API - Coverages (#110) (#516)
* add support for OGC API - Coverages

* fix coverage CRS ref

* fix ref to OACov schemas for testing

* move spectral testing to after_success

* update docs

* add mask param to rasterio provider
2020-08-21 09:52:17 -04:00
alex-mathew 1a1dae6d8e Issue 499 - Moving id field of geojson providers to root (#501)
* moved id field to root in sample geojson provider files (geopython#499)

* modified geojson data provider to get id from root (geopython#499)

* updated geojson provider tests (geopython#499)
2020-07-31 06:58:55 -04:00
Tom Kralidis fc4c6c9bef minor alignments (#498)
* minor alignments

* fix test

* fix test
2020-07-25 14:44:30 -04:00
Tom Kralidis 284bf96b90 reuse CSV file for tests (#492) 2020-07-22 12:27:20 -04:00