Commit Graph

140 Commits

Author SHA1 Message Date
Tom Kralidis 6915efbcad add links, refactor job handling, HTML updates (#599)
* add links, refactor job handling, HTML updates

* fix test

* misc fixes
2020-12-30 09:30:12 -05:00
Richard Law 60202129ec implement processing manager (#365)
* add process manager

* process manager updating

* control connectivity from inside manager

* missing method on execute_process function signature

* adds tinydb depenedency for requirements-provider

* adds some processes tests

* process manager under server config, not a process

* additional html rendering for jobs and processes

* fixes typo in docstring

* complete html templating for jobs and processes

* considers timezone when formatting processing start/end datetimes

* expanded locales; locale and tz environment variables

* makes environment TZ a global from pygeoapi/__init__.py

* don't redefine built-in format

* adds placeholder values in process form if given as examples

* placeholder in form handles null example

* changes to process progress update

* updates and changes to processing web UI

* adds python3-distutils dep, and env changes for TZs and locales

* adds tinydb dependency for processes

* Adds libpq-dev and related deps for processing - should eventually be reconciled

* minor margin change

* fix bug in postJob.js that prevented results from displaying

* move process elements around to economise on space

* adds second, complex process

* post data may be bytes

* make external process example

* adds redis process manager (may be moved to third-party plugin eventually)

* reintroduce default async processing; refactoring managers; improving sync/async processing ui

* throw pygeoapi ManagerExecuteError, not redis exception

* adds test for async-execute

* restores original test host

* separates async execute handler function

* adds support for file uploads as processing inputs

* adds endpoint for file download as process output

* naming consistency

* Fix job submission form in case server url is relative (#549)

`new URL()` fails if the first argument isn't a complete url

* Implement job deletion in UI, api and tinydb

* Also allow deletion from job list

The UI could be fancier by reloading the page or just removing the
deleted job right away, but I don't want to start building that now.

* Delete output file when deleting a job

* typo in test case

* move delete button in job view

* remove Dockerfile changes (cf #470)

* removes some docs/source*.rst absent in master

* fixes whitespace

* fixes pip installs

* removes more locale-specific content, cf 70

* fixes duplicate app following merge

* removes redis as default plugin

* removes ENV_TZ, cf #407

* fix typo

* fixes syntax error

* process execution test cleanup

* Fix format detection in execute_process (#589)

Before, html was always selected.

* remove db files in code directories, fix flake8, specific functionality

* remove interactive capabilities in lieu of Swagger

* Add basic openapi documentation for job detail endpoint (#590)

It's somewhat superficial still, e.g. jobId is missing as
proper parameter in the UI.

* Show elapsed time for currently running jobs (#591)

* flake8, jobs workflow

* update docs, DB path in test config

* more code cleanup

* manager workflow cleanup

* add default dummy manager

* update docs

* fix routes

* fixed per comments

* fix per comments

* fix per comments

* fix travis

* simply describe process logic

* simplify describe process logic

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
Co-authored-by: Jorge de Jesus <jorge.jesus@gmail.com>
Co-authored-by: totycro <bernhard.mallinger@eox.at>
Co-authored-by: Richard Law <richard.law@cartoncloud.com.au>
2020-12-22 06:53:31 -05:00
Richard Law c111140f0b Bug/datetime validation (#570)
* fixes #569

* additional test case for test_validate_datetime

* additional test case for test_validate_datetime

* linting
2020-11-13 06:41:16 -05:00
Tom Kralidis c043405508 add support for propertyname and skipGeometry parameters (#51) (#565)
* add support for propertyname and skipGeometry parameters (#51)

* fix tests

* fix tests

* fix tests

* fix vars

* add tests
2020-11-02 10:07:37 +01:00
Tom Kralidis 42a3424ae8 add bbox and datetime support to OACov providers (#560) 2020-10-29 16:48:58 -04:00
Tom Kralidis 2c567d25f7 fix conformance links (#558) 2020-10-22 09:54:29 -04:00
Tom Kralidis 1fa275b832 fix OGR WFS tests using MapServer demo (#552) 2020-10-14 06:57:54 -04:00
Tom Kralidis 8a65087eef [WIP] refactor bbox and datetime for reuse by other APIs (#551)
* refactor bbox and datetime for reuse by other APIs

* update plugin query arguments

* test min/max and temporal
2020-10-10 16:02:23 -04:00
Gregory Petrochenkov 44992caab3 Add zarr to xarray provider (#546)
* Progress

* Add temp directory to zarr zip

* Update readme with zarr data info

* Remove numpy from util.py

* Review revisions

* Fix file system test
2020-09-24 18:02:15 -04:00
Tom Kralidis c579b6ddb2 fix failing test (#541) 2020-09-24 15:03:45 -04:00
Tom Kralidis f36e16d7e9 various fixes (#538)
* fix version output

* sort directory names

* unify tile tests

* fix typo in docs

* set OpenAPI info.version to pygeoapi version, add tiles output format

* align request/response headers, update not found tiles to return 404
2020-09-21 12:37:04 +02:00
Francesco Bartoli 4f52536051 Add tiles ogc api (#419)
* Add ogc api tiles implementation

* Add json response to tiles endpoint

* Return correct json payload

* Fix url generation

* Add vector tile layer with clickable items

* Fix typo

* Fix oafeat collection tiles link

* Add format to metadata link

* Fix almost all flake8 errors

* Fix metadata link

* Add routes to serving tiles

* Refactor tile implementation with an own module

* Add http client for mvt

* Fix visibility of tiles template

* Fix tiles in collection item page

* Change source to data coherently with stac provider

* Fix flake8 errors

* Fix test for conformance classes

* Fix flake8 errors


Fix flake8 errors

* Refactor to align with new configuration structure

* Fix MVT plugin name

* Add openapi tiles path and schemas

* Add ogc api tiles implementation

* Add json response to tiles endpoint

* Return correct json payload

* Fix url generation

* Add vector tile layer with clickable items

* Fix typo

* Fix oafeat collection tiles link

* Add format to metadata link

* Fix almost all flake8 errors

* Fix metadata link

* Add routes to serving tiles

* Refactor tile implementation with an own module

* Add http client for mvt

* Fix visibility of tiles template

* Fix tiles in collection item page

* Change source to data coherently with stac provider

* Fix flake8 errors

* Fix test for conformance classes

* Fix flake8 errors


Fix flake8 errors

* Refactor to align with new configuration structure

* Fix MVT plugin name

* Add openapi tiles path and schemas

* Fix flake errors

* Fix merge and tests

* Add test configuration and data for tiles

* Change mapbox vector tile provider name

* Fix provider inspection and filter

* Fix tiles rendering in collection page

* Add mimetype to the mvt provider

* Fix mimetype

* Rename functions accordingly to singular and plural forms

* Fix tiling schemes in tiles template

* Serve mvt through the pygeoapi route

* Refactor and add test for collection tiles

* Fix test for not being tiles

* Remove httpx dependency

* Fix missing comma

* Fix tests

* Fix flake8

* Add initial metadata implementation

* Fix route with preprocess

* Fix tiles template


Fix tiles template

* Complete tileset api and template

* Fix flake8

* Complete metadata with tilejson response

* Add support for local vector tiles

* Fix flake8

* Fix plugin tile object and provider name

* Add a todo comment

* testing alternate plugin setup, fix some HTML, add OpenAPI endpoint

* remove URI template

* add docs

* Fix failing test

* Fix flake8 error

Co-authored-by: Francesco Bartoli <francesco.bartoli@wfp.org>
Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2020-09-20 09:50:40 -04:00
Tom Kralidis 2fb20c59f6 fix coverage query with no parameters (#528)
* fix file dump, update docs

* fix file dump, update docs

* fix file dump, update docs

* support data read by file or network
2020-09-11 15:19:11 -04:00
Tom Kralidis 4d2f229c88 fix rasterio driver output (#527)
* fix rasterio driver output

* fix rasterio driver output

* rename driver
2020-09-10 16:24:29 -04:00
Tom Kralidis daf9b19195 [WIP] various xarray updates (#526)
* various xarray updates

* fix ref

* add support non-numeric axis subsetting, HTTP 204 on no content

* add info on COADS test data
2020-09-10 10:36:51 -04:00
Gregory Petrochenkov 8cb34a587a implement xarray Coverage Provider (#525)
* add support for OGC API - Coverages

* fix coverage CRS ref

* fix ref to OACov schemas for testing

* Halfway progress

* Adjust rangeSubset

* Fixes regarding pr comments

* Version zarr

* Changing dependency order

* Remove zarr

* Add gpkg for tests

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2020-09-09 09:51:34 -04:00
alex-mathew 7ddcc9963c Issue 499 - Move id field back to properties and adjust logic to check id field in both root and properties (#515)
* moved id field to root in sample geojson provider files (geopython#499)

* modified geojson data provider to get id from root (geopython#499)

* updated geojson provider tests (geopython#499)

* updated position of id field (geopython#499)

* refactored code (geopython#499)

* refactored geojson provider (geopython#499)
2020-08-25 18:02:29 -04:00
Tom Kralidis 85939109b1 minor docs updates and test cleanups 2020-08-22 10:49:18 -04:00
Tom Kralidis da824fba8f add support for OGC API - Coverages (#110) (#516)
* add support for OGC API - Coverages

* fix coverage CRS ref

* fix ref to OACov schemas for testing

* move spectral testing to after_success

* update docs

* add mask param to rasterio provider
2020-08-21 09:52:17 -04:00
alex-mathew 1a1dae6d8e Issue 499 - Moving id field of geojson providers to root (#501)
* moved id field to root in sample geojson provider files (geopython#499)

* modified geojson data provider to get id from root (geopython#499)

* updated geojson provider tests (geopython#499)
2020-07-31 06:58:55 -04:00
Tom Kralidis fc4c6c9bef minor alignments (#498)
* minor alignments

* fix test

* fix test
2020-07-25 14:44:30 -04:00
Tom Kralidis 284bf96b90 reuse CSV file for tests (#492) 2020-07-22 12:27:20 -04:00
Tom Kralidis 28157426f5 update providers to allow 1..n representations (#489) 2020-07-13 06:28:11 -04:00
Tom Kralidis 3938420dc2 fix raw output for OAProc (#487) 2020-07-12 22:39:18 +02:00
Francesco Bartoli 1d4536f3f1 Fix travis (#476)
* Fix csv ogr test

* Fix ogr wfs test from complex geojson
2020-06-30 14:41:47 -04:00
Tom Kralidis 89f8cafbb5 fix flake8 (#446) 2020-06-04 21:26:55 -04:00
Richard Law c5134f07ce fixes #444 (#445) 2020-06-04 20:22:58 -04:00
Tom Kralidis 1d576fd752 fix OpenAPI generation (#428)
* fix OpenAPI generation

* fix
2020-04-27 11:15:04 -04:00
Tom Kralidis 3debd0c750 update configuration to be resource specific (#393) 2020-04-26 09:57:07 -04:00
Tom Kralidis 6916099315 fix queryables refs from providers, add link relations (#422)
* fix queryables refs from providers, add link relations

* fix tests

* fix tests again
2020-04-25 21:04:21 -04:00
Tom Kralidis 54732d26c9 implement queryables endpoint (#417)
* implement queryables endpoint

* fix tests now that CSV provider provides properties

* filter queryables if provider properties are set
2020-04-23 08:59:02 -04:00
Francesco Bartoli b8dbd9673c Catch http errors and item not found (#416)
Change class name


Fix flake8


Fix exceptions order


Add missing class name changes


Fix get function for geojson provider


Fix get function for ES provider


Fix ES tests


Fix ES tests


Change word


Fix get function for csv and mongo providers


Fix import


Fix flake8


Fix get function for sqlite/gpkg provider


Fix get function for PG provider


Fix postgres tests


Fix postgres tests


Fix postgres tests
2020-04-21 14:48:02 -04:00
Francesco Bartoli 0a2097713e Improve strategy to close #406 (#415)
* Improve strategy to close #406

* Fix travis failure

* Fix sphinx error

* Fix typo
2020-04-20 14:49:43 -04:00
Francesco Bartoli c8e535add5 Add csv support via ogr (#410)
Add tests for CSV driver from OGR


Add csv example layer to the OGR test configuration
2020-04-16 12:39:08 +02:00
Francesco Bartoli c99d1ba1ed Fix #406 (#408) 2020-04-16 12:23:02 +02:00
ynte 793d6cc89d Proposal for enabling property filtering with ogr provider (#220)
* enable property filtering on ogr provider

* property filtering tests for current ogr backend suites

* Fix blank line containing whitspace

Co-authored-by: ynte <ynte@geoloep.nl>
2020-04-13 14:03:28 +02:00
Francesco Bartoli 8d1c812162 Fix issue 137 (#396)
* Fix regression on string messages

* Fix paging for ESRIJSON driver

* catch ogr http errors

* Fix test and inherit from intermediate class

Co-authored-by: Francesco Bartoli <francesco.bartoli@geobeyond.it>
2020-04-12 23:47:42 +02:00
Tom Kralidis 39b22d4dd1 optimize test data ES loader (thanks @jvanulde) (#394) 2020-04-12 12:09:56 -04:00
Tom Kralidis c151579991 add STAC catalog functionality (#389)
* add STAC catalog functionality

* add example to config, add sample raster data

* fix tests
2020-04-12 12:08:17 -04:00
Tom Kralidis 799533106f Update README.md 2020-04-10 18:19:30 -04:00
Francesco Bartoli 560fcc9fb1 Fix issue 390 (#391)
* Add decorated utility function to ignore gdal errors

* Replace push and pop error handler with decorated function

* Add docstrings

* Add test for OGR esrijson driver

* Make functions private

* Fix flake8

Co-authored-by: Francesco Bartoli <francesco.bartoli@wfp.org>
2020-04-10 15:23:53 -04:00
Francesco Bartoli 473e7f1691 Fix issue #371 (#372)
* Fix geojson bigger than 200MB not raising exception

* Log the actual error

* Remove replication of fixtures

* Fix flake8

* Fix flake8

* Add error handler class


Add generic error exception for providers


Fix flake8


Fix output message for geojson object size of collections

* Use popup error handler

Co-authored-by: Francesco Bartoli <francesco.bartoli@wfp.org>
2020-03-22 22:49:01 +01:00
Jorge Samuel Mendes de Jesus 6be38cee9d implements hit results with filtering on Sqlite/geopackage dataprovier (#373)
* pytest and flake8 check

* review requests

* SQL queries on caps
2020-03-06 07:13:01 -05:00
Francesco Bartoli 72b9a647ff GDAL3 support (#359)
* Bump gdal to v3

* Fix query duplicates with bbox for PG

* Add backward compatibility for axis order

* Move travis to bionic dist


Fix gdal version 3 on travis by pinning latest ubuntu unstable release


Fix ubuntugis sources on travis


Remove pinned python gdal version


Add mongodb start


Enable mongod service


Fix package name


Install mongo from official community repos


Add mongo enable and start commands

* Fix wrong axis mapping strategy


Add author

* Move base image to bullseye for docker build

* Fix remaining unchanged places for #364

* Make image size smaller

* Add docker build and cite tests


Fix python version


Move steps after docker build elsewhere


Fix uppercase characters


Fix internal port


Daemonize the run process

* Check logs and run test command from the entrypoint

* Remove the execution of mongo tests for test command

Remove the execution of mongo tests for test command

Fix failed command

* Add delay and move build to install step

Remove grep command for now


Use single quote
2020-03-03 23:03:14 +01:00
Jorge S. Mendes de Jesus a16ac938cc where implementation on resulttype=hits 2020-02-23 08:34:35 +01:00
Tom Kralidis d0d7ed48da Implement configurable OGC schema locations (#347) 2020-01-20 14:56:04 -05:00
paul van genuchten 01122e7318 Place is a type not a property (#343)
* fix for #342
Place is a type not a property

* fix the build

* test place type in dataset>spatial

* fix line length
2020-01-16 15:07:42 -05:00
Tom Kralidis c994d47c91 add test for high limits (#340) 2020-01-15 07:22:50 -05:00
Tom Kralidis af9fbb5f74 fix ES paging refs and local zero-based paging (#338) 2020-01-13 16:42:21 -05:00
Tom Kralidis 56500e6155 [Elasticsearch] update to support for ES 7 (#336) 2020-01-12 21:07:21 -05:00