Commit Graph

526 Commits

Author SHA1 Message Date
Tom Kralidis 6915efbcad add links, refactor job handling, HTML updates (#599)
* add links, refactor job handling, HTML updates

* fix test

* misc fixes
2020-12-30 09:30:12 -05:00
Tom Kralidis e481891f7c fix item STAC HTML map 2020-12-23 12:51:24 -05:00
Kevin Ngai 63e4050102 Add bbox leaflet map to collection page (#595)
Co-authored-by: Kevin Ngai <asptken@geomet-dev-03.cmc.ec.gc.ca>
2020-12-23 12:37:03 -05:00
totycro 0ffe1c57d9 Minor improvements and cleanup of process API (#594)
* Move tinydb configuration from base manager to tinydb

* Rename job_result to job

It's actually the job metadata

* Make get_job_result only return the result

The status is part of the metadata which is already returned by get_job

* Detect format from request headers, not response headers

* Default to json output, not html

This is a reasonable default because mostly browsers want html, and they
always request html, whereas dev tools used by api users might not
always request json.

* Don't provide outputs when executing async an job

In this case, we don't have any outputs yet.

* Make connection an optional attribute for managers

instead of only defining it for tinydb
2020-12-23 11:27:58 -05:00
Richard Law 60202129ec implement processing manager (#365)
* add process manager

* process manager updating

* control connectivity from inside manager

* missing method on execute_process function signature

* adds tinydb depenedency for requirements-provider

* adds some processes tests

* process manager under server config, not a process

* additional html rendering for jobs and processes

* fixes typo in docstring

* complete html templating for jobs and processes

* considers timezone when formatting processing start/end datetimes

* expanded locales; locale and tz environment variables

* makes environment TZ a global from pygeoapi/__init__.py

* don't redefine built-in format

* adds placeholder values in process form if given as examples

* placeholder in form handles null example

* changes to process progress update

* updates and changes to processing web UI

* adds python3-distutils dep, and env changes for TZs and locales

* adds tinydb dependency for processes

* Adds libpq-dev and related deps for processing - should eventually be reconciled

* minor margin change

* fix bug in postJob.js that prevented results from displaying

* move process elements around to economise on space

* adds second, complex process

* post data may be bytes

* make external process example

* adds redis process manager (may be moved to third-party plugin eventually)

* reintroduce default async processing; refactoring managers; improving sync/async processing ui

* throw pygeoapi ManagerExecuteError, not redis exception

* adds test for async-execute

* restores original test host

* separates async execute handler function

* adds support for file uploads as processing inputs

* adds endpoint for file download as process output

* naming consistency

* Fix job submission form in case server url is relative (#549)

`new URL()` fails if the first argument isn't a complete url

* Implement job deletion in UI, api and tinydb

* Also allow deletion from job list

The UI could be fancier by reloading the page or just removing the
deleted job right away, but I don't want to start building that now.

* Delete output file when deleting a job

* typo in test case

* move delete button in job view

* remove Dockerfile changes (cf #470)

* removes some docs/source*.rst absent in master

* fixes whitespace

* fixes pip installs

* removes more locale-specific content, cf 70

* fixes duplicate app following merge

* removes redis as default plugin

* removes ENV_TZ, cf #407

* fix typo

* fixes syntax error

* process execution test cleanup

* Fix format detection in execute_process (#589)

Before, html was always selected.

* remove db files in code directories, fix flake8, specific functionality

* remove interactive capabilities in lieu of Swagger

* Add basic openapi documentation for job detail endpoint (#590)

It's somewhat superficial still, e.g. jobId is missing as
proper parameter in the UI.

* Show elapsed time for currently running jobs (#591)

* flake8, jobs workflow

* update docs, DB path in test config

* more code cleanup

* manager workflow cleanup

* add default dummy manager

* update docs

* fix routes

* fixed per comments

* fix per comments

* fix per comments

* fix travis

* simply describe process logic

* simplify describe process logic

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
Co-authored-by: Jorge de Jesus <jorge.jesus@gmail.com>
Co-authored-by: totycro <bernhard.mallinger@eox.at>
Co-authored-by: Richard Law <richard.law@cartoncloud.com.au>
2020-12-22 06:53:31 -05:00
Richard Law 4678622381 Merge pull request #587 from eurodatacube/fix-dockerfile-pythonh-missing
Add missing build dependencies libpython3-dev and libgdal-dev
2020-12-21 11:54:58 +13:00
Bernhard Mallinger f05c329382 Upgrade libgdal from 2.7 to 2.8
The libgdal27 doesn't seem to be available any more in the current
debian docker image.
2020-12-14 09:51:57 +01:00
Bernhard Mallinger b8eb7577e9 Bump python deps as suggested by @dsusviela
Bumping the gevent version to 20.9.0 and greenlet to 0.4.17

See https://github.com/geopython/pygeoapi/pull/587#issuecomment-742109790
2020-12-14 09:50:53 +01:00
Bernhard Mallinger 10030e9a96 Add missing build dependencies libpython3-dev and libgdal-dev
I'm not sure why these are required now, apparently some update changed
some dependencies.
2020-12-07 10:28:54 +01:00
Just van den Broecke c790158885 #584 fix Dockerfile reinstall python3-gdal after cleanup 2020-12-02 13:04:29 +01:00
Just van den Broecke 90cc97b507 quickfix for #584, pin setuptools to v49.3.1 2020-12-02 11:48:34 +01:00
paul van genuchten f4859d6bd6 google expects plain schema-org url (#576)
* google expects plain schema-org url
https://yoast.com/json-ld/
resolves #574

* Revert "google expects plain schema-org url"

This reverts commit 7f09d4ce3cdd82a538071dec0f50191c7fa01520.

* hack to replace full path for short path, because pyld requires full, search engine expects short
2020-12-01 17:54:49 -05:00
paul van genuchten 8c699a1b4b displays the collection type in list (#579)
* displays the collection type in list

* fix key in link
improve param name
2020-11-28 16:51:41 -05:00
Tom Kralidis 1bbcb91784 Merge pull request #580 from pvgenuchten/increase-item-zoom 2020-11-28 12:40:40 -05:00
Paul 1b32ff1cdb increase max zoom for item zoom
fixes #573

*my smallest PR ever*
2020-11-28 00:01:56 +01:00
Paul Weidner 48d00d67d5 Update example vector provider plugin query arguments. (#578) 2020-11-22 22:17:14 -05:00
Tom Kralidis 21c4d36a40 add distutils to fix pyproj install 2020-11-19 18:15:08 -05:00
Tom Kralidis b69e6f939b add distutils to fix pyproj install 2020-11-19 18:10:25 -05:00
Angelos Tzotsos 9ca7f9b819 back to dev 2020-11-19 17:57:53 +02:00
Angelos Tzotsos 2228e6e7f7 update release version 2020-11-19 17:33:09 +02:00
paul van genuchten 4e5cfd8e50 remove schema-org microdata, because it is now embedded as json-ld (#575)
resolves #574
2020-11-18 22:36:20 -05:00
Richard Law c111140f0b Bug/datetime validation (#570)
* fixes #569

* additional test case for test_validate_datetime

* additional test case for test_validate_datetime

* linting
2020-11-13 06:41:16 -05:00
Tom Kralidis 99785562b4 fix ref (#566) 2020-11-02 08:48:11 -05:00
Tom Kralidis c043405508 add support for propertyname and skipGeometry parameters (#51) (#565)
* add support for propertyname and skipGeometry parameters (#51)

* fix tests

* fix tests

* fix tests

* fix vars

* add tests
2020-11-02 10:07:37 +01:00
paul van genuchten b4f936b843 reproject dataset bounds to 4326 while analyzing stac content (#559)
* reproject dataset bounds to 4326 while analyzing stac content

* fixes code formatting, indicated by tests
adds the shp.xml file as asset

* fix indent

* as suggested by @tomkralidis, only reproject if projection != 4326 or nill

* better param names
2020-11-01 22:52:46 -05:00
Tom Kralidis 42a3424ae8 add bbox and datetime support to OACov providers (#560) 2020-10-29 16:48:58 -04:00
Tom Kralidis 2c567d25f7 fix conformance links (#558) 2020-10-22 09:54:29 -04:00
Tom Kralidis 1fa275b832 fix OGR WFS tests using MapServer demo (#552) 2020-10-14 06:57:54 -04:00
Tom Kralidis 671f4defe2 add datetime support in OACov (#556) 2020-10-11 19:22:57 -04:00
Tom Kralidis 8a65087eef [WIP] refactor bbox and datetime for reuse by other APIs (#551)
* refactor bbox and datetime for reuse by other APIs

* update plugin query arguments

* test min/max and temporal
2020-10-10 16:02:23 -04:00
Tom Kralidis bf9de14bc7 Merge pull request #550 from tomkralidis/minor-fixes-2020-10-10 2020-10-10 10:56:03 -04:00
Tom Kralidis 3cffb7bfda remove unused pathinfo arg for coverages 2020-10-10 08:40:45 -04:00
Tom Kralidis 7812fda63e fix typos in docstrings 2020-10-10 08:40:21 -04:00
Tom Kralidis 9b43b3efa2 fix static folder pathing for blueprints 2020-10-08 17:25:28 -04:00
Tom Kralidis ae369ca77e Update zappa_settings.json
fix zappa config for AWS lambda
2020-10-06 13:22:07 -04:00
Tom Kralidis 6d1dcece0e app mount updates (#548)
* refactor WSGI mounting and add docs

* fix ref

* fix Starlette routing for OACov
2020-09-30 07:07:41 -04:00
Kevin Ngai 90ab772c7c Add templates config yml split into path and static (#523)
- updated configuration.rst doc for templates config
- added docs on HTML templating; dedicated page
- typo fix in configuration.rst
- added debug logging for templates path
- added static config in starlette_app.py
- flake8 fixes
2020-09-29 15:44:00 -04:00
Fabian Schindler 8fe060173a Using Blueprint/route list for flask/starlette (#539)
* Using Blueprint/route list for flask/starlette

* Fixing starlette route setup decorator

* Fixing flake8 linting issue
2020-09-29 12:57:08 -04:00
Just van den Broecke 50cdf4efad Fixes #545 and #547 - pin greenlet to 0.4.16
See also https://github.com/geopython/pygeoapi/issues/545#issuecomment-698999509 .
2020-09-29 11:24:03 +02:00
totycro 007cb64808 Improve docker image cache usage (#542)
Before, any file change in the repo would cause a reinstall of all
dependencies.

This commit splits the dockerfile in 2 phases:
First the files relevant for installation are added and all dependencies
are installed. Then the actual source code is added.

This way, rebuilding after sources have been changes takes a few seconds
rather than minutes.
2020-09-28 15:09:10 +02:00
totycro ad8c66d09a Improve signal handling in entry point (#544)
Currently gunicorn is started in the entrypoint as subprocess. This
means that the entrypoint stays as process with PID 1 and eats the
signals that e.g. kubernetes sends to it. Thus when trying to update
the container, kubernetes must terminate the process ungracefully.

See e.g. this resources for a more detailed explanation:
https://hynek.me/articles/docker-signals/

This change replaces the entrypoint-shellscript with gunicorn, such that
signals are directly forwarded to gunicorn, where they are then handled.

One drawback of this approach is that lines after `exec gunicorn` are
not executed anymore. This currently affects `echo "END /entrypoint.sh"`.

A different approach would be to use `tini` to handle the system (see
link above). To do this, we would need to install tini.
2020-09-28 10:43:38 +02:00
Gregory Petrochenkov 44992caab3 Add zarr to xarray provider (#546)
* Progress

* Add temp directory to zarr zip

* Update readme with zarr data info

* Remove numpy from util.py

* Review revisions

* Fix file system test
2020-09-24 18:02:15 -04:00
Tom Kralidis c579b6ddb2 fix failing test (#541) 2020-09-24 15:03:45 -04:00
Tom Kralidis 2ea8a5f6cb Misc fixes 2020 09 23 (#540)
* fix OACov output headers, safeguard coverage plugin loading

* fix ref, add query debug logging

* normalize mimetypes
2020-09-23 21:01:32 -04:00
Tom Kralidis f36e16d7e9 various fixes (#538)
* fix version output

* sort directory names

* unify tile tests

* fix typo in docs

* set OpenAPI info.version to pygeoapi version, add tiles output format

* align request/response headers, update not found tiles to return 404
2020-09-21 12:37:04 +02:00
Francesco Bartoli 4f52536051 Add tiles ogc api (#419)
* Add ogc api tiles implementation

* Add json response to tiles endpoint

* Return correct json payload

* Fix url generation

* Add vector tile layer with clickable items

* Fix typo

* Fix oafeat collection tiles link

* Add format to metadata link

* Fix almost all flake8 errors

* Fix metadata link

* Add routes to serving tiles

* Refactor tile implementation with an own module

* Add http client for mvt

* Fix visibility of tiles template

* Fix tiles in collection item page

* Change source to data coherently with stac provider

* Fix flake8 errors

* Fix test for conformance classes

* Fix flake8 errors


Fix flake8 errors

* Refactor to align with new configuration structure

* Fix MVT plugin name

* Add openapi tiles path and schemas

* Add ogc api tiles implementation

* Add json response to tiles endpoint

* Return correct json payload

* Fix url generation

* Add vector tile layer with clickable items

* Fix typo

* Fix oafeat collection tiles link

* Add format to metadata link

* Fix almost all flake8 errors

* Fix metadata link

* Add routes to serving tiles

* Refactor tile implementation with an own module

* Add http client for mvt

* Fix visibility of tiles template

* Fix tiles in collection item page

* Change source to data coherently with stac provider

* Fix flake8 errors

* Fix test for conformance classes

* Fix flake8 errors


Fix flake8 errors

* Refactor to align with new configuration structure

* Fix MVT plugin name

* Add openapi tiles path and schemas

* Fix flake errors

* Fix merge and tests

* Add test configuration and data for tiles

* Change mapbox vector tile provider name

* Fix provider inspection and filter

* Fix tiles rendering in collection page

* Add mimetype to the mvt provider

* Fix mimetype

* Rename functions accordingly to singular and plural forms

* Fix tiling schemes in tiles template

* Serve mvt through the pygeoapi route

* Refactor and add test for collection tiles

* Fix test for not being tiles

* Remove httpx dependency

* Fix missing comma

* Fix tests

* Fix flake8

* Add initial metadata implementation

* Fix route with preprocess

* Fix tiles template


Fix tiles template

* Complete tileset api and template

* Fix flake8

* Complete metadata with tilejson response

* Add support for local vector tiles

* Fix flake8

* Fix plugin tile object and provider name

* Add a todo comment

* testing alternate plugin setup, fix some HTML, add OpenAPI endpoint

* remove URI template

* add docs

* Fix failing test

* Fix flake8 error

Co-authored-by: Francesco Bartoli <francesco.bartoli@wfp.org>
Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2020-09-20 09:50:40 -04:00
Tom Kralidis 029d8c85f7 support xarray single time dimensions (#531) 2020-09-15 17:21:47 -04:00
Tom Kralidis 2fb20c59f6 fix coverage query with no parameters (#528)
* fix file dump, update docs

* fix file dump, update docs

* fix file dump, update docs

* support data read by file or network
2020-09-11 15:19:11 -04:00
Tom Kralidis 4d2f229c88 fix rasterio driver output (#527)
* fix rasterio driver output

* fix rasterio driver output

* rename driver
2020-09-10 16:24:29 -04:00
Tom Kralidis daf9b19195 [WIP] various xarray updates (#526)
* various xarray updates

* fix ref

* add support non-numeric axis subsetting, HTTP 204 on no content

* add info on COADS test data
2020-09-10 10:36:51 -04:00