Commit Graph

52 Commits

Author SHA1 Message Date
Richard Law 60202129ec implement processing manager (#365)
* add process manager

* process manager updating

* control connectivity from inside manager

* missing method on execute_process function signature

* adds tinydb depenedency for requirements-provider

* adds some processes tests

* process manager under server config, not a process

* additional html rendering for jobs and processes

* fixes typo in docstring

* complete html templating for jobs and processes

* considers timezone when formatting processing start/end datetimes

* expanded locales; locale and tz environment variables

* makes environment TZ a global from pygeoapi/__init__.py

* don't redefine built-in format

* adds placeholder values in process form if given as examples

* placeholder in form handles null example

* changes to process progress update

* updates and changes to processing web UI

* adds python3-distutils dep, and env changes for TZs and locales

* adds tinydb dependency for processes

* Adds libpq-dev and related deps for processing - should eventually be reconciled

* minor margin change

* fix bug in postJob.js that prevented results from displaying

* move process elements around to economise on space

* adds second, complex process

* post data may be bytes

* make external process example

* adds redis process manager (may be moved to third-party plugin eventually)

* reintroduce default async processing; refactoring managers; improving sync/async processing ui

* throw pygeoapi ManagerExecuteError, not redis exception

* adds test for async-execute

* restores original test host

* separates async execute handler function

* adds support for file uploads as processing inputs

* adds endpoint for file download as process output

* naming consistency

* Fix job submission form in case server url is relative (#549)

`new URL()` fails if the first argument isn't a complete url

* Implement job deletion in UI, api and tinydb

* Also allow deletion from job list

The UI could be fancier by reloading the page or just removing the
deleted job right away, but I don't want to start building that now.

* Delete output file when deleting a job

* typo in test case

* move delete button in job view

* remove Dockerfile changes (cf #470)

* removes some docs/source*.rst absent in master

* fixes whitespace

* fixes pip installs

* removes more locale-specific content, cf 70

* fixes duplicate app following merge

* removes redis as default plugin

* removes ENV_TZ, cf #407

* fix typo

* fixes syntax error

* process execution test cleanup

* Fix format detection in execute_process (#589)

Before, html was always selected.

* remove db files in code directories, fix flake8, specific functionality

* remove interactive capabilities in lieu of Swagger

* Add basic openapi documentation for job detail endpoint (#590)

It's somewhat superficial still, e.g. jobId is missing as
proper parameter in the UI.

* Show elapsed time for currently running jobs (#591)

* flake8, jobs workflow

* update docs, DB path in test config

* more code cleanup

* manager workflow cleanup

* add default dummy manager

* update docs

* fix routes

* fixed per comments

* fix per comments

* fix per comments

* fix travis

* simply describe process logic

* simplify describe process logic

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
Co-authored-by: Jorge de Jesus <jorge.jesus@gmail.com>
Co-authored-by: totycro <bernhard.mallinger@eox.at>
Co-authored-by: Richard Law <richard.law@cartoncloud.com.au>
2020-12-22 06:53:31 -05:00
paul van genuchten b4f936b843 reproject dataset bounds to 4326 while analyzing stac content (#559)
* reproject dataset bounds to 4326 while analyzing stac content

* fixes code formatting, indicated by tests
adds the shp.xml file as asset

* fix indent

* as suggested by @tomkralidis, only reproject if projection != 4326 or nill

* better param names
2020-11-01 22:52:46 -05:00
Kevin Ngai 90ab772c7c Add templates config yml split into path and static (#523)
- updated configuration.rst doc for templates config
- added docs on HTML templating; dedicated page
- typo fix in configuration.rst
- added debug logging for templates path
- added static config in starlette_app.py
- flake8 fixes
2020-09-29 15:44:00 -04:00
Tom Kralidis da824fba8f add support for OGC API - Coverages (#110) (#516)
* add support for OGC API - Coverages

* fix coverage CRS ref

* fix ref to OACov schemas for testing

* move spectral testing to after_success

* update docs

* add mask param to rasterio provider
2020-08-21 09:52:17 -04:00
Tom Kralidis 28157426f5 update providers to allow 1..n representations (#489) 2020-07-13 06:28:11 -04:00
Francesco Bartoli 3ec0570274 Add spectral command to travis for linting the openapi document
Fix trailing slash in server url


Fix tag not declared in the root mapping key


Fix missing operationId


Add spectral command to travis for linting the openapi document
2020-07-02 17:26:18 +02:00
Tom Kralidis 1d576fd752 fix OpenAPI generation (#428)
* fix OpenAPI generation

* fix
2020-04-27 11:15:04 -04:00
Tom Kralidis 3debd0c750 update configuration to be resource specific (#393) 2020-04-26 09:57:07 -04:00
Tom Kralidis b12cc966a1 Issue 277 documentation (#392)
* first pass docs scaffolding

* more updates

* introduction, how pygeoapi works, installation

* start documenting configuration

* configuration and linked data

* administration

* administration, running

* docker

* linked data into config

* minor updates

* api docs

* add mod_wsgi info

* add OpenAPI refresh

* add tour, data-publishing

* license

* OAPIF

* data publishing: OAPIF

* publishing

* plugins

* contributing, support

* development, ogc compliance

* fix flake8
2020-04-12 17:27:31 -04:00
Tom Kralidis c151579991 add STAC catalog functionality (#389)
* add STAC catalog functionality

* add example to config, add sample raster data

* fix tests
2020-04-12 12:08:17 -04:00
Tom Kralidis d0d7ed48da Implement configurable OGC schema locations (#347) 2020-01-20 14:56:04 -05:00
Richard Law 17ed14107d Add JSON-LD representations (#246)
* Changes to sample pygeopai-config.yml
- improved terms_of_service and contact.url for SEO
- fixes typo for contact.instructions

* whitespace

* dynamically embed JSON-LD representation in head

* json-led representation for root, collections, and collection

* updates sample configuration with metadata that won't cause JSON-LD validation issues

* adds support for feature-level JSON-LD representations of collections and items

* improves consistency between microdata and JSON-LD root metadata

* valid Dataset spatial; and better use of @id

* role →  instructions

* adds jsonld f param to open api definition

* working with new temporal extent interface

* removes comment

* identify and retain NIR/HTTP IDs rather than construct them with reference to pygeoapi

* don't pop a feature.properties id, to retain JSON-LD reference integrity

* add/update tests for inclusion of json-ld responses

* bug fixes

* fix bug where id was assumed to be string

* better url checking

* renames format variable to avoid reserved word

* removes top-level CRS in obs sample config

* add json-ld textMimeType for serverless

* adds flask-cors dependency to allow cors

* moves json-ld-requesting script to bottom of body

* corrects schema.org url

* fixes urls

* adds pyld dependency

* adds tests for json-ld representations (incomplete)

* more tests

* make pyld a dev requirement only

* linting

* changes from revision

* removes merge artifact
2019-12-19 08:33:42 -05:00
Tom Kralidis 7e9b2c49bf Update sample title (#275)
* update test collection titles

* update test collection titles
2019-10-13 09:19:41 -04:00
Tom Kralidis 611dffcb6b OpenAPI and datetime updates (#244)
* support CRS in collections, refactor OpenAPI responses, update collections model

* add test for invalid format for /api
2019-09-24 10:14:36 -04:00
Tom Kralidis 5ab45769db disable local cors for default dev install 2019-07-18 14:16:21 +00:00
Tom Kralidis fa510a6109 simplify configuration (#154) (#188) 2019-06-20 17:46:19 -04:00
Tom Kralidis 1fd8056b7c address OSGeo incubation committee comments 2019-06-04 03:17:00 +00:00
Tom Kralidis 2c47ee1161 update attribution 2019-05-27 07:00:45 -04:00
Tom Kralidis 145bf632a2 make map URL configurable (thanks @alexandreleroux) 2019-05-27 06:26:40 -04:00
Just van den Broecke e9fc22e72c Issue #127 ogrpaging for all OGR source types (#128)
* #127 Paging for ALL OGR Drivers, tested: GPKG, Shapefile, SQLite, GeoJSON

* #127 Paging for ALL OGR Drivers, tested: GPKG, Shapefile, SQLite, GeoJSON fix SQLite Test

* #127 Paging for ALL OGR Drivers require GDAL greater or eq 2.2

* geopython/pygeoapi#112 geopython/pygeoapi#127 close OGR  ResultSet when layer from ExecuteSQL created

* geopython/pygeoapi#112 geopython/pygeoapi#127 grr flake8 fix
2019-05-21 11:34:54 -04:00
Just van den Broecke 4e5233f39d Introduce OGRProvider: WFS backend support (#119)
* #89 fix check_format function

* #89 fix check_format function - fix default return

* geopython/pygeoapi#58 first version OGRProvider with working WFS tests/config

* geopython/pygeoapi#58 second version, ogr2ogr-like reprojection support - more to follow

* geopython/pygeoapi#58 3rd version, support Sourcetypes SHP and GPKG with tests+travis stuff

* geopython/pygeoapi#58 fixed geopython/pygeoapi#95 for OGR Provider and its tests

* geopython/pygeoapi#58 use more performant backend-WFS in pygeoapi-config.yml

* #58 4th version: WFS backend ok, including OGR Python paging gotchas fixed ready for PR

* #58 fix Travis build for GDAL Python bindings

* #58 fix Travis build: Unit tests failed: missing Shapefile .zip now added

* #58 #119 rework from PR comments: config, tests, quotes
2019-05-15 12:57:12 -04:00
Tom Kralidis 946ab0da3d update pygeoapi website (#99) 2019-04-26 10:35:12 -04:00
Tom Kralidis 2518d4b0a8 add processing framework (#84) 2019-03-05 06:36:43 -05:00
Tom Kralidis 2d7c86e47b trim trailing spaces 2018-11-26 19:50:17 -05:00
Jorge Samuel Mendes de Jesus c621b21391 Postgis provider support (#72)
* 	new file:   docker/Dockerfile
Dockerfile for pygeoapi

* testing postgis on travis

* testing 9.6

* install postgis as package

* create database

* postgis dataset - travis

* missing path

* 	modified:   tests/data/hotosm_bdi_waterways.sql.gz

Change in name of dataset

* postgresql code

* 	modified:   pygeoapi/provider/postgresql.py
update with origin

* Postgresql working provider, typo in geopackage

* update to travis. Change on gpkg

* missing dep (psycopg2)

* pytest

* problem with operator in travis

* ProviderConnectionError implementation

* postgresql with schema support

* Changes requested by @tomkralidis

* missing pygeoapi-config.yml

* delete openapi.yml

deleted openapi.yml
2018-11-16 11:53:31 -05:00
Jorge Samuel Mendes de Jesus 66266102a3 Geopackage support (#64)
* 	new file:   docker/Dockerfile
Dockerfile for pygeoapi

* typo

* typo

* geopackage provider

* test travis

* changed href to http

* incorrect field in yml

* missing flake8 check

* POI test dataset

* unit tests

* working unitttests and update to readme

* .gitignore docker

* requests for PR #64

* Update __init__.py

Flake8 whitespace

* flake8 fix and removal of incorrect dockerfile
2018-09-11 12:32:11 -04:00
Jorge Samuel Mendes de Jesus c1e9943c48 Docker composition (ES docker + pygeoapi docker) (#31)
* 	new file:   docker/Dockerfile
Dockerfile for pygeoapi

* Using 0.0.0.0 in pygeoapi-config otherwise no docker for flask

* ES and pygeoapi docker

* Final file structure

* generic port 5000 and localhost

* sqlit3 suport in alpine and fixes on sqlite provider

* B4 sync with upstream

* Working composition problems with ES features

* Cosmetic changes update to README.md

* using load_es_data.py from github local openapi.yml

* added docker pull only of pygeoapi

* b4 rebase

* Using 0.0.0.0 in pygeoapi-config otherwise no docker for flask

* ES and pygeoapi docker

* Final file structure

* generic port 5000 and localhost

* sqlit3 suport in alpine and fixes on sqlite provider

* B4 sync with upstream

* Cosmetic changes update to README.md

* using load_es_data.py from github local openapi.yml

* added docker pull only of pygeoapi

* b4 rebase

* updates and sync

* missing proper local.config.yml

* wait-for pygeoapi

* sync with master config

* 	deleted:    docker/ES/wait-for-it.sh
	new file:   docker/pygeoapi/docker-entrypoint.sh
	new file:   docker/pygeoapi/run_pygeoapi.sh
	openapi.yml
	pygeoapi/provider/__sqlite.py

Missing sh files

* missing set +e

* wait for ES to be yellow or green

* More detailed Readme

* 	new file:   .gitignore

* simple image docker

* missing docker run

* 	new file:   Dockerfile
	new file:   docker.config.yml
	new file:   run_pygeoapi.sh
Simple image

* typos on README
2018-07-03 14:52:17 -04:00
Tom Kralidis 0dfb9adc4b fix CSV driver refs and config 2018-05-20 20:18:17 -04:00
Tom Kralidis b822f8137b remove shapely dependency and cast CSV geometries as common use case of x/y columns 2018-05-03 19:23:40 -04:00
Tom Kralidis 6d4a58b454 add support for hreflang in links 2018-05-03 19:56:31 +00:00
Tom Kralidis 515da29a41 add support for custom templates definition 2018-05-03 00:55:02 +00:00
Tom Kralidis af5aa82e26 add properties support 2018-04-23 01:54:28 +00:00
Tom Kralidis e0cbf35f12 update item html templates, GeoJSON identifiers in test data 2018-04-08 00:34:22 +00:00
Tom Kralidis b736a2d59b s/SQlite/SQLite 2018-04-07 19:07:15 +00:00
Tom Kralidis c8b94c9a10 fix links, logging 2018-04-05 16:42:09 +00:00
Tom Kralidis 7683775d89 fix ref 2018-04-03 19:42:59 -04:00
Tom Kralidis deeebb6668 remove geometry type from config 2018-04-03 12:55:03 +00:00
Jorge Samuel Mendes de Jesus 1de0439e2d Spatiallite3 provider implementation (#17)
* 	new file:   docker/Dockerfile
Dockerfile for pygeoapi

* 	modified:   pygeoapi/provider/__init__.py
	new file:   tests/data/ne_110m_lakes.sqlite
	new file:   tests/json_marshmallow.py
	new file:   tests/test_sqlite_provider.py

Sqlit implementation and testing marshmallows

* 	new file:   provider/sqlite.py
	new file:   provider/tmp_parser.py
Sqlite provider

* Testing sqlalchemy

* query for sqlite3

* Countries dataset, message in assert

* yml config

* table in data link and query implemented without limit

* PR of refactor

* functional sqlite3 driver

* flake8

* pipreq for complete list of requirements

* updated readme with working examples, extra requirements

* typos, SQLite removed Dockerfile and ne_100m_lakes.sqlite

* update requirements

* pypandoc in requirements-dev.txt
2018-04-02 12:05:26 -04:00
Tom Kralidis 557a9e3480 implement bind configuration 2018-04-01 22:06:30 -04:00
Tom Kralidis c01829265e refactor provider initialization 2018-04-01 09:57:25 -04:00
Tom Kralidis d63c23850a first pass links 2018-03-29 16:50:19 +00:00
Tom Kralidis 403fa4648b Revert "remove application level CORS (leave to server)"
This reverts commit 8e931e644f.
2018-03-27 21:26:21 -04:00
Tom Kralidis 8e931e644f remove application level CORS (leave to server) 2018-03-27 21:18:30 -04:00
Tom Kralidis 9015b16e6d implement OpenAPI from single config 2018-03-27 21:15:43 -04:00
Tom Kralidis 43b44574f5 update config, add conformance 2018-03-21 09:18:59 -04:00
Tom Kralidis 2cd3621ddd refactor configuration, remove file:// 2018-03-09 12:21:54 -05:00
Tom Kralidis d6e4b1dc00 implement logging, first pass cli (#14) 2018-03-08 08:38:42 -05:00
Norman Barker 957df9b844 Swagger gen (#10)
* initial swagger gen

* removed support for html returns

* geojson configuration

* working swagger ui

* config rename

* observation example

* missing call in csv driver

* pr comments
2018-03-07 16:44:04 -05:00
Tom Kralidis d1d2a589b3 add CSV provider, misc updates 2018-03-07 00:08:05 +00:00
Matthew Perry 7829772ae6 example dataset for GeoJSON provider 2018-03-06 13:51:31 -07:00