71 Commits

Author SHA1 Message Date
Tom Kralidis 2f33ff5d15 update records properties (#1391)
* update records properties

* fix
2023-11-06 21:12:52 +02:00
Alexander Pilz d9c377ef62 Feature/mongodb jobmanager (#1186)
* MongoDB Jobmanager

* Add MongoDB-Jobmanager

Example configuration included

* Requested Fixes

Fixes Issues mentioned in latest Review

* Flake8 fixes

Code is fixed on the basis of a flake8 report

* Update ogcapi-processes.rst

* Update ogcapi-processes.rst

* Flake8 fixes

Fixes of flake8 errors
2023-03-30 07:38:43 -04:00
Benjamin Webb e1af5e1ca5 Add json-ld templating for feature collection item (#868)
* Render template if specified in config and provide example config

Update linked_data.py

Merge branch 'geopython:master' into jsonld

Rended from json-ld output

Render jinja2 json-ld template from json-ld output instead of json output

Merge branch 'geopython:master' into jsonld

Add documentation

- Add documentation
- Add test to workflow

Update pygeoapi-test-config.yml

Update test_api.py

Update api.py

Update linked_data.py

Move template declaration in configuration

Update docs

Update configuration.rst

Update configuration.rst

* Updates per requested changes

* Fix spelling

* Fix json-ld template pathing

* Remove  root path for the templating

* Move json-ld template from api.py

- Move single item json-ld templating to inside geojson2jsonld
- Reformat json-ld configuration for context and item_template to children of json-ld block
- Update docs and example configurations

* Fix ref

* Use FileSystemLoader to control template search path

search for templates is in order of `template_paths` list

* s/json-ld/linked-data/ig

rename json-ld to more generic name
2023-02-15 21:12:19 -05:00
Tom Kralidis 608fba5664 add support for OAMaps 🗺️ (#668) 2022-12-01 11:55:46 +02:00
Tom Kralidis 5d072f4efa OAProc updates (#840)
* update OAProc jobs as a top-level resource (#838)

* update config

* update copyright year
2022-01-04 08:43:39 -05:00
Benjamin Webb 205ff3002c Add Gzip to response types (#795)
* Add gzip to API.py

* pytest for gzip

* fix pytest

* Update test_api.py

* Add gzip to server block of configuration files

* Update api.py to include gzip opts from config

Update logic behind gzip compression  such that compression will only happen when gzip in the Accept-Encoding request header and config.server.gzip is True. Use server charset for encoding/decoding and include charset in Content-Type when compressed.

* Update pytest to use gzip config options

Update pytest to test with (gzip in Accept-Encoding & server.gzip = False), (gzip in Accept-Encoding & server.gzip = True), (gzip not in Accept-Encoding & server.gzip = True)

* Better Content Negotiation (#7)

* Change content negotiation in _get_format()

* Add to pytest and force h to always be string

* Force F_JSON to default response

* Update test_api.py for utf-16 encoding

* better content negotiation pytest.

* Add comments to pytest

* Fix flake 8

* Improve docstring for gzip decorator function
2021-10-27 20:17:02 -04:00
Tom Kralidis 26ae0bfe46 Update pygeoapi-config.yml 2021-09-21 07:15:37 -04:00
Tom Kralidis 89eff49ce0 add support for configuration schema and validation (#553) (#719)
* add support for configuration schema and validation (#553)

* rename types

* minor doc fix

* update i18n regexes

* make openapi click options required arguments
2021-07-06 17:01:05 -04:00
Tom Kralidis 48efbdec53 various fixes (#706) 2021-06-09 06:09:32 -04:00
Sander Schaminee 023f24d26b Multilingual support (alternative) (#664)
* Created localization (l10n) module + tests. Added l10n support to API and plugins (wip).

* Big refactor:

* All routed API methods are now decorated by @pre_process (consistency) and no longer have a headers+format argument but a request argument (**kwargs also removed)
* The pre_process decorator turns an incoming Flask/Starlette request into a generic APIRequest instance
* The new APIRequest class extracts all relevant info (params, data, locale, etc.) from the request and exposes them as properties
* Removed a lot of boilerplate (i.e. format checking) and wrapped that into methods
* Updated server-specific API calls in each route method (pass entire request object, not headers and query params)

* Several improvements and fixes:

* Updated OpenAPI page with "l" query param
* Added example translations (metadata)
* Changed plugin signature: added explicit locale attribute (instead of **kwargs)
* Moved locale processing to get_plugin_locale() function in l10n module
* API should pass raw requested locale to plugins, locale should always be set
* Fixed API tests and added APIRequest tests
* Prepared utils.py for Jinja2 i18n extension
* Rebased on commit b40297a8 and fixed compatibility with #661 and #662

* Updated documentation for language support

* Rebased and fixed compatibility with PR #658:

* Fixed EDR provider signature (added locale)
* Fixed EDR API routes and query function (and improved parameter-name handling)
* Fixed EDR tests

* Translate entire config in render_j2_template for requested locale:

* Added new translate_dict function to l10n module (+ tests)
* Updated all render_j2_template calls with locale parameter
* Updated pygeoapi-test-config.yml with some language structs

* Minor improvements

* support both 'language' and 'languages' property in server config and provider definitions
* renamed and modified translate_dict() to more generic translate_struct() function (l10n module)
* remove Content-Language header from provider responses if provider has no language support and format is json(ld)
* updated tests

* Leave provider locale handling to API

* Moved code to determine locale from providers to API class (and remove for formatters and processes)
* Removed locale parameter from plugin __init__ signatures
* Removed locale parameter from load_plugin()
* Added **kwargs to provider implementations for get, query, get_metadata, get_coverage_domainset and get_coverage_rangetype method signatures
* Added language=<locale> to all API calls to provider get, query, get_metadata, get_coverage_domainset and get_coverage_rangetype methods

* Use 'lang' instead of 'l' as language query parameter

* Updated Open API
* Updated documentation
* Fixed tests

* Implemented requested PR changes:

* Added usage examples to the APIRequest docstring
* Removed language support from coverage functions
* Updated plugins.rst and language.rst to match new behavior
* Removed language struct from resource links in pygeoapi-config.yml
* Rebased on latest master (fixed test_api.py)

* Rebased and applied fixes:

* Data property in APIRequest now is an awaitable attribute (fixed for Starlette compatibility)
* Named references to 'l' parameter to 'lang'

* Final changes/improvements:

* Make sure that Content-Language is always set;
* Added more tests to ensure that the default language returned is the first configured language (if no language was requested by the user);
* Updated docs;
* Replaced re-occuring strings with constants in api.py;
* Fixed Flake8 checks.

* add missing async to starlette routes (#704)

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2021-06-08 18:46:35 -04:00
Tom Kralidis d92f8012d1 update OARec metadata models per spec changes (#701) 2021-05-31 18:27:53 -04:00
Jo 7a9a8b19bf - Updated attribution to reflect the change of background map (#700)
Co-authored-by: doublebyte1 <info@doublebyte.net>
2021-05-31 10:41:42 -04:00
Jo ba3bf1621f - changed default background map to OSM (#697)
Co-authored-by: doublebyte1 <info@doublebyte.net>
2021-05-27 17:43:46 -04:00
Tom Kralidis 7e568f5838 fix 5 minute install (#686) (#691) 2021-05-24 21:49:11 -04:00
Tom Kralidis 951a1fb486 implement OGC EDR API (#658)
* implement OGC EDR API
* add docs/tests
* fix tests
2021-03-15 17:37:40 +02:00
paul van genuchten 1bd1ad509b add contact and url's to record (#636) 2021-02-21 16:18:35 -05:00
Tom Kralidis cc6dc2e33b update config, fix ref (#634) 2021-02-18 21:54:48 -05:00
Tom Kralidis 0c1d98e868 implement OGC API - Records (#404) (#631) 2021-02-18 17:56:23 -05:00
paul van genuchten 96675af970 a proposal for #133 (#608)
indicate which field to use as label for the item
2021-02-18 06:12:40 -05:00
Richard Law 60202129ec implement processing manager (#365)
* add process manager

* process manager updating

* control connectivity from inside manager

* missing method on execute_process function signature

* adds tinydb depenedency for requirements-provider

* adds some processes tests

* process manager under server config, not a process

* additional html rendering for jobs and processes

* fixes typo in docstring

* complete html templating for jobs and processes

* considers timezone when formatting processing start/end datetimes

* expanded locales; locale and tz environment variables

* makes environment TZ a global from pygeoapi/__init__.py

* don't redefine built-in format

* adds placeholder values in process form if given as examples

* placeholder in form handles null example

* changes to process progress update

* updates and changes to processing web UI

* adds python3-distutils dep, and env changes for TZs and locales

* adds tinydb dependency for processes

* Adds libpq-dev and related deps for processing - should eventually be reconciled

* minor margin change

* fix bug in postJob.js that prevented results from displaying

* move process elements around to economise on space

* adds second, complex process

* post data may be bytes

* make external process example

* adds redis process manager (may be moved to third-party plugin eventually)

* reintroduce default async processing; refactoring managers; improving sync/async processing ui

* throw pygeoapi ManagerExecuteError, not redis exception

* adds test for async-execute

* restores original test host

* separates async execute handler function

* adds support for file uploads as processing inputs

* adds endpoint for file download as process output

* naming consistency

* Fix job submission form in case server url is relative (#549)

`new URL()` fails if the first argument isn't a complete url

* Implement job deletion in UI, api and tinydb

* Also allow deletion from job list

The UI could be fancier by reloading the page or just removing the
deleted job right away, but I don't want to start building that now.

* Delete output file when deleting a job

* typo in test case

* move delete button in job view

* remove Dockerfile changes (cf #470)

* removes some docs/source*.rst absent in master

* fixes whitespace

* fixes pip installs

* removes more locale-specific content, cf 70

* fixes duplicate app following merge

* removes redis as default plugin

* removes ENV_TZ, cf #407

* fix typo

* fixes syntax error

* process execution test cleanup

* Fix format detection in execute_process (#589)

Before, html was always selected.

* remove db files in code directories, fix flake8, specific functionality

* remove interactive capabilities in lieu of Swagger

* Add basic openapi documentation for job detail endpoint (#590)

It's somewhat superficial still, e.g. jobId is missing as
proper parameter in the UI.

* Show elapsed time for currently running jobs (#591)

* flake8, jobs workflow

* update docs, DB path in test config

* more code cleanup

* manager workflow cleanup

* add default dummy manager

* update docs

* fix routes

* fixed per comments

* fix per comments

* fix per comments

* fix travis

* simply describe process logic

* simplify describe process logic

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
Co-authored-by: Jorge de Jesus <jorge.jesus@gmail.com>
Co-authored-by: totycro <bernhard.mallinger@eox.at>
Co-authored-by: Richard Law <richard.law@cartoncloud.com.au>
2020-12-22 06:53:31 -05:00
paul van genuchten b4f936b843 reproject dataset bounds to 4326 while analyzing stac content (#559)
* reproject dataset bounds to 4326 while analyzing stac content

* fixes code formatting, indicated by tests
adds the shp.xml file as asset

* fix indent

* as suggested by @tomkralidis, only reproject if projection != 4326 or nill

* better param names
2020-11-01 22:52:46 -05:00
Kevin Ngai 90ab772c7c Add templates config yml split into path and static (#523)
- updated configuration.rst doc for templates config
- added docs on HTML templating; dedicated page
- typo fix in configuration.rst
- added debug logging for templates path
- added static config in starlette_app.py
- flake8 fixes
2020-09-29 15:44:00 -04:00
Tom Kralidis da824fba8f add support for OGC API - Coverages (#110) (#516)
* add support for OGC API - Coverages

* fix coverage CRS ref

* fix ref to OACov schemas for testing

* move spectral testing to after_success

* update docs

* add mask param to rasterio provider
2020-08-21 09:52:17 -04:00
Tom Kralidis 28157426f5 update providers to allow 1..n representations (#489) 2020-07-13 06:28:11 -04:00
Francesco Bartoli 3ec0570274 Add spectral command to travis for linting the openapi document
Fix trailing slash in server url


Fix tag not declared in the root mapping key


Fix missing operationId


Add spectral command to travis for linting the openapi document
2020-07-02 17:26:18 +02:00
Tom Kralidis 1d576fd752 fix OpenAPI generation (#428)
* fix OpenAPI generation

* fix
2020-04-27 11:15:04 -04:00
Tom Kralidis 3debd0c750 update configuration to be resource specific (#393) 2020-04-26 09:57:07 -04:00
Tom Kralidis b12cc966a1 Issue 277 documentation (#392)
* first pass docs scaffolding

* more updates

* introduction, how pygeoapi works, installation

* start documenting configuration

* configuration and linked data

* administration

* administration, running

* docker

* linked data into config

* minor updates

* api docs

* add mod_wsgi info

* add OpenAPI refresh

* add tour, data-publishing

* license

* OAPIF

* data publishing: OAPIF

* publishing

* plugins

* contributing, support

* development, ogc compliance

* fix flake8
2020-04-12 17:27:31 -04:00
Tom Kralidis c151579991 add STAC catalog functionality (#389)
* add STAC catalog functionality

* add example to config, add sample raster data

* fix tests
2020-04-12 12:08:17 -04:00
Tom Kralidis d0d7ed48da Implement configurable OGC schema locations (#347) 2020-01-20 14:56:04 -05:00
Richard Law 17ed14107d Add JSON-LD representations (#246)
* Changes to sample pygeopai-config.yml
- improved terms_of_service and contact.url for SEO
- fixes typo for contact.instructions

* whitespace

* dynamically embed JSON-LD representation in head

* json-led representation for root, collections, and collection

* updates sample configuration with metadata that won't cause JSON-LD validation issues

* adds support for feature-level JSON-LD representations of collections and items

* improves consistency between microdata and JSON-LD root metadata

* valid Dataset spatial; and better use of @id

* role →  instructions

* adds jsonld f param to open api definition

* working with new temporal extent interface

* removes comment

* identify and retain NIR/HTTP IDs rather than construct them with reference to pygeoapi

* don't pop a feature.properties id, to retain JSON-LD reference integrity

* add/update tests for inclusion of json-ld responses

* bug fixes

* fix bug where id was assumed to be string

* better url checking

* renames format variable to avoid reserved word

* removes top-level CRS in obs sample config

* add json-ld textMimeType for serverless

* adds flask-cors dependency to allow cors

* moves json-ld-requesting script to bottom of body

* corrects schema.org url

* fixes urls

* adds pyld dependency

* adds tests for json-ld representations (incomplete)

* more tests

* make pyld a dev requirement only

* linting

* changes from revision

* removes merge artifact
2019-12-19 08:33:42 -05:00
Tom Kralidis 7e9b2c49bf Update sample title (#275)
* update test collection titles

* update test collection titles
2019-10-13 09:19:41 -04:00
Tom Kralidis 611dffcb6b OpenAPI and datetime updates (#244)
* support CRS in collections, refactor OpenAPI responses, update collections model

* add test for invalid format for /api
2019-09-24 10:14:36 -04:00
Tom Kralidis 5ab45769db disable local cors for default dev install 2019-07-18 14:16:21 +00:00
Tom Kralidis fa510a6109 simplify configuration (#154) (#188) 2019-06-20 17:46:19 -04:00
Tom Kralidis 1fd8056b7c address OSGeo incubation committee comments 2019-06-04 03:17:00 +00:00
Tom Kralidis 2c47ee1161 update attribution 2019-05-27 07:00:45 -04:00
Tom Kralidis 145bf632a2 make map URL configurable (thanks @alexandreleroux) 2019-05-27 06:26:40 -04:00
Just van den Broecke e9fc22e72c Issue #127 ogrpaging for all OGR source types (#128)
* #127 Paging for ALL OGR Drivers, tested: GPKG, Shapefile, SQLite, GeoJSON

* #127 Paging for ALL OGR Drivers, tested: GPKG, Shapefile, SQLite, GeoJSON fix SQLite Test

* #127 Paging for ALL OGR Drivers require GDAL greater or eq 2.2

* geopython/pygeoapi#112 geopython/pygeoapi#127 close OGR  ResultSet when layer from ExecuteSQL created

* geopython/pygeoapi#112 geopython/pygeoapi#127 grr flake8 fix
2019-05-21 11:34:54 -04:00
Just van den Broecke 4e5233f39d Introduce OGRProvider: WFS backend support (#119)
* #89 fix check_format function

* #89 fix check_format function - fix default return

* geopython/pygeoapi#58 first version OGRProvider with working WFS tests/config

* geopython/pygeoapi#58 second version, ogr2ogr-like reprojection support - more to follow

* geopython/pygeoapi#58 3rd version, support Sourcetypes SHP and GPKG with tests+travis stuff

* geopython/pygeoapi#58 fixed geopython/pygeoapi#95 for OGR Provider and its tests

* geopython/pygeoapi#58 use more performant backend-WFS in pygeoapi-config.yml

* #58 4th version: WFS backend ok, including OGR Python paging gotchas fixed ready for PR

* #58 fix Travis build for GDAL Python bindings

* #58 fix Travis build: Unit tests failed: missing Shapefile .zip now added

* #58 #119 rework from PR comments: config, tests, quotes
2019-05-15 12:57:12 -04:00
Tom Kralidis 946ab0da3d update pygeoapi website (#99) 2019-04-26 10:35:12 -04:00
Tom Kralidis 2518d4b0a8 add processing framework (#84) 2019-03-05 06:36:43 -05:00
Tom Kralidis 2d7c86e47b trim trailing spaces 2018-11-26 19:50:17 -05:00
Jorge Samuel Mendes de Jesus c621b21391 Postgis provider support (#72)
* 	new file:   docker/Dockerfile
Dockerfile for pygeoapi

* testing postgis on travis

* testing 9.6

* install postgis as package

* create database

* postgis dataset - travis

* missing path

* 	modified:   tests/data/hotosm_bdi_waterways.sql.gz

Change in name of dataset

* postgresql code

* 	modified:   pygeoapi/provider/postgresql.py
update with origin

* Postgresql working provider, typo in geopackage

* update to travis. Change on gpkg

* missing dep (psycopg2)

* pytest

* problem with operator in travis

* ProviderConnectionError implementation

* postgresql with schema support

* Changes requested by @tomkralidis

* missing pygeoapi-config.yml

* delete openapi.yml

deleted openapi.yml
2018-11-16 11:53:31 -05:00
Jorge Samuel Mendes de Jesus 66266102a3 Geopackage support (#64)
* 	new file:   docker/Dockerfile
Dockerfile for pygeoapi

* typo

* typo

* geopackage provider

* test travis

* changed href to http

* incorrect field in yml

* missing flake8 check

* POI test dataset

* unit tests

* working unitttests and update to readme

* .gitignore docker

* requests for PR #64

* Update __init__.py

Flake8 whitespace

* flake8 fix and removal of incorrect dockerfile
2018-09-11 12:32:11 -04:00
Jorge Samuel Mendes de Jesus c1e9943c48 Docker composition (ES docker + pygeoapi docker) (#31)
* 	new file:   docker/Dockerfile
Dockerfile for pygeoapi

* Using 0.0.0.0 in pygeoapi-config otherwise no docker for flask

* ES and pygeoapi docker

* Final file structure

* generic port 5000 and localhost

* sqlit3 suport in alpine and fixes on sqlite provider

* B4 sync with upstream

* Working composition problems with ES features

* Cosmetic changes update to README.md

* using load_es_data.py from github local openapi.yml

* added docker pull only of pygeoapi

* b4 rebase

* Using 0.0.0.0 in pygeoapi-config otherwise no docker for flask

* ES and pygeoapi docker

* Final file structure

* generic port 5000 and localhost

* sqlit3 suport in alpine and fixes on sqlite provider

* B4 sync with upstream

* Cosmetic changes update to README.md

* using load_es_data.py from github local openapi.yml

* added docker pull only of pygeoapi

* b4 rebase

* updates and sync

* missing proper local.config.yml

* wait-for pygeoapi

* sync with master config

* 	deleted:    docker/ES/wait-for-it.sh
	new file:   docker/pygeoapi/docker-entrypoint.sh
	new file:   docker/pygeoapi/run_pygeoapi.sh
	openapi.yml
	pygeoapi/provider/__sqlite.py

Missing sh files

* missing set +e

* wait for ES to be yellow or green

* More detailed Readme

* 	new file:   .gitignore

* simple image docker

* missing docker run

* 	new file:   Dockerfile
	new file:   docker.config.yml
	new file:   run_pygeoapi.sh
Simple image

* typos on README
2018-07-03 14:52:17 -04:00
Tom Kralidis 0dfb9adc4b fix CSV driver refs and config 2018-05-20 20:18:17 -04:00
Tom Kralidis b822f8137b remove shapely dependency and cast CSV geometries as common use case of x/y columns 2018-05-03 19:23:40 -04:00
Tom Kralidis 6d4a58b454 add support for hreflang in links 2018-05-03 19:56:31 +00:00
Tom Kralidis 515da29a41 add support for custom templates definition 2018-05-03 00:55:02 +00:00