Commit Graph

974 Commits

Author SHA1 Message Date
Tom Kralidis 70feda7deb add DOI (#1220) 2023-04-17 19:28:03 -04:00
Tom Kralidis 627f2228ab ES: change PUT update argument (#1216) 2023-04-12 12:19:10 -04:00
Ricardo Garcia Silva 5e9c82a34d Process manager generate job ids (#1209)
* Modified process manager to generate job id

* Continued to refactor in order to let process manager generate job id

* fix syntax error

* reorder imports
2023-04-12 06:11:13 -04:00
Just van den Broecke 7c6993719d add CRS Support for OGC API Feature pygeoapi Provider (#1174)
* OGC API - Features Part 2 (groundwork+CRS-BBOX) from PR #1155 - contributes to issue #1128

* #1128 provide conformance class for OAPIF Part 2 in /conformance page

* #1128 bitten by flake8...

* #1128 configurability CRS Feature Providers with syntax, defaults and tests

* #1128 configurability CRS Feature Providers refine for default values

* #1128 display supported CRSs in HTML Collection template

* #1128 config, mmetadata and tests for storageCRS and storageCrsCoordinateEpoch

* #1128 WIP for bbox-crs parameter support

* #1128 utility function and tests for default/mandatory supprted CRS list

* #1128 default supported CRS adaptation to OAPIF Part 2 standard

* #1128 grr flake8 whitespace

* #1128 start adding full API tests OGR for bbox-crs and crs parms

* #1128 fix flake8

* #1128 fix flake8 - install GDAL in workflow main for OGR tests

* #1128 fix flake8 - install GDAL in workflow main for OGR tests - need pip package?

* #1128 fix flake8 - install GDAL in workflow main for OGR tests - using libgdal-dev gdal-bin

* #1128 fix SensorThings test for main.yml Workflow

* #1128 fix SensorThings test for main.yml Workflow nr 2

* #1128 make all OGR tests working again

* #1128 make all OGR tests working again - flake8

* #1128 make all OGR tests working again - GeoSolutions WFS bbox

* #1128 #1155 add documentation for OGC OAPIF Part 2 CRS CRS BBOX support

* #1128 #1155 refine documentation for OGC OAPIF Part 2 CRS CRS BBOX support

* #1128 #1155 refine documentation to align with #1149

* #1128 #1155 rework from review OAS and pygeoapi config schema

* #1128 #1155 minor: compile Re for CRS URI only once as global var

* #1128 merge in changes from PR #1173 - fix missing import

* WIP Ogcapi features part 2 - Support for crs query parameter (#1149)

* feat(ogcapi_features_crs): start implementing crs support from ogcapi features part2

* Pass input and output CRSs WKT instead of crs transformation object

* fix longs lines and blank lines

* fix typo

* fix import for type annotation not supported by python version

* fix variable visibility in local scope

* fix tabs/spaces indentations

* Add support for the crs parameter to OGRProvider

* make flake8 happy

* Make crs transformation mechanism more consistent between PostgreSQL and OGR providers

* test(util): add two test functions in util.py

New functions: test_get_crs_from_uri and test_get_transform_from_crs

* fix too long lines...

* Update get_crs_from_uri and corresponding test function

* fix(get_crs_from_uri): make the error more explicit in if wrong crs uri format

* flake8 again...

* Keep support for source_srs/target_srs in config for OGRProvider

* revert changes made to pygeoapi-config-0.x.yml, overlap with PR 1155

* test: add test data and update test config file

* Extract 'crs' and 'storage_crs' and provider level instead of collection level

* feat(crs): new decorator to support coordinates transformation of feature collections

* feat(crs): 'crs' query parameter for CSVProvider

* test(crs): add tests for 'crs' query parameter

* test: update number of collections in test_describe_collections

* test: update number of collections in test_filter_dict_by_key_value

* fix(crs_transform): change the crs transformation decorator

Change the logic of the decorator so that it works for both functions that
return FeatureCollections and for functions tha return single Features.

* test: add tests for get_collection_item end-point with 'crs' parameter

* fix(test_get_collection_item_crs): id as path parameter, not query parameter

* test: unpack coordinates to create point geometry

* feat(crs): add suuport for crs query parameter for all providers of type 'feature'

* docs(crs): add documentation to illustrate use of 'crs' query parameters

* docs(crs): more data access examples

* fix typo and add new line

* refactor: specify None as default value for crs_transform_out parameter in _sqlalchemy_to_feature method

* changes for PR 1149, test_api and style formatting

* CRS84 as default crs also for test_get_collection_items_crs

* test(crs): test coordinates transformation implementation of PostgreSQLProvider

* test(crs): move tests to test_postgresql_provider

* fix test function calls

* change test to ensure returned features are the same

* add json format to request object

* test(crs): test coordinates transformation implementation of OGRProvider

* refactor(crs): make more compact get_collection_item and get_collection_items

Define two new static methods in API class, to create crs_transform_wkt and
setting content-crs header. These methods can be re-used in both
get_collection_item and get_collection_items methods and removes code
duplication.

---------

Co-authored-by: Just van den Broecke <just@justobjects.nl>

* #1178 fix flake8 error

* #1178 use EPSG:28992 i.s.o. 32631 - fix unit test OGR Shapefile

* #1174 use CRS-compliant Axis ordering for crs support

* #1174 fix and honour CRS 4258disable native CRS Transform in OGR Provider - Axis ordering not honoured...

* #1174 remove ADR tests rom test_util.py

* #1174 enable native CRS transform again in OGR Provider

* #1174 enable native CRS transform again in OGR Provider - fix config

* #1174 remove support for source/target_srs in OGRProvider - enforce transforms always based on storageCRS

* #1174 fix tests Postgresql Provider for Transforms

* #1174 fix tests Postgresql Provider for Transforms

* #1174 add tests for OGR Transformation and Axis Order

* #1174 Suppress potential axis-swapping in OGR ExportToJSON

* #1174 minor fix test - unassign spatialref before setgeom infeat

* #1174 minor fix test - unassign spatialref before setgeom infeat - flake8

* #1174 solve CI WFS test failures with GDAL HTTP config options

* #1174 bbox and bbox-crs defs local in openapi.py for CITE validators

* #1174 merge master - #1152 #1203 etc

* #1174 small doc changes

* #1174 move GeomObject typedef to beginning of util.py

* #1174 added debug logging in transform Decorator func

---------

Co-authored-by: Mathieu Tachon <92298764+MTachon@users.noreply.github.com>
2023-04-11 09:34:48 -04:00
Ricardo Garcia Silva d594653150 Put plugin integration in a more prominent place in docs (#1208) 2023-04-11 08:59:10 -04:00
Tom Kralidis f92ca98765 various provider fixes (#1205)
* improve CRS detection for STAC item

* fix ES id ref
2023-04-06 18:20:59 +02:00
Tom Kralidis 48b5b3b163 add HTTP OPTIONS handling to Django (#1203) 2023-04-04 09:49:22 +02:00
Sander Schaminee 41f3bd801a API design rule support (#1152)
* API design rule support (#1134):

- Change config model
- Change Flask, Starlette, and Django apps for API rule adherence
- Add Flask and Starlette mock clients to test API rule adherence
- Add get_base_url() util function to replace all config['server']['url'] refs
- Ensure that any internal links have URL prefixes if needed
- Add tests and update docs

* Prevent fcntl import error (breaks tests when running locally on Windows)

* Prefer trailing slash in landing page URL when strict_slashes=True (#1134)
2023-04-03 21:30:58 -04:00
Ricardo Garcia Silva c047582155 Generate openapi job endpoint definitions without checking for manager (#1202)
* Generate openapi job endpoint definitions without checking for manager

* Remove has_manager test
2023-04-03 20:30:58 -04:00
Mathieu Tachon 1a1d6362c1 fix(automap): fixes naming conflicts for PostgreSQLProvider (#1184)
* fix(automap): fix naming conflicts for PostgreSQLProvider

Automaping classes and relationships from database schema throws an error if a
column name is the same as a relationship name (see
https://docs-sqlalchemy.readthedocs.io/ko/latest/orm/extensions/automap.html#handling-simple-naming-conflicts).

* renaming function name_for_scalar_relationship -> _name_for_scalar_relationship

* test: test that PostgreSQL provider can handle naming conflicts

Re-use data file from https://github.com/geopython/pygeoapi/pull/1185, and make
to new and empty tables that create naming conflicts when queried.

* Change test function post_collection_items -> get_collection_items

* fix path to table
2023-03-31 13:05:31 +02:00
Tom Kralidis e8acad7120 add support for STAC Azure blob storage backend (#1196) 2023-03-30 16:49:16 -04:00
Benjamin Webb 0f7a8a122c Use sqlalchemy URL function (#1158)
- Use SQL Alchemy URL creation instead of passing a string to create_engine function
- Add Cloud SQL example to docs
2023-03-30 14:52:33 -04:00
Benjamin Webb 9debd89cd8 SensorThings API provider refactor (#1183)
* SensorThingsAPI Provider refactor

- Make code more readable
- Prevent empty SensorThings entity from erroring out pygeoapi

* Fix SensorThings Provider

* Only count on hits

* Fix ref

* Fix ref

- Speed up get_fields
- Simplify intralink process

* Cleanup STA examples

* Add docstrings to functions

* Small fixes

- Split out inner functions
- Add default @iot.id
- Add more debug statements

* Refactor STA paging

- Refactor STA paging
- Reorganize _geometry function
- Update logging
2023-03-30 07:45:12 -04:00
Alexander Pilz d9c377ef62 Feature/mongodb jobmanager (#1186)
* MongoDB Jobmanager

* Add MongoDB-Jobmanager

Example configuration included

* Requested Fixes

Fixes Issues mentioned in latest Review

* Flake8 fixes

Code is fixed on the basis of a flake8 report

* Update ogcapi-processes.rst

* Update ogcapi-processes.rst

* Flake8 fixes

Fixes of flake8 errors
2023-03-30 07:38:43 -04:00
Tom Kralidis 466a1df723 Merge pull request #1190 from tomkralidis/tinydb-updates 2023-03-28 03:41:53 -04:00
Tom Kralidis afc9409491 Merge pull request #4 from rouault/tinydb_fix_bbox_intersects 2023-03-28 03:35:14 -04:00
Even Rouault 13d9cf532f TinyDB provider: fix bbox_intersects() to work with all polygon geometries 2023-03-28 09:30:09 +02:00
Tom Kralidis 21ad9ea8a1 update TinyDBCatalogue based on OARec updates 2023-03-28 03:11:52 -04:00
Even Rouault 4689daab19 TinyDB provider: set [properties][extent][spatial][bbox] from geometry if not provided (#1188) 2023-03-27 12:24:48 -04:00
Even Rouault fcacf69c5b OGC API Features Part 4 fixes: create without id + options (#1187) 2023-03-25 01:32:50 -04:00
Francesco Bartoli 02d6349fd4 Fix returned properties for an item from postgres (#1182)
Fix returned properties for an item from postgres
2023-03-18 16:19:13 +01:00
Tom Kralidis fbf0f4c81a safeguard id detection (#1175) 2023-03-15 21:47:07 +01:00
Sander Schaminee 91051e5df0 Add enclosure link prefetcher and tests (#1169) (#1173) 2023-03-15 09:38:34 -04:00
Aontnio Cereicllo d3590f9047 Upgrading elastisearch.py lib in docker image (#1162)
* Upgrading elastisearch.py lib in docker image

* Unused compatibility mode
2023-03-14 13:18:10 -04:00
Benjamin Webb 4a18393662 Allow STA entity to be URL encoded (#1171) 2023-03-14 12:41:32 -04:00
Benjamin Webb 2f8fff4c8b Use consistent version of leaflet (#1170)
Import consistency for leaflet versions across the templates that use maps

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2023-03-14 11:18:33 -04:00
Benjamin Webb 8eb3e129f3 Fix STA tests (#1168) 2023-03-13 10:00:23 -04:00
Tom Kralidis 4c8d446ae2 default item transactions to GeoJSON payloads (#1166) 2023-03-13 06:59:09 -04:00
Tom Kralidis 2c6de08a44 make locale directory configurable (#1167) 2023-03-13 06:24:32 -04:00
Tom Kralidis a75a606005 safeguard ES aliases (#1164)
* safeguard ES aliases

* add debug
2023-03-06 14:23:31 -05:00
Jo 185255f8e4 - switched coordinate order, to match Leaflet LatLngBounds object (#1163)
https://leafletjs.com/reference.html#latlngbounds
2023-03-06 08:07:19 -05:00
Just van den Broecke 57cfbb5e3f Move PostgreSQL API tests out of test_api.py (#1160)
Moved Postgresql tests from `test_api.py` to `test_postgres_provider.py`.
2023-03-02 17:05:59 +01:00
Benjamin Webb 38299e2dcc Show all columns in /items (#1148)
- Show all columns on /items and overflow table in x direction.
- Render properties missing keys as None in /items HTML table
2023-02-27 18:15:58 -05:00
Tom Kralidis 579251ff5b support GeoJSON id in ES backend (#1159)
* support GeoJSON id in ES backend

* fix
2023-02-27 18:10:50 -05:00
Francesco Frassinelli 0fc4f829f2 Accept additional arguments for get(...) (#1154) 2023-02-26 16:57:09 -05:00
Tom Kralidis af312f115d add server limit to EDR calls that support paging (#1153) 2023-02-21 16:43:32 -05:00
Benjamin Webb 906c0fd619 Fix setting default templates (#1145) 2023-02-17 19:44:09 -05:00
Benjamin Webb 1a9f38347c Add limit query arg to processes (#1147)
* Add limit query arg to processes

* Update per requested changes
2023-02-17 16:41:10 -05:00
Tom Kralidis ddd9c70f93 unpin pygeoif (#1146)
* unpin pygeoif

* Update requirements-docker.txt

* Update requirements-provider.txt
2023-02-16 19:49:48 +02:00
Tom Kralidis 5008a4de01 add radius to flask/django/starlette routes (#1144) 2023-02-16 11:54:52 -05:00
Tom Kralidis fa0351a626 support EDR radius queries (#1143) 2023-02-15 22:18:04 -05:00
Peter Garnæs 9834e4db84 EDR bbox parameter support for cube queries (#1127)
* bbox query parameter support for EDR cube queries

As per Requirement A.61 C, D and E, as well as A.7 in the EDR standard.

* bbox queries support z-axis dimension

* xarray-edr provider cube query support, edr cube unit tests

---------

Co-authored-by: Peter Garnæs <pga@dmi.dk>
2023-02-15 21:26:24 -05:00
Benjamin Webb e1af5e1ca5 Add json-ld templating for feature collection item (#868)
* Render template if specified in config and provide example config

Update linked_data.py

Merge branch 'geopython:master' into jsonld

Rended from json-ld output

Render jinja2 json-ld template from json-ld output instead of json output

Merge branch 'geopython:master' into jsonld

Add documentation

- Add documentation
- Add test to workflow

Update pygeoapi-test-config.yml

Update test_api.py

Update api.py

Update linked_data.py

Move template declaration in configuration

Update docs

Update configuration.rst

Update configuration.rst

* Updates per requested changes

* Fix spelling

* Fix json-ld template pathing

* Remove  root path for the templating

* Move json-ld template from api.py

- Move single item json-ld templating to inside geojson2jsonld
- Reformat json-ld configuration for context and item_template to children of json-ld block
- Update docs and example configurations

* Fix ref

* Use FileSystemLoader to control template search path

search for templates is in order of `template_paths` list

* s/json-ld/linked-data/ig

rename json-ld to more generic name
2023-02-15 21:12:19 -05:00
Mathieu Tachon 25e60c4a8b fix(conformance): updated link to OpenAPI requirements class (#1142) 2023-02-15 20:53:46 -05:00
Benjamin Webb 6844a85220 Use upstream encoding for gzip with CSV formatter (#1140) 2023-02-14 22:31:45 -05:00
Mathieu Tachon 70804d22fa feat(OGRProvider): allow to use 'geom_field' in the config for OGR pr… (#1141)
* feat(OGRProvider): allow to use 'geom_field' in the config for OGR provider

* docs: show an example of using OGR for publishing data from a PostGIS database

* fix: does not check for empty geometry in this PR

* docs: add back ESRIJSON source_type
2023-02-14 19:29:09 -05:00
Mathieu Tachon 74ddb1215d fix(geom2schemageo): use only x,y dimensions when rendering GeoShape (#1133)
* fix(geom2schemageo): use only x,y dimensions when rendering GeoShape

Ignore possible third dimension when unpacking the coordinates of shapely
geometrical object in function 'geom2schemageo'. This fixes the ValueError
thrown when requesting features in JSON-LD format with 3D geometries.

* fixing unpacking of multipoint coordinates
2023-02-13 08:00:56 -05:00
Mathieu Tachon 7f047348b5 fix(OGRProvider_init): make the 'id_field' attribute mandatory in provider config (#1135)
As for the 'layer' attribute, the 'id_field' should be specified in the
configuration file for the OGRProvider. The 'get' method of a OGRProvider
instance (e.g. when requesting a single feature) needs the feature ID when
setting an attribute filter on the layer. If the 'id_field' attribute is
omitted in the configuration file, the error thrown is more explicit.
2023-02-10 15:06:23 -05:00
Benjamin Webb 30f279b7ad Fix CSV provider for a single item (#1138)
* Add larger test data for CSV provider

* Fix CSV provider for single item
2023-02-10 14:48:24 -05:00
Benjamin Webb 983530ac55 Allow Sqlite provider to return None as a geometry if it is missing. (#1136) 2023-02-09 22:11:26 -05:00