Commit Graph

1126 Commits

Author SHA1 Message Date
Aontnio Cereicllo c6d506aa0b Get tiles metadata also from URL (#1526) 2024-02-01 10:15:02 +01:00
Ricardo Garcia Silva 2d0fc5df3e Convert coordinates to storage crs when filtering via cql (#1489)
* Allow either URL or URN CRS URIs

* Implemented transformation of geometries in CQL filter

* Fixed flake8 issue

* Removed commented out code

* Implemented support for the filter-crs query parameter

* Remove unneeded print() calls
2024-01-30 20:16:12 -05:00
Jo de787b0f22 Return a 400 status code for an "out-of-range" get tile request (#1503)
* - return a 400 error, when the MVT_elastic provider receives a request for a tile out of range

* -Catching errors on get_tile requests for mvt-elastic provider:
 - if the status code is bellow 500, it throws a ProviderInvalidQueryError (status code 404)
 - otherwise it throws a server error (status code 500)
- the api catching of errors is left as it was before, as the appropriated status code is being sent by the provider.

* - updated condition on tiles request (mvt-provider), to exclude responses with a status of 500 from throwing a client error
2024-01-30 20:11:13 -05:00
Jo 18eeaf79e4 Addressing Requirements class 2 (Requirements Class Tileset) of the OGC API Tiles Standard (#1497)
* - refactored mvt classes to support all implemented metadata formats, regardless of the provider

* - fixed formatting issues

* Implementing basic tile metadata methods

* Fixing yml models

* Adding additional format

* Fixing schema set on load

* Removing unused field from documentation

* - added support to TileMatrixSets endpoint
- added TileMatrixSet definitions for WorldCRS84Quad and WebMercatorQuad

* - added tiling-schemes link in the json representation of the landing page

* - added html pages for tilematrixset endpoints

* - advertise json and html representations of the tiling schemes in the landing page

* - Use api definition of Well-known TileMatrixSets in the tiling-schemes element of the tiles metadata page

* - added tiling-scheme url on tiles metadata page, for es and tippecanoe providers

* - fixed flak8 formatting errors

* - updated number of links  on the landing page, on the api test

* - Manage tile matrix set id dinamically, on tilematrix set flask endpoint

* - renamed functions to lower case

* - renamed tilematrix set functions on flask

* - Use TtileMatrixSetId parameter in tilematrixset api function

* - added support to TileMatrixSet endpoints on starlette

* - added test for tileMatrixSets api endpoint

* - added test for the tilematrixset endpoint

* - added routes for django

---------

Co-authored-by: Antonio Cerciello <antonio.cerciello@gmail.com>
2024-01-30 20:10:43 -05:00
Bernhard Mallinger 278004d703 Pass OPENAPI_DOCUMENT to API also in django mode (#1510)
* Pass OPENAPI_DOCUMENT to API also in django mode

Fixes #1509

* Add basic test for django

This only verifies that one page loads, which however
covers most of the django setup.

* Install django requirements in CI
2024-01-30 14:57:45 -05:00
Ricardo Garcia Silva 8e7f8fc26d Added pre-commit config file (#1508)
* Added .pre-commit-config.yaml file

* Initial pass of pre-commit with its default rules and flake8

* Revert "Initial pass of pre-commit with its default rules and flake8"

This reverts commit 86d39e964e8122580d675f4639ae2d15b85c02e2.

* Made pre-commit only act on code files

* Added sql to the pre-commit file pattern

* Initial pass of pre-commit
2024-01-30 12:37:16 -05:00
Jo c5353d889a Give support to serving mvt-tippecanoe tiles from an url (#1507)
* - give support to serving mvt-tippecanoe tiles from an url

* - replaced use of os.path by Path
2024-01-30 17:40:47 +01:00
Jo 3f991a225e Merge pull request #1511 from doublebyte1/conformance
Update conformance declaration for OGC API - Tiles
2024-01-30 09:03:17 +00:00
Jeff McKenna ac95a88511 expand TinyDB path message, for ProviderConnectionError (#1518)
* expand TinyDB msg

* expand TinyDB msg

* fix long line

* fix long line
2024-01-29 11:27:40 -05:00
Ricardo Garcia Silva 8ffc354adf Fixed warnings related to migration to sqlalchemy2 in tests (#1520) 2024-01-29 08:25:01 -05:00
doublebyte 04bbecbd02 - update conformance declaration for OGC API - Tiles 2024-01-22 15:12:16 +00:00
Jo e1fe05c1c1 - return 200 on the response headers of a successfull get tile request (#1500) 2024-01-17 14:05:55 +01:00
Aontnio Cereicllo 09cb2c07bd Tiles Metadata provider refactor (#1482)
* - refactored mvt classes to support all implemented metadata formats, regardless of the provider

* - fixed formatting issues

* Implementing basic tile metadata methods

* Fixing yml models

* Adding additional format

* Fixing schema set on load

* Removing unused field from documentation

* Change method name to generic vendor

* Keeping extra metadata info for tippecanoe provider

* Fix flake validations error

---------

Co-authored-by: doublebyte <doublebyte@hushmail.com>
2024-01-15 20:21:07 +01:00
Tom Kralidis c36f8ad9ba Update dummy.py 2024-01-15 13:56:13 -05:00
Matt Walker ae94e17952 Log exception detail when debugging a process (#1496)
Swap LOGGER.error for LOGGER.exception so that details of an exception that occurs during process execution are logged including the traceback to make debugging easier.
2024-01-15 13:54:22 -05:00
Bernhard Mallinger 1da681bf1b Generalize provider error handling to processes (#1495)
This allows triggering errors from a processor with specific
http status code, ogc exception code and a custom message.

This is very useful the processor realizes that the input parameters
don't make sense or are not allowed, in which case it can supply a
descriptive error message and an http status code different from 500.
2024-01-15 13:54:00 -05:00
Bernhard Mallinger af5dbb8350 Add tests for get_job_result (#1494)
This api method did not have any tests until now.

This also contains a small fix: If no content type was requested,
then by default, the header specified the content type
`application/json`, whereas the content was actually html. Note that
this is rarely a problem because browsers do request html explicitly.
2024-01-15 13:53:06 -05:00
Ricardo Garcia Silva f0638f445f Removed the dependency on nest-asyncio (#1493) 2024-01-15 13:52:37 -05:00
Ricardo Garcia Silva 62b8d67018 fixed failing starlette tests (#1491) 2024-01-15 01:07:26 +01:00
Tom Kralidis 635562f966 import Admin module based on runtime configuration (#1469) (#1485) 2024-01-09 17:32:47 -05:00
Sarah Jordan fe7ea108a7 Data return for STAC collections (#1483)
* render data viewing page for collections #1409

render data viewing page for STAC collections if there is no STAC item below.

* flake8

* flake8

* add additional check for collections

* render data viewing page for collections #1409

render data viewing page for STAC collections if there is no STAC item below.

* flake8

* flake8

* add additional check for collections
2024-01-09 16:14:02 -05:00
Francesco Bartoli e9bb5dcc95 Check docker image CVEs in vulnerabilities workflow (#1480)
* Add step for checking docker image CVEs

* Add buildx command and path to Dockerfile

* Fix working directory

Fix working directory

Fix working directory

Fix working directory

* Use trivy action

Use trivy action

* Use github placeholder over custom variable
2024-01-06 19:12:33 -05:00
Francesco Bartoli 3c5a4cda9c Add a step in CI for CVEs scanning (#1478)
* Add a step in CI for CVEs scanning

* Move the check for vulnerabilities on its own workflow

* Clean the clone job and comment accepted risk

* Add a badge for vulnerabilities clearance in the readme
2024-01-06 11:37:37 -05:00
Benjamin Webb 0a47ad28c9 Update starlette documentation (#1461)
* Add starlette / docker

* Update Starlette documentation

* Remove starlette docker example

* Revert "Add starlette / docker"

This reverts commit 6cb6d6f10a789c9c6a0adf5e925857be5183009d.

* Partially revert "Update starlette documentation"
2024-01-05 19:32:09 -05:00
Ricardo Garcia Silva 22ac69aa75 Added method for translating between generic geometry name and the name of the actual geometry column in the postgres DB (#1453) 2024-01-05 16:36:51 +01:00
Tom Kralidis cfa21f627c Django: remove unused code (#1476) 2024-01-05 06:55:26 -05:00
Bernhard Mallinger 80201db7ee Logfile rotation and configurable log format (#1438)
* #1389 added 1st draft

* fixed flake8 issues

* #1388 added formatable  date and time

* 1st draf of docu

* #1389 added documentation for logfile rotation

* #1389 adjusted documentation for rotation

* Fix import order

* Add advanced logging config to schema

* Remove .vscode folder from .gitignore

* Move references to bottom of file in docs

* Use single quotes

* Restore original .gitignore

* Update configuration.rst

* Update log.py

* Update pygeoapi-config-0.x.yml

---------

Co-authored-by: L K <lucka.kadlecova@gmail.com>
Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2024-01-04 10:30:09 -05:00
Bernhard Mallinger 68689921a9 Dockerfile improvements (#1252) 2024-01-04 07:22:44 -05:00
Jo 68199c88da Updated well-known tileMatrixSetURIs to normative references on the OGC definition server (#1471)
* - Updated well-known tileMatrixSetURIs to normative references on the OGC definition server.

* - removed extra line
2024-01-03 17:46:00 -05:00
Tom Kralidis d66e790df1 Update admin-api.rst 2024-01-03 10:48:48 -05:00
Benjamin Webb 8e122d1a61 Add Admin API (#1137)
* Add Admin API

- Create `admin.py` to serve as Admin API Core
- Create `flask_admin.py` to create flask blueprint for admin API
- Consolidate configuration getter
- Add Pathlib serializing
- Add docker example

* Add integration tests

- Amend admin example to allow writing to configuration. If FS is read only admin API does not work. Returns a 500 and logs `OSError: [Errno 30] Read-only file system: '/pygeoapi/local.config.yml' `

* Preserve env variables in configuration

* Use common accessor functions

- Use common configuration accessor methods for Django and Starlette

* GET returns raw config file

Return configuration with environment variables preserved on GET requests

* Safeguard env variables for root cfg view

"bind": {
"host": "localhost",
"port": "6000"
}
->
"bind": {
"host": "${HOST}",
"port": "${PORT}"
}

* Simplify admin HTML imports

- Use jinja recursion to expand the configuration
- Remove vue from templates

* Create admin API documentation

* Use render_item_value in admin template

* Add Admin API

- Create `admin.py` to serve as Admin API Core
- Create `flask_admin.py` to create flask blueprint for admin API
- Consolidate configuration getter
- Add Pathlib serializing
- Add docker example

* Update GitHub Actions deployment

* Update admin entrypoint

Update admin entrypoint to align with upstream pygeoapi implementation

* Make requested changes

Co-Authored-By: Tom Kralidis <tomkralidis@gmail.com>

* Amend test url

Co-Authored-By: Tom Kralidis <tomkralidis@gmail.com>

* Fix Admin CI tests

* Add PUT and PATCH for root configuration

- Add put and patch for root configuration
- Add CI tests for PUT and PATCH of root

* Update OpenAPI document wording

* Update entrypoint.sh

Replace tabs with spaces

* Remove unused step

Error from rebasing. Admin API tests are moved to their own job.

* Use jsonpatch

- Use debian supported packaging
- Use custom merge function

* Move test data location

* Create Starlette and Django app

- Fold flask_admin.py into flask_app.py

Co-Authored-By: Tom Kralidis <tomkralidis@gmail.com>

* Make requirements-admin.txt

Move admin dependencies to requirements-admin.txt

* Delete guiblock.html

* Update test count for STAC

Update expected test count for addt'l admin test data

* Relegate config warning to config.py

* Move admin tests out of example

* Delete admin docker example

* Update admin-api.rst

* Update pygeoapi-config-0.x.yml

* Update configuration.rst

* Update config.py

* Update admin.py

* Update admin.py

---------

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2024-01-03 10:45:07 -05:00
Tom Kralidis 649a02ee48 update PyPI development status classifier (#1468) 2024-01-03 09:23:56 -05:00
Bernhard Mallinger 5b7d0604b4 Simplify provider error handling (#1392)
* Use generic error handling in get_collection_item

* Use generic error handling in get_collection_items

* Allow specifying a user-facing message in ProviderGenericError

* ose generic error handling in more instances

* Use generic error handling in more instances

* Linting fixes
2024-01-03 08:39:01 -05:00
Tom Kralidis 05965a2982 Happy New Year! 🎉 2024-01-01 08:34:50 -05:00
Tom Kralidis 2a5bdae3a3 move all examples into separate GitHub repository (#1456) (#1464) 2024-01-01 08:16:15 -05:00
Francesco Bartoli 7ad8e3651c Add description to options method (#1466) 2024-01-01 08:15:25 -05:00
Tom Kralidis 3ab73cb19b update PR template (#1463)
* Update PULL_REQUEST_TEMPLATE.md

* add RFC2 to checklist
2024-01-01 08:15:06 -05:00
Jo fc31cb3e73 MVT Provider refactoring (#1454)
* - Refactor MVTProvider, to support subclassing
- Added provider for MVT Tippecanoe

* - fixed test configuration for the MVT tippecanoe provider

* - added MVT elastic provider

* - added documentation for mvt providers
- fixed formatting

* - removed tiles publishing from elastic provider example
- updated gitignore to ignore data folder from elastic example

* - added docker example for MVT-elastic backend provider

* - Added docker example for tippecanoe MVT provider

* - updated README of the docker examples section, to catch up with the provided examples.

* - renamed mvt-elastic and mvt-tippecanoe docker example folders.

* - renamed elastic and tippecanoe folders to lower case

* - fixed formatting issue

* - s/Mabox/Mapbox/

* - put plugin provider names in single line

* - renamed MVT provider base class to BaseMVTProvider.

* - added MVT base class

* - reviwed uppercase on .gitignore
- add folder to .gitignore

* - removed new line and brackets

* - removed redundant line with base class

---------

Co-authored-by: doublebyte1 <info@doublebyte.net>
2023-12-28 22:23:54 +01:00
Ricardo Garcia Silva b78d28bcfd Modified starlette test client initialization to not receive a base URL (#1447) 2023-12-20 11:07:28 -05:00
Tom Kralidis 07b40a4305 add notes on using RFC1738 URLs for ES connections (#1413) (#1441)
* add notes on using RFC1738 URLs for ES connections (#1413)

* add CQL info to record providers
2023-12-19 06:42:53 -05:00
Jo cc20d1e62a Merge pull request #1431 from geopython/edr-collections
provide EDR metadata and links also for collections endpoint
2023-12-18 09:20:53 +00:00
Tom Kralidis be940be5d1 fix EDR parameter names (#1433) 2023-12-16 08:17:58 -05:00
Tom Kralidis 0a24d58a27 fix 2023-12-16 08:17:58 -05:00
Tom Kralidis 7866529301 provide EDR metadata and links also for collections endpoint 2023-12-16 08:17:58 -05:00
xkosubek f5b6505b67 Fixes and adjustments for the Oracle provider (#1410)
* Added support for table synonyms

* Added new parameters to query and manipulator call

* Changed error types

* Mount volumes to oracle container

* workflow part 2

* workflow part 3

* workflow part 4

* Changed file permissions to 777

* Deleted folder

* Recreated folder

* Changed to official Oracle Docker-Image

* Added Chown user

* back to gvenzl/oracle-xe:latest

* Tried docker-entrypoint-startdb.d

* Added addnab/docker-run-action@v3

* Added port and deamon mode

* next try

* added job.container.network

* next try

* + docker ps

* next try

* using docker run

* next try

* next try

* Changed len of array to 11

* Use sdo_util.from_geojsonfor create and update

* Flake8 changes

* Fixed error with views

* Added crs_transform_spec support

* Without default_crs

* Updated documentation for Oracle Provider

* changes for flake8

* Added crs_transform_spec support to get function

* review changes

* Added configurable SDO operator
2023-12-16 07:21:25 -05:00
Bernhard Mallinger d25d0cfa65 Use same format handling in collection coverage as elsewhere (#1412)
This now fixes the issue that the `Accept` header wasn't considered for
format selection.

This adds netcdf as a known mime type in order to be able to test this,
but also for actual clients to use.

Note that if we want to allow other mime types such as `application/x-grib2`,
we also have to add them to `FORMAT_TYPES`.

Fixes #1390
2023-12-16 07:20:09 -05:00
Ricardo Garcia Silva e11cac67ac Upgrade ci to use ubuntu 22 04 (#1427)
* Refactored get_collection_coverage tests

Using pytest's parametrize mark. This makes it easier to isolate test cases in order to fix the errors that are showing up when running under Python3.8

* Add Python 3.8 to the CI matrix

* Reverted tests back to non-parametrized form

* Dropped Python 3.7 from main github CI workflow

* replaced ubuntu 20.04 with ubuntu 22.04 in CI workflow files

---------

Co-authored-by: Ricardo Garcia Silva <ricardo@kartoza.com>
2023-12-16 07:19:29 -05:00
Ricardo Garcia Silva 2de1b895e4 Fixed openapi fixture usage in tests (#1429) 2023-12-16 07:18:11 -05:00
Bernhard Mallinger 1fc18653e8 Handle missing 'example' in describe_processes (#1430)
Fixes #1424
2023-12-16 07:17:44 -05:00
Sarah Jordan c2b06c9052 fix coords for non-cube query (#1432) 2023-12-16 04:23:46 -05:00