Commit Graph

880 Commits

Author SHA1 Message Date
Benjamin Webb fffed87f2d Use Sensorthings action for API tests (#1055) 2022-12-08 16:33:18 -05:00
Tom Kralidis f9fb2c6005 move to f-strings for string interpolation (#1064)
* update from format() to f-strings

* fix

* fix

* fix ref

* update headers
2022-12-08 16:09:19 -05:00
Francesco Frassinelli 64e63ac4c0 Fix typo in plugins.rst example (#1063) 2022-12-07 16:51:44 -05:00
Aontnio Cereicllo 0be1ce54af Updates to Tileset Metadata endpoint (#1051)
* Adding metadata to matrix endpoint

* Adding Tileset metadata with mandatory fields

* Making metadata.json not mandatory, except when it is

* Adding pydantic models

Using models in place of json text

* Adding metadata to matrix endpoint

* Adding Tileset metadata with mandatory fields

* Making metadata.json not mandatory, except when it is

* Adding pydantic models

Using models in place of json text

* Fix tilesets template

Co-authored-by: Francesco Bartoli <francesco.bartoli@geobeyond.it>
2022-12-05 11:33:07 +01:00
Jo 111cfa5a98 - update docs with example of configuration for rendering pg_tileserv vector tiles (#1058) 2022-12-04 07:06:22 -05:00
Jo c2fe864a34 - removing deadcode (#1057) 2022-12-04 06:46:01 -05:00
Tom Kralidis c3b0e7868e minor updates to #1040 (#1054)
* minor updates to #1040
* s/crs/bbox-crs
* write status to stdout if --output-file is used
2022-12-02 09:58:45 +02:00
Tom Kralidis ccb94e132d Update introduction.rst 2022-12-01 09:16:45 -05:00
Jo 85a4984bbe Add Support for a generic url template, for publishing vector tiles as OGC API Tiles (#1050)
* - Added support to read from a generic url template as vector tile backend.

* - Add support for not rendering the tileset metadata

* - added support for z/y/x vector tile layers

* - fixed formatting issues

* - refactored code to use get_layer function, everywhere we need to parse username
- added a couple of debug statements, to make sure we are getting the layer name
correctly
- added an error for url templates that follow a schema, which is not supported yet

* - Added documentation for tiles provider, to show how to read a generic url

* - fixed formatting issues

* - added example of ES vector tiles in the docker-config of elasticsearch

Co-authored-by: doublebyte1 <info@doublebyte.net>
2022-12-01 08:34:16 -05:00
Tom Kralidis 1aa80368f7 fix temporal handling 2022-12-01 14:39:54 +02:00
Tom Kralidis 3ebaaaa27b remove sizing 2022-12-01 14:39:54 +02:00
Tom Kralidis 7fb420c142 add OAMaps django support (#668) 2022-12-01 14:39:54 +02:00
Tom Kralidis 608fba5664 add support for OAMaps 🗺️ (#668) 2022-12-01 11:55:46 +02:00
Tom Kralidis 1aa9630c5f add pygeoif and pygeofilter to debian requirements (#1045) 2022-11-29 11:02:50 -05:00
Tom Kralidis af72c79267 add requirements note for providers (#1040) (#1041) 2022-11-23 08:03:44 -05:00
Tom Kralidis fdf19c5f29 move numpy based tests to xarray/zarr provider (#1042) 2022-11-20 19:31:20 -05:00
Alex ba15a8a4fe Fixes a "TypeError: argument of type 'NoneType' is not iterable" when self.options is None (#1037) 2022-11-20 19:19:33 -05:00
Peter Garnæs 8938733292 Json serialize numpy types (#913)
* Json serialize numpy types

This commit fixes an issue with the current xarray provider. When a
NetCDF file is loaded into an xarray dataset, attributes of data
variables that are numbers will be loaded as numpy.float64 or
numpy.int64. These attributes are passed directly to metadata objects
that are json serialized, creating the issue.

This commit thereby fixes #895

* Avoid numpy import in util module
2022-11-20 19:12:02 -05:00
Tom Kralidis 2c380375ec Update README.md 2022-11-15 09:20:06 -05:00
Angelos Tzotsos 09f2eff771 back to dev 2022-11-15 16:08:53 +02:00
Angelos Tzotsos 5e7015c2c7 update release version 2022-11-15 15:38:29 +02:00
Tom Kralidis 5ed6fb3170 add OGC RI for EDR (#1036) 2022-11-10 14:43:39 -05:00
Tom Kralidis db8fae61eb minor fixes following #964 (#1030) 2022-10-29 02:41:45 -04:00
John A Stevenson 644281359a Add CQL to PostgreSQL provider (via pygeofilter and sqlalchemy) (#964)
* Add pygeofilter, unpin psycopg2

* Add pygeofilter SQLAlchemy query demo

* Add note about primary key

* Add todos to script

* Make script generic

* Add (commented out) PostgreSQL provider hotosm_bdi_waterways to config

* Update to use local test database

* Return result as dictionary

* Pass GEOM_FIELD as variable

* Add offset and limt, fix two names

* Implement order by

* Use function for cql query

* Add cql query block to query and test

* Link up to db engine

* Convert WKB geometry to GeoJSON

* Add shapely as explicit dependency

* Add tests for CQL queries

* Add test using CROSSES with LINESTRING

* Uncomment Docker PostGIS test layer

* First pass at wiring up API

The CQL is read from the request but the parser fails to convert to
AST.

* Turn logging to DEBUG

Example query
http://localhost:5000/collections/hot_osm_waterways/items?cql=%27osm_id%20BETWEEN%2080800000%20AND%2080900000%20AND%20name%20IS%20NULL%27

* Create dedicated pygeoapi-config.yml for testing PostgreSQL

* Add test for PostgreSQL CQL

* Return 400 for bad CQL

* Tidy up old files

* Bring API error type tests together

* Only reflect the table of interest

* Add pygeofilter to requirements

* Remove local setup_env.sh from .gitignore

* Remove lark exception handling

* Remove lark codes from tests, docstring change

* Split cql test into good and error

* Rename cql parameter to filter

* Add optional filter-lang parameter with test

* Initial changes

* Rename cql text parser

* Use CQL-JSON POST for PostgreSQL query

* Refactor to get data early and handle errors

* Add error tests for CQL-JSON PostgreSQL

* Add tests for prev/next and provider instantiation

* As filter-lang is set the block in not needed

* Clarify test fixtures

* Use filterq only in all CQL calls

* Make engine and table_model attributes of provider

* Handle different instantiation failures

* Use SQLAlchemy for get_fields()

* Minor doc edits

* Use SQLAlchemy for get()

* Simplify _sqlalchemy_to_feature

* Store database parameters in dedicated function

* Strip out non-SQLAlchemy code

* Add draft query() based on CQL filters

* Add property filtering to query()

* Implement select_properties and skip_geometry

* Implement bbox filter

* Implement properties subset defined in config

* Clean up materialized view test

We only need to test the fields that are present to be sure that the
view has been accessed, so other checks have been removed.

* Flake8 fixes and tidy up

* Add combined test for CQL with bbox and properties

* Create sessions directly where required

* Add Engine store to allow connection reuse

* Replace cql_ast with filterq

* Tidy up tests by using fixture for api

* Add authors

* Use Session() as context manager

A session requires a connection to the database, as provided by the
SQLAlchemy engine.  When the session goes out of scope, e.g. at exit
from the function when it was used, it should be garbage collected and
the connection freed.  However, during load testing it was found that
this wasn't happening.  This commit uses a context manager pattern to
start the session, which ensures that the session is closed at the end
of the `with` block.

* Return fields type as {'type': ...}

* Cache the table_model with the engine

Generating the table_model by "reflecting" the database is expensive but
the resulting model doesn't change.  If we cache the table_model with
the engine we save on a round-trip to the database and speed up response
time.

* Use separate stores for Engine and table_models

Using a combined store based on both engine and table_model meant that
a pygeoapi serving multiple tables on the same database would maintain
a connection pool for each table.  This could eventually saturate the
number of available connections.  This commit separates the store into
two parts.

* Update documentation with example

* Manage or post items based on POST content type

* ghcr build action

* only ghcr job

* run criteria edit

* Update README.md

* job = main

* remove success requirememt

* add tag

* tags: bgs-cql

* Update main.yml

* Update README.md

* remove binary

* update

* Update actions to latest versions

* Dockerfile refactor

* Dockerfile refactor

* restore dockerhub

* fix pygeoif version

* # 3.6 test & dockerhub ref

* Pin pygeoif version to less than 1

* Replace psycopg-binary with psycopg

* Split Docker build into sections

* Pin click version >7,<=8

* Add further cql examples

* Update workflows to match GeoPython repo

* update flake python version

* Unquote PostgreSQL column names

* Update container workflow uses versions

* flake8 use python version 3.7

* Use pre ping to get db connection

* Test that properties are set in provider

* Amendments as requested by @tomkralidis

* Correction

* Fix reviewed changes in api code

Co-authored-by: Colin Blackburn <colb@bgs.ac.uk>
Co-authored-by: KoalaGeo <eddlewis85@gmail.com>
Co-authored-by: Edd <edlew@bgs.ac.uk>
Co-authored-by: Colin Blackburn <ximenesuk@users.noreply.github.com>
2022-10-29 02:15:31 -04:00
KoalaGeo 44412e1adc flake use python version 3.7 (#1028)
3.7.14 no longer works
2022-10-25 08:39:41 -04:00
vvmruder dd379d20cc Update administration.rst (#1026)
Fix typo in doc
2022-10-25 08:28:04 -04:00
Tom Kralidis 20c1f32966 Add marker clustering for point collections (#1025) 2022-10-19 16:07:16 -04:00
Tom Kralidis 6447493a76 add support for provider plugin schemas (#1022) 2022-10-19 15:43:57 -04:00
KoalaGeo ad3e224f62 Actions pr (#1021)
* Removed branches requirement on build action

Build action will now run whenever Lint job is completed: All pushes, PR to Master branch & releases

* Put lint in same main.yml

Lint is still a requirement for the build job

* Docker build only runs on master branch
2022-10-17 16:19:32 -04:00
KoalaGeo 49b0ff31bb Actions simplification (#1020)
* Removed branches requirement on build action

Build action will now run whenever Lint job is completed: All pushes, PR to Master branch & releases

* Put lint in same main.yml

Lint is still a requirement for the build job
2022-10-17 15:59:51 -04:00
KoalaGeo 6a92b4cd8f Removed branches requirement on build action (#1018)
Build action will now run whenever Lint job is completed: All pushes, PR to Master branch & releases
2022-10-17 14:50:38 -04:00
Jo fd5ea069ea - upgrading the docker example to use ES=8, running in compatibility mode (so we don't break the python client) (#985) 2022-10-16 16:04:11 -04:00
Tom Kralidis 2e686267ac Merge pull request #1016 from tomkralidis/safeguard-bbox-crs
safeguard bbox-crs
2022-10-14 23:33:18 -04:00
Tom Kralidis ea93b45d3b safeguard bbox-crs 2022-10-14 23:30:55 -04:00
Tom Kralidis 2c3278fdc4 Merge pull request #1012 from federal-geospatial-platform/rasterio-default-crs 2022-10-14 22:00:11 -04:00
Alex 1c030ebd78 Merge pull request #1 from tomkralidis/fgp-update
update refs, add doc
2022-10-14 21:58:23 -04:00
Tom Kralidis 66f2d86e83 update refs, add doc 2022-10-14 19:02:16 -04:00
Alex bef9625e39 Support accents in yaml config file (#1009) 2022-10-14 13:51:12 -04:00
Tom Kralidis 3016cb73c6 do not set ES dates as raw (#1015)
* do not set ES dates as raw

* safeguard
2022-10-14 12:54:25 -04:00
Just van den Broecke 93bbc12aac fixes issue #1013 - new URL Dutch RDInfo WFS (#1014) 2022-10-13 18:33:45 -04:00
Alex-NRCan 4b8e3f57b1 Small fix to pass the git Lint / flake8_py3 cron job 2022-10-12 20:15:30 -04:00
Alex-NRCan 88a0028e0c Rasterio now supporting more spatial references than the hardcoded 4326 and exposing bbox-crs parameter in the openapi 2022-10-11 15:34:26 -04:00
Paul van Genuchten c0d725e5df adds a configuration for a skinned pygeoapi (#1006)
* adds a configuration for a skinned pygeoapi

* Update and rename readme.md to README.md

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
2022-10-09 07:53:28 -04:00
Tom Kralidis bac7eee771 Merge pull request #1005 from tomkralidis/updates-2022-09-30
updates 2022 09 30
2022-09-30 09:26:50 -04:00
Tom Kralidis e191bf4768 detect whether an HTTP POST to .../items is a transaction or CQL 2022-09-30 09:22:59 -04:00
Tom Kralidis d08b8e6ab2 fix media type for queryables responses 2022-09-30 09:09:28 -04:00
Tom Kralidis 87cfa59e31 add pygeometa plugin to docs 2022-09-30 09:08:52 -04:00
KoalaGeo 852f7ed0cc Remove **.py requirement (#1002)
* Remove **.py requirement

* use paths-ignore

* change python version to 3.7.14

* Rename pytorch > pygeoapi
2022-09-29 22:46:16 -04:00
Just van den Broecke a7d763bf12 Update badges (#1001)
* Update badges

* add Lint badge (see #1000)
* display Workflow names as in Workflow .yml files

* Change Lint Badge order
2022-09-28 07:55:05 -04:00
KoalaGeo d6b87e8536 Move flake8 test in new Lint GH Workflow thus before Provider tests (#1000)
* Move flake before provider tests

* Move flake8 to separate job

* Corrected run logic
2022-09-28 12:51:10 +02:00