Add cql-json support for ES (#723)

Fix starlette event loop


Fix starlette event loop


Fix starlette event loop


Fix starlette event loop


Fix provider regression


Make method public


Make method public


Move function to the helpers utility


Add the CQL lifecycle for development


Add CQL docs


Fix flake8


Isolate import for starlette codepath

docs/source/cql.rst

@@ -0,0 +1,50 @@
+.. _cql:
+
+CQL support
+===========
+
+Limitations
+-----------
+
+The support to CQL is limited to `Simple CQL filter <https://portal.ogc.org/files/96288#cql-core>`_ and thus it allows to query with the
+following predicates:
+
+- comparison predicates
+- spatial predicates
+- temporal predicates
+
+Formats
+-------
+
+At the moment pygeoapi supports only the CQL dialect with the JSON encoding `CQL-JSON <https://portal.ogc.org/files/96288#simple-cql-JSON>`_.
+
+Providers
+---------
+
+As of now the available providers supported for CQL filtering are limited to only :ref:`Elasticsearch <Elasticsearch>`.
+
+Queries
+^^^^^^^
+
+The following type of queries are supported right now:
+
+- ``between`` predicate query
+- Logical ``and`` query with ``between`` and ``eq`` expression
+- Spatial query with ``bbox``
+
+Examples
+^^^^^^^^
+
+A ``between`` example for a specific property through an HTTP POST request:
+
+.. code-block:: bash
+
+  curl --location --request POST 'http://localhost:5000/collections/nhsl_hazard_threat_all_indicators_s_bc/items?f=json&limit=50&filter-lang=cql-json' \
+  --header 'Content-Type: application/query-cql-json' \
+  --data-raw '{
+    "between": {
+      "value": { "property": "properties.MHn_Intensity" },
+      "lower": 0.59,
+      "upper": 0.60
+    }
+  }'

docs/source/data-publishing/ogcapi-features.rst

@@ -15,17 +15,17 @@ pygeoapi core feature providers are listed below, along with a matrix of support
 parameters.
 
 .. csv-table::
-   :header: Provider, properties (filters), resulttype, bbox, datetime, sortby, properties (display)
+   :header: Provider, properties (filters), resulttype, bbox, datetime, sortby, properties (display), CQL
    :align: left
 
-   CSV,❌,results/hits,❌,❌,❌,✅
-   Elasticsearch,✅,results/hits,✅,✅,✅,✅
-   GeoJSON,❌,results/hits,❌,❌,❌,❌
-   MongoDB,✅,results,✅,✅,✅,❌
-   OGR,✅,results/hits,✅,❌,❌,❌
-   PostgreSQL,✅,results/hits,✅,❌,❌,❌
-   SQLiteGPKG,✅,results/hits,✅,❌,❌,❌
-   SensorThingsAPI,✅,results/hits,✅,✅,✅,✅
+   CSV,❌,results/hits,❌,❌,❌,✅,❌
+   Elasticsearch,✅,results/hits,✅,✅,✅,✅,✅
+   GeoJSON,❌,results/hits,❌,❌,❌,❌,❌
+   MongoDB,✅,results,✅,✅,✅,❌,❌
+   OGR,✅,results/hits,✅,❌,❌,❌,❌
+   PostgreSQL,✅,results/hits,✅,❌,❌,❌,❌
+   SQLiteGPKG,✅,results/hits,✅,❌,❌,❌,❌
+   SensorThingsAPI,✅,results/hits,✅,✅,✅,✅,❌
 
 
 Below are specific connection examples based on supported providers.
@@ -65,6 +65,7 @@ To publish a GeoJSON file, the file must be a valid GeoJSON FeatureCollection.
          data: tests/data/file.json
          id_field: id
 
+.. _Elasticsearch:
 
 Elasticsearch
 ^^^^^^^^^^^^^
@@ -87,6 +88,11 @@ To publish an Elasticsearch index, the following are required in your index:
          id_field: geonameid
          time_field: datetimefield
 
+This provider has the support for the CQL queries as indicated in the table above.
+
+.. seealso::
+  :ref:`cql` for more details on how to use the Common Query Language to filter the collection with specific queries.
+
 OGR
 ^^^
 

docs/source/development.rst

@@ -23,6 +23,39 @@ To run all tests, simply run ``pytest`` in the repository.  To run a specific te
 run ``pytest tests/test_api.py``, for example.
 
 
+CQL extension lifecycle
+-----------------------
+
+Limitations
+^^^^^^^^^^^
+
+This workflow is valid only for the `CQL-JSON` format.
+
+Schema
+^^^^^^
+
+The Common Query Language (CQL) is the part 3 of the standard OGC API - Features. This extension has its specification available at 
+`OGC API - Features - Part 3: Filtering and the Common Query Language (CQL) <https://portal.ogc.org/files/96288>`_ and the schema exists in development at
+`cql.json <https://github.com/opengeospatial/ogcapi-features/blob/master/extensions/cql/standard/schema/cql.json>`_.
+
+Model generation
+^^^^^^^^^^^^^^^^
+
+pygeoapi uses a class-based python model interface to translate the schema into python objects defined by `pydantic <https://pydantic-docs.helpmanual.io/>`_ models.
+The model is generated with the pre-processing of the schema through the utility ``datamodel-codegen``:
+
+.. code-block:: bash
+
+   # Generate from local downloaded json schema file
+   datamodel-codegen  --input ~/Download/cql-schema.json --input-file-type jsonschema --output ./pygeoapi/models/cql_update.py --class-name CQLModel
+
+How to merge
+^^^^^^^^^^^^
+
+Once the new pydantic models have been generated then the content of the python file ``cql_update.py`` can be used to replace the old classes within the ``cql.py`` file.
+Update everything above the function ``get_next_node`` and then verify if the tests for the CQL are still passing, for example ``test_post_cql_json_between_query`` 
+in ``tests/test_elasticsearch__provider.py``.
+
 Working with Spatialite on OSX
 ------------------------------
 

docs/source/index.rst

@@ -29,6 +29,7 @@ pygeoapi |release| documentation
    data-publishing/index
    plugins
    html-templating
+   cql
    language
    development
    ogc-compliance