Files
pygeoapi/.github/workflows/main.yml
T
Just van den Broecke 7c6993719d add CRS Support for OGC API Feature pygeoapi Provider (#1174)
* OGC API - Features Part 2 (groundwork+CRS-BBOX) from PR #1155 - contributes to issue #1128

* #1128 provide conformance class for OAPIF Part 2 in /conformance page

* #1128 bitten by flake8...

* #1128 configurability CRS Feature Providers with syntax, defaults and tests

* #1128 configurability CRS Feature Providers refine for default values

* #1128 display supported CRSs in HTML Collection template

* #1128 config, mmetadata and tests for storageCRS and storageCrsCoordinateEpoch

* #1128 WIP for bbox-crs parameter support

* #1128 utility function and tests for default/mandatory supprted CRS list

* #1128 default supported CRS adaptation to OAPIF Part 2 standard

* #1128 grr flake8 whitespace

* #1128 start adding full API tests OGR for bbox-crs and crs parms

* #1128 fix flake8

* #1128 fix flake8 - install GDAL in workflow main for OGR tests

* #1128 fix flake8 - install GDAL in workflow main for OGR tests - need pip package?

* #1128 fix flake8 - install GDAL in workflow main for OGR tests - using libgdal-dev gdal-bin

* #1128 fix SensorThings test for main.yml Workflow

* #1128 fix SensorThings test for main.yml Workflow nr 2

* #1128 make all OGR tests working again

* #1128 make all OGR tests working again - flake8

* #1128 make all OGR tests working again - GeoSolutions WFS bbox

* #1128 #1155 add documentation for OGC OAPIF Part 2 CRS CRS BBOX support

* #1128 #1155 refine documentation for OGC OAPIF Part 2 CRS CRS BBOX support

* #1128 #1155 refine documentation to align with #1149

* #1128 #1155 rework from review OAS and pygeoapi config schema

* #1128 #1155 minor: compile Re for CRS URI only once as global var

* #1128 merge in changes from PR #1173 - fix missing import

* WIP Ogcapi features part 2 - Support for crs query parameter (#1149)

* feat(ogcapi_features_crs): start implementing crs support from ogcapi features part2

* Pass input and output CRSs WKT instead of crs transformation object

* fix longs lines and blank lines

* fix typo

* fix import for type annotation not supported by python version

* fix variable visibility in local scope

* fix tabs/spaces indentations

* Add support for the crs parameter to OGRProvider

* make flake8 happy

* Make crs transformation mechanism more consistent between PostgreSQL and OGR providers

* test(util): add two test functions in util.py

New functions: test_get_crs_from_uri and test_get_transform_from_crs

* fix too long lines...

* Update get_crs_from_uri and corresponding test function

* fix(get_crs_from_uri): make the error more explicit in if wrong crs uri format

* flake8 again...

* Keep support for source_srs/target_srs in config for OGRProvider

* revert changes made to pygeoapi-config-0.x.yml, overlap with PR 1155

* test: add test data and update test config file

* Extract 'crs' and 'storage_crs' and provider level instead of collection level

* feat(crs): new decorator to support coordinates transformation of feature collections

* feat(crs): 'crs' query parameter for CSVProvider

* test(crs): add tests for 'crs' query parameter

* test: update number of collections in test_describe_collections

* test: update number of collections in test_filter_dict_by_key_value

* fix(crs_transform): change the crs transformation decorator

Change the logic of the decorator so that it works for both functions that
return FeatureCollections and for functions tha return single Features.

* test: add tests for get_collection_item end-point with 'crs' parameter

* fix(test_get_collection_item_crs): id as path parameter, not query parameter

* test: unpack coordinates to create point geometry

* feat(crs): add suuport for crs query parameter for all providers of type 'feature'

* docs(crs): add documentation to illustrate use of 'crs' query parameters

* docs(crs): more data access examples

* fix typo and add new line

* refactor: specify None as default value for crs_transform_out parameter in _sqlalchemy_to_feature method

* changes for PR 1149, test_api and style formatting

* CRS84 as default crs also for test_get_collection_items_crs

* test(crs): test coordinates transformation implementation of PostgreSQLProvider

* test(crs): move tests to test_postgresql_provider

* fix test function calls

* change test to ensure returned features are the same

* add json format to request object

* test(crs): test coordinates transformation implementation of OGRProvider

* refactor(crs): make more compact get_collection_item and get_collection_items

Define two new static methods in API class, to create crs_transform_wkt and
setting content-crs header. These methods can be re-used in both
get_collection_item and get_collection_items methods and removes code
duplication.

---------

Co-authored-by: Just van den Broecke <just@justobjects.nl>

* #1178 fix flake8 error

* #1178 use EPSG:28992 i.s.o. 32631 - fix unit test OGR Shapefile

* #1174 use CRS-compliant Axis ordering for crs support

* #1174 fix and honour CRS 4258disable native CRS Transform in OGR Provider - Axis ordering not honoured...

* #1174 remove ADR tests rom test_util.py

* #1174 enable native CRS transform again in OGR Provider

* #1174 enable native CRS transform again in OGR Provider - fix config

* #1174 remove support for source/target_srs in OGRProvider - enforce transforms always based on storageCRS

* #1174 fix tests Postgresql Provider for Transforms

* #1174 fix tests Postgresql Provider for Transforms

* #1174 add tests for OGR Transformation and Axis Order

* #1174 Suppress potential axis-swapping in OGR ExportToJSON

* #1174 minor fix test - unassign spatialref before setgeom infeat

* #1174 minor fix test - unassign spatialref before setgeom infeat - flake8

* #1174 solve CI WFS test failures with GDAL HTTP config options

* #1174 bbox and bbox-crs defs local in openapi.py for CITE validators

* #1174 merge master - #1152 #1203 etc

* #1174 small doc changes

* #1174 move GeomObject typedef to beginning of util.py

* #1174 added debug logging in transform Decorator func

---------

Co-authored-by: Mathieu Tachon <92298764+MTachon@users.noreply.github.com>
2023-04-11 09:34:48 -04:00

142 lines
5.0 KiB
YAML

name: Build
on:
push:
paths-ignore:
- '**.md'
pull_request:
branches:
- master
paths-ignore:
- '!**.md'
release:
types:
- released
jobs:
flake8_py3:
runs-on: ubuntu-latest
steps:
- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.7
architecture: x64
- name: Checkout pygeoapi
uses: actions/checkout@master
- name: Install flake8
run: pip install flake8
- name: Run flake8
uses: suo/flake8-github-action@releases/v1
with:
checkName: 'flake8_py3' # NOTE: this needs to be the same as the job name
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
main:
needs: [flake8_py3]
runs-on: ubuntu-20.04
strategy:
matrix:
include:
- python-version: 3.7
env:
PYGEOAPI_CONFIG: "$(pwd)/pygeoapi-config.yml"
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
name: Setup Python ${{ matrix.python-version }}
with:
python-version: ${{ matrix.python-version }}
- name: Configure sysctl limits
run: |
sudo swapoff -a
sudo sysctl -w vm.swappiness=1
sudo sysctl -w fs.file-max=262144
sudo sysctl -w vm.max_map_count=262144
- name: Install and run PostgreSQL/PostGIS 📦
uses: huaxk/postgis-action@v1
with:
postgresql password: ${{ secrets.DatabasePassword || 'postgres' }}
postgresql db: 'test'
- name: Install and run Elasticsearch 📦
uses: getong/elasticsearch-action@v1.2
with:
elasticsearch version: '8.3.1'
host port: 9200
container port: 9200
host node port: 9300
node port: 9300
discovery type: 'single-node'
- name: Install and run MongoDB
uses: supercharge/mongodb-github-action@1.5.0
with:
mongodb-version: 4.4
- name: Install and run SensorThingsAPI
uses: cgs-earth/sensorthings-action@v0.0.2
- name: Install sqlite and gpkg dependencies
uses: awalsh128/cache-apt-pkgs-action@latest
with:
packages: libsqlite3-mod-spatialite
version: 4.3.0a-6build1
- name: Install GDAL with Python bindings
uses: awalsh128/cache-apt-pkgs-action@latest
with:
packages: gdal-bin libgdal-dev
version: 3.0.4
- name: Install requirements 📦
run: |
pip3 install -r requirements.txt
pip3 install -r requirements-starlette.txt
pip3 install -r requirements-dev.txt
pip3 install -r requirements-provider.txt
python3 setup.py install
pip3 install --upgrade numpy elasticsearch
pip3 install --upgrade numpy "sqlalchemy<2"
pip3 install --global-option=build_ext --global-option="-I/usr/include/gdal" GDAL==`gdal-config --version`
#pip3 install --upgrade rasterio==1.1.8
- name: setup test data ⚙️
run: |
python3 tests/load_es_data.py tests/data/ne_110m_populated_places_simple.geojson geonameid
python3 tests/load_es_data.py tests/cite/ogcapi-features/canada-hydat-daily-mean-02HC003.geojson IDENTIFIER
python3 tests/load_mongo_data.py tests/data/ne_110m_populated_places_simple.geojson
gunzip < tests/data/hotosm_bdi_waterways.sql.gz | psql postgresql://postgres:${{ secrets.DatabasePassword || 'postgres' }}@localhost:5432/test
psql postgresql://postgres:${{ secrets.DatabasePassword || 'postgres' }}@localhost:5432/test -f tests/data/dummy_data.sql
- name: run unit tests ⚙️
env:
POSTGRESQL_PASSWORD: ${{ secrets.DatabasePassword || 'postgres' }}
run: |
pytest tests/test_api.py
pytest tests/test_api_ogr_provider.py
pytest tests/test_config.py
pytest tests/test_csv__formatter.py
pytest tests/test_csv__provider.py
pytest tests/test_elasticsearch__provider.py
pytest tests/test_esri_provider.py
pytest tests/test_filesystem_provider.py
pytest tests/test_geojson_provider.py
pytest tests/test_mongo_provider.py
pytest tests/test_ogr_csv_provider.py
pytest tests/test_ogr_esrijson_provider.py
pytest tests/test_ogr_gpkg_provider.py
pytest tests/test_ogr_shapefile_provider.py
pytest tests/test_ogr_sqlite_provider.py
pytest tests/test_ogr_wfs_provider.py
pytest tests/test_openapi.py
pytest tests/test_postgresql_provider.py
pytest tests/test_rasterio_provider.py
pytest tests/test_sensorthings_provider.py
pytest tests/test_socrata_provider.py
pytest tests/test_sqlite_geopackage_provider.py
pytest tests/test_tinydb_catalogue_provider.py
pytest tests/test_util.py
pytest tests/test_xarray_netcdf_provider.py
pytest tests/test_xarray_zarr_provider.py
- name: build docs 🏗️
run: cd docs && make html
- name: failed tests 🚩
if: ${{ failure() }}
run: |
pip3 list -v