huanld/pygeoapi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Multilingual support (alternative) (#664)

Browse Source 
* Created localization (l10n) module + tests. Added l10n support to API and plugins (wip).

* Big refactor:

* All routed API methods are now decorated by @pre_process (consistency) and no longer have a headers+format argument but a request argument (**kwargs also removed)
* The pre_process decorator turns an incoming Flask/Starlette request into a generic APIRequest instance
* The new APIRequest class extracts all relevant info (params, data, locale, etc.) from the request and exposes them as properties
* Removed a lot of boilerplate (i.e. format checking) and wrapped that into methods
* Updated server-specific API calls in each route method (pass entire request object, not headers and query params)

* Several improvements and fixes:

* Updated OpenAPI page with "l" query param
* Added example translations (metadata)
* Changed plugin signature: added explicit locale attribute (instead of **kwargs)
* Moved locale processing to get_plugin_locale() function in l10n module
* API should pass raw requested locale to plugins, locale should always be set
* Fixed API tests and added APIRequest tests
* Prepared utils.py for Jinja2 i18n extension
* Rebased on commit b40297a8 and fixed compatibility with #661 and #662

* Updated documentation for language support

* Rebased and fixed compatibility with PR #658:

* Fixed EDR provider signature (added locale)
* Fixed EDR API routes and query function (and improved parameter-name handling)
* Fixed EDR tests

* Translate entire config in render_j2_template for requested locale:

* Added new translate_dict function to l10n module (+ tests)
* Updated all render_j2_template calls with locale parameter
* Updated pygeoapi-test-config.yml with some language structs

* Minor improvements

* support both 'language' and 'languages' property in server config and provider definitions
* renamed and modified translate_dict() to more generic translate_struct() function (l10n module)
* remove Content-Language header from provider responses if provider has no language support and format is json(ld)
* updated tests

* Leave provider locale handling to API

* Moved code to determine locale from providers to API class (and remove for formatters and processes)
* Removed locale parameter from plugin __init__ signatures
* Removed locale parameter from load_plugin()
* Added **kwargs to provider implementations for get, query, get_metadata, get_coverage_domainset and get_coverage_rangetype method signatures
* Added language=<locale> to all API calls to provider get, query, get_metadata, get_coverage_domainset and get_coverage_rangetype methods

* Use 'lang' instead of 'l' as language query parameter

* Updated Open API
* Updated documentation
* Fixed tests

* Implemented requested PR changes:

* Added usage examples to the APIRequest docstring
* Removed language support from coverage functions
* Updated plugins.rst and language.rst to match new behavior
* Removed language struct from resource links in pygeoapi-config.yml
* Rebased on latest master (fixed test_api.py)

* Rebased and applied fixes:

* Data property in APIRequest now is an awaitable attribute (fixed for Starlette compatibility)
* Named references to 'l' parameter to 'lang'

* Final changes/improvements:

* Make sure that Content-Language is always set;
* Added more tests to ensure that the default language returned is the first configured language (if no language was requested by the user);
* Updated docs;
* Replaced re-occuring strings with constants in api.py;
* Fixed Flake8 checks.

* add missing async to starlette routes (#704)

Co-authored-by: Tom Kralidis <tomkralidis@gmail.com>
This commit is contained in:
Sander Schaminee
2021-06-09 00:46:35 +02:00
committed by GitHub
parent 4bc929ef87
commit 023f24d26b
27 changed files with 2925 additions and 1643 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/index.rst
+1
View File
@@ -29,6 +29,7 @@ pygeoapi |release| documentation
data-publishing/index
plugins
html-templating
language
development
ogc-compliance
contributing
docs/source/language.rst
+233
View File
@@ -0,0 +1,233 @@
.. _language:
Multilingual support
====================
pygeoapi is language-aware and can handle multiple languages if these have been defined in pygeoapi's configuration (see `maintainer guide`_).
Providers can also handle multiple languages if configured. These may even be different from the languages that pygeoapi
supports. Out-of-the-box, pygeoapi "speaks" English. System messages and exceptions are always English only.
The following sections provide more information how to use and set up languages in pygeoapi.
End user guide
--------------
There are 2 ways to affect the language of the results returned by pygeoapi, both for the HTML and JSON(-LD) formats:
1. After the requested pygeoapi URL, append a ``lang=<code>`` query parameter, where ``<code>`` should be replaced by a well-known language code.
This can be an ISO 639-1 code (e.g. `de` for German), optionally accompanied by an ISO 3166-1 alpha-2 country code (e.g. `de-CH` for Swiss-German).
Please refer to this `W3C article <https://www.w3.org/International/articles/language-tags/>`_ for more information or
this `list of language codes <http://www.lingoes.net/en/translator/langcode.htm>`_ for more examples.
Another option is to send a complex definition with quality weights (e.g. `de-CH, de;q=0.9, en;q=0.8, fr;q=0.7, \*;q=0.5`).
pygeoapi will then figure out the best match for the requested language.
For example, to view the pygeoapi landing page in Canadian-French, you could use this URL:
https://demo.pygeoapi.io/master?lang=fr-CA
2. Alternatively, you can set an ``Accept-Language`` HTTP header for the requested pygeoapi URL. Language tags that are valid for
the ``lang`` query parameter are also valid for this header value.
Please note that if your client application (e.g. browser) is configured for a certain language, it will likely set this
header by default, so the returned response should be translated to the language of your client app. If you don't want this,
you can either change the language of your client app or append the ``lang`` parameter to the URL, which will override
any language defined in the ``Accept-Language`` header.
Notes
^^^^^
- If pygeoapi cannot find a good match to the requested language, the response is returned in the default language (US English mostly).
The default language is the *first* language defined in pygeoapi's server configuration YAML (see `maintainer guide`_).
- Even if pygeoapi *itself* supports the requested language, provider plugins may not support that particular language or perhaps don't even
support any language at all. In that case the provider will reply in its own "unknown" language, which may not be the same language
as the default pygeoapi server language set in the ``Content-Language`` HTTP response header.
- It is up to the creator of the provider to properly define at least 1 supported language in the provider configuration, as described
in the `developer guide`_. This will ensure that the ``Content-Language`` HTTP response header is always set properly.
- If pygeoapi found a match to the requested language, the response will include a ``Content-Language`` HTTP header,
set to the best-matching server language code. This is the default behavior for most pygeoapi requests. However, note that some responses
(e.g. exceptions) always have a ``Content-Language: en-US`` header, regardless of the requested language.
- For results returned by a **provider**, the ``Content-Language`` HTTP header will be set to the best-matching
provider language or the best-matching pygeoapi server language if the provider is not language-aware.
- If the provider supports a requested language, but pygeoapi does *not* support that same language, the ``Content-Language``
header will contain both the provider language *and* the best-matching pygeoapi server language.
- Please note that the ``Content-Language`` HTTP response header only *indicates the language of the intended audience*.
It does not necessarily mean that the content is actually written in that particular language.
Maintainer guide
----------------
Every pygeoapi instance should support at least 1 language. In the server configuration, there must be a ``language``
or a ``languages`` (note the `s`) property. The property can be set to a single language tag or a list of tags respectively.
If you wish to set up a multilingual pygeoapi instance, you will have to add more than 1 language to the
server configuration YAML file (i.e. ``pygeoapi-config.yml``). First, you will have to add the supported language tags/codes
as a list. For example, if you wish to support American English and Canadian French, you could do:
.. code-block:: yaml
server:
bind: ...
url: ...
mimetype: ...
encoding: ...
languages:
- en-US
- fr-CA
Next, you will have to provide translations for the configured languages. This involves 3 steps:
1. `Add translations for configurable text values`_ in the server YAML file;
2. Verify if there are any Jinja2 HTML template translations for the configured language(s);
3. Make sure that the provider plugins you need can handle this language as well, if you have the ability to do so.
See the `developer guide`_ for more details.
Notes
^^^^^
- The **first** language you define in the configuration determines the default language, i.e. the language that pygeoapi will
use if no other language was requested or no best match for the requested language could be found.
- It is not possible to **disable** language support in pygeoapi. The functionality is always on and a ``Content-Language``
HTTP response header is always set. If results should be available in a single language, you'd have to set that language only
in the pygeoapi configuration.
- Results returned from a provider may be in a different language than pygeoapi's own server language. The "raw" requested language
is always passed on to the provider, even if pygeoapi itself does not support it. For more information, see the `end user guide`_
and the `developer guide`_.
Add translations for configurable text values
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For most of the text values in pygeoapi's server configuration where it makes sense, you can add translations.
Consider the ``metadata`` section for example. The English-only version looks similar to this:
.. code-block:: yaml
metadata:
identification:
title: pygeoapi default instance
description: pygeoapi provides an API to geospatial data
keywords:
- geospatial
- data
- api
If you wish to make these text values available in English and French, you could use the following language struct:
.. code-block:: yaml
metadata:
identification:
title:
en: pygeoapi default instance
fr: instance par défaut de pygeoapi
description:
en: pygeoapi provides an API to geospatial data
fr: pygeoapi fournit une API aux données géospatiales
keywords:
en:
- geospatial
- data
- api
fr:
- géospatiale
- données
- api
In other words: each plain text value should be replaced by a dictionary, where the language code is the key and the translated text represents the matching value.
For lists, this can be applied as well (see ``keywords`` example above), as long as you nest the entire list under a language key instead of each list item.
Note that the example above uses generic language tags, but you can also supply more localized tags (with a country code) if required.
pygeoapi should always be able find the best match to the requested language, i.e. if the user wants Swiss-French (`fr-CH`) but pygeoapi can only find `fr` tags,
those values will be returned. However, if a `fr-CH` tag can also be found, that value will be returned and not the `fr` value.
.. todo:: Add docs on HTML templating.
Developer guide
---------------
If you are a developer who wishes to create a pygeoapi provider plugin that "speaks" a certain language,
you will have to fully implement this yourself. Needless to say, if your provider depends on some backend, it will only make sense to
implement language support if the backend can be queried in another language as well.
You are free to set up the language support anyway you like, but there are a couple of steps you'll have to walk through:
1. You will have to define the supported languages in the provider configuration YAML. This can be done in a similar fashion
as the ``languages`` configuration for pygeoapi itself, as described in the `maintainer guide`_ section above.
For example, a TinyDB records provider that supports English and French could be set up like:
.. code-block:: yaml
my-records:
type: collection
..
providers:
- type: record
name: TinyDBCatalogue
data: ..
languages:
- en
- fr
2. If your provider implements any of the ``query``, ``get`` or ``get_metadata`` methods of the base class and you wish
to make them language-aware, either add an implicit ``**kwargs`` parameter or an explicit ``language=None`` parameter
to the method signature.
An example Python code block for a custom provider with a language-aware ``query`` method could look like this:
.. code-block:: python
class MyCoolVectorDataProvider(BaseProvider):
"""My cool vector data provider"""
def __init__(self, provider_def):
super().__init__(provider_def)
def query(self, startindex=0, limit=10, resulttype='results', bbox=[],
datetime_=None, properties=[], sortby=[], select_properties=[],
skip_geometry=False, q=None, language=None):
LOGGER.debug(f'Provider queried in {language.english_name} language')
# Implement your logic here, returning JSON in the requested language
Alternatively, you could also use ``**kwargs`` in the ``query`` method and get the ``language`` value:
.. code-block:: python
def query(self, **kwargs):
LOGGER.debug(f"Provider locale set to: {kwargs.get('language')}")
# Implement your logic here, returning JSON in the requested language
This is all that is required. The pygeoapi API class will make sure that the correct HTTP ``Content-Language`` headers are set on the response object.
Notes
^^^^^
- If your provider implements any of the aforementioned ``query``, ``get`` and ``get_metadata`` methods,
it **must** add a ``**kwargs`` or ``language=None`` parameter, even if it does not need to use the language parameter.
- Contrary to the pygeoapi server configuration, adding a ``language`` or ``languages`` (both are supported) property to the
provider definition is **not** required and may be omitted. In that case, the passed-in ``language`` parameter language-aware provider methods
(``query``, ``get``, etc.) will be set to ``None``. This results in the following behavior:
- HTML responses returned from the providers will have the ``Content-Language`` header set to the best-matching pygeoapi server language.
- JSON(-LD) responses returned from providers will **not** have a ``Content-Language`` header if ``language`` is ``None``.
- If the provider supports a requested language, the passed-in ``language`` will be set to the best matching
`Babel Locale instance <http://babel.pocoo.org/en/latest/api/core.html#babel.core.Locale>`_.
Note that this may be the provider default language if no proper match was found.
No matter the output format, API responses returned from providers will always contain a best-matching ``Content-Language``
header if one ore more supported provider languages were defined.
- For general information about building plugins, please visit the :ref:`plugins` page.
docs/source/plugins.rst
+5 -2
View File
@@ -70,7 +70,7 @@ The below template provides a minimal example (let's call the file ``mycoolvecto
def query(self,startindex=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
select_properties=[], skip_geometry=False):
select_properties=[], skip_geometry=False, **kwargs):
# open data file (self.data) and process, return
return {
@@ -98,6 +98,9 @@ its base provider, all other functionality is left to the provider implementatio
Each base class documents the functions, arguments and return types required for implementation.
.. note:: You can add language support to your plugin using :ref:`these guides<language>`.
Connecting to pygeoapi
^^^^^^^^^^^^^^^^^^^^^^
@@ -166,7 +169,7 @@ The below template provides a minimal example (let's call the file ``mycoolraste
def get_coverage_rangetype(self):
# return a CIS JSON RangeType
def query(self, bands=[], subsets={}, format_='json'):
def query(self, bands=[], subsets={}, format_='json', **kwargs):
# process bands and subsets parameters
# query/extract coverage data
if format_ == 'json':
pygeoapi-config.yml
+47 -13
View File
@@ -34,7 +34,10 @@ server:
url: http://localhost:5000
mimetype: application/json; charset=UTF-8
encoding: utf-8
language: en-US
languages:
# First language is the default language
- en-US
- fr-CA
# cors: true
pretty_print: true
limit: 10
@@ -56,12 +59,21 @@ logging:
metadata:
identification:
title: pygeoapi default instance
description: pygeoapi provides an API to geospatial data
title:
en: pygeoapi default instance
fr: instance par défaut de pygeoapi
description:
en: pygeoapi provides an API to geospatial data
fr: pygeoapi fournit une API aux données géospatiales
keywords:
- geospatial
- data
- api
en:
- geospatial
- data
- api
fr:
- géospatiale
- données
- api
keywords_type: theme
terms_of_service: https://creativecommons.org/licenses/by/4.0/
url: http://example.org
@@ -129,10 +141,19 @@ resources:
lakes:
type: collection
title: Large Lakes
description: lakes of the world, public domain
title:
en: Large Lakes
fr: Grands Lacs
description:
en: lakes of the world, public domain
fr: lacs du monde, domaine public
keywords:
- lakes
en:
- lakes
- water bodies
fr:
- lacs
- plans d'eau
links:
- type: text/html
rel: canonical
@@ -211,17 +232,30 @@ resources:
canada-metadata:
type: collection
title: Sample metadata records from open.canada.ca
description: Sample metadata records from open.canada.ca
title:
en: Open Canada sample data
fr: Exemple de donn\u00e9es Canada Ouvert
description:
en: Sample metadata records from open.canada.ca
fr: Exemples d'enregistrements de m\u00e9tadonn\u00e9es sur ouvert.canada.ca
keywords:
- canada
- open data
en:
- canada
- open data
fr:
- canada
- donn\u00e9es ouvertes
links:
- type: text/html
rel: canonical
title: information
href: https://open.canada.ca/en/open-data
hreflang: en-CA
- type: text/html
rel: canonical
title: informations
href: https://ouvert.canada.ca/fr/donnees-ouvertes
hreflang: fr-CA
extents:
spatial:
bbox: [-180,-90,180,90]
pygeoapi/api.py
+1041 -749
View File
File diff suppressed because it is too large Load Diff
pygeoapi/flask_app.py
+49 -201
View File
@@ -96,6 +96,20 @@ if (OGC_SCHEMAS_LOCATION is not None and
mimetype=get_mimetype(basename_))
def get_response(result: tuple):
""" Creates a Flask Response object and updates matching headers.
:param result: The result of the API call.
This should be a tuple of (headers, status, content).
:returns: A Response instance.
"""
headers, status, content = result
response = make_response(content, status)
if headers:
response.headers = headers
return response
@BLUEPRINT.route('/')
def landing_page():
"""
@@ -103,15 +117,7 @@ def landing_page():
:returns: HTTP response
"""
headers, status_code, content = api_.landing_page(
request.headers, request.args)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.landing_page(request))
@BLUEPRINT.route('/openapi')
@@ -121,22 +127,13 @@ def openapi():
:returns: HTTP response
"""
with open(os.environ.get('PYGEOAPI_OPENAPI'), encoding='utf8') as ff:
if os.environ.get('PYGEOAPI_OPENAPI').endswith(('.yaml', '.yml')):
openapi = yaml_load(ff)
openapi_ = yaml_load(ff)
else: # JSON file, do not transform
openapi = ff
openapi_ = ff
headers, status_code, content = api_.openapi(
request.headers, request.args, openapi)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.openapi(request, openapi_))
@BLUEPRINT.route('/conformance')
@@ -146,16 +143,7 @@ def conformance():
:returns: HTTP response
"""
headers, status_code, content = api_.conformance(request.headers,
request.args)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.conformance(request))
@BLUEPRINT.route('/collections')
@@ -168,16 +156,7 @@ def collections(collection_id=None):
:returns: HTTP response
"""
headers, status_code, content = api_.describe_collections(
request.headers, request.args, collection_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.describe_collections(request, collection_id))
@BLUEPRINT.route('/collections/<collection_id>/queryables')
@@ -189,16 +168,7 @@ def collection_queryables(collection_id=None):
:returns: HTTP response
"""
headers, status_code, content = api_.get_collection_queryables(
request.headers, request.args, collection_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_collection_queryables(request, collection_id))
@BLUEPRINT.route('/collections/<collection_id>/items')
@@ -212,20 +182,10 @@ def collection_items(collection_id, item_id=None):
:returns: HTTP response
"""
if item_id is None:
headers, status_code, content = api_.get_collection_items(
request.headers, request.args, collection_id)
else:
headers, status_code, content = api_.get_collection_item(
request.headers, request.args, collection_id, item_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_collection_items(request, collection_id))
return get_response(
api_.get_collection_item(request, collection_id, item_id))
@BLUEPRINT.route('/collections/<collection_id>/coverage')
@@ -237,16 +197,7 @@ def collection_coverage(collection_id):
:returns: HTTP response
"""
headers, status_code, content = api_.get_collection_coverage(
request.headers, request.args, collection_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_collection_coverage(request, collection_id))
@BLUEPRINT.route('/collections/<collection_id>/coverage/domainset')
@@ -258,16 +209,8 @@ def collection_coverage_domainset(collection_id):
:returns: HTTP response
"""
headers, status_code, content = api_.get_collection_coverage_domainset(
request.headers, request.args, collection_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_collection_coverage_domainset(
request, collection_id))
@BLUEPRINT.route('/collections/<collection_id>/coverage/rangetype')
@@ -279,16 +222,8 @@ def collection_coverage_rangetype(collection_id):
:returns: HTTP response
"""
headers, status_code, content = api_.get_collection_coverage_rangetype(
request.headers, request.args, collection_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_collection_coverage_rangetype(
request, collection_id))
@BLUEPRINT.route('/collections/<collection_id>/tiles')
@@ -300,16 +235,8 @@ def get_collection_tiles(collection_id=None):
:returns: HTTP response
"""
headers, status_code, content = api_.get_collection_tiles(
request.headers, request.args, collection_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_collection_tiles(
request, collection_id))
@BLUEPRINT.route('/collections/<collection_id>/tiles/<tileMatrixSetId>/metadata') # noqa
@@ -322,16 +249,8 @@ def get_collection_tiles_metadata(collection_id=None, tileMatrixSetId=None):
:returns: HTTP response
"""
headers, status_code, content = api_.get_collection_tiles_metadata(
request.headers, request.args, collection_id, tileMatrixSetId)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_collection_tiles_metadata(
request, collection_id, tileMatrixSetId))
@BLUEPRINT.route('/collections/<collection_id>/tiles/\
@@ -349,17 +268,8 @@ def get_collection_tiles_data(collection_id=None, tileMatrixSetId=None,
:returns: HTTP response
"""
headers, status_code, content = api_.get_collection_tiles_data(
request.headers, request.args, collection_id,
tileMatrixSetId, tileMatrix, tileRow, tileCol)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_collection_tiles_data(
request, collection_id, tileMatrixSetId, tileMatrix, tileRow, tileCol))
@BLUEPRINT.route('/processes')
@@ -372,15 +282,7 @@ def get_processes(process_id=None):
:returns: HTTP response
"""
headers, status_code, content = api_.describe_processes(
request.headers, request.args, process_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.describe_processes(request, process_id))
@BLUEPRINT.route('/processes/<process_id>/jobs', methods=['GET', 'POST'])
@@ -395,28 +297,17 @@ def get_process_jobs(process_id=None, job_id=None):
:returns: HTTP response
"""
if job_id is None:
if request.method == 'GET': # list jobs
headers, status_code, content = api_.get_process_jobs(
request.headers, request.args, process_id)
return get_response(api_.get_process_jobs(request, process_id))
elif request.method == 'POST': # submit job
headers, status_code, content = api_.execute_process(
request.headers, request.args, request.data, process_id)
return get_response(api_.execute_process(request, process_id))
else:
if request.method == 'DELETE': # dismiss job
headers, status_code, content = api_.delete_process_job(
process_id, job_id)
return get_response(api_.delete_process_job(process_id, job_id))
else: # Return status of a specific job
headers, status_code, content = api_.get_process_jobs(
request.headers, request.args, process_id, job_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_process_jobs(
request, process_id, job_id))
@BLUEPRINT.route('/processes/<process_id>/jobs/<job_id>/results',
@@ -430,16 +321,8 @@ def get_process_job_result(process_id=None, job_id=None):
:returns: HTTP response
"""
headers, status_code, content = api_.get_process_job_result(
request.headers, request.args, process_id, job_id)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_process_job_result(
request, process_id, job_id))
@BLUEPRINT.route('/processes/<process_id>/jobs/<job_id>/results/<resource>',
@@ -454,16 +337,8 @@ def get_process_job_result_resource(process_id, job_id, resource):
:returns: HTTP response
"""
headers, status_code, content = api_.get_process_job_result_resource(
request.headers, request.args, process_id, job_id, resource)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_process_job_result_resource(
request, process_id, job_id, resource))
@BLUEPRINT.route('/collections/<collection_id>/position')
@@ -485,18 +360,9 @@ def get_collection_edr_query(collection_id, instance_id=None):
:returns: HTTP response
"""
query_type = request.path.split('/')[-1]
headers, status_code, content = api_.get_collection_edr_query(
request.headers, request.args, collection_id, instance_id, query_type)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_collection_edr_query(request, collection_id,
instance_id, query_type))
@BLUEPRINT.route('/stac')
@@ -506,16 +372,7 @@ def stac_catalog_root():
:returns: HTTP response
"""
headers, status_code, content = api_.get_stac_root(
request.headers, request.args)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_stac_root(request))
@BLUEPRINT.route('/stac/<path:path>')
@@ -527,16 +384,7 @@ def stac_catalog_path(path):
:returns: HTTP response
"""
headers, status_code, content = api_.get_stac_path(
request.headers, request.args, path)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
return get_response(api_.get_stac_path(request, path))
APP.register_blueprint(BLUEPRINT)
pygeoapi/l10n.py
+479
View File
@@ -0,0 +1,479 @@
# =================================================================
#
# Authors: Sander Schaminee <sander.schaminee@geocat.net>
#
# Copyright (c) 2021 GeoCat BV
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
# files (the "Software"), to deal in the Software without
# restriction, including without limitation the rights to use,
# copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following
# conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
# =================================================================
import logging
from typing import Union
from collections import OrderedDict
from copy import deepcopy
from babel import Locale
from babel import UnknownLocaleError as _UnknownLocaleError
from urllib import parse
LOGGER = logging.getLogger(__name__)
# Specifies the name of a request query parameter used to set a locale
QUERY_PARAM = 'lang'
# Cache Babel Locale lookups by string
_lc_cache = {}
# Cache translated configurations
_cfg_cache = {}
class LocaleError(Exception):
""" General exception for any kind of locale parsing error. """
pass
def str2locale(value, silent: bool = False) -> Union[Locale, None]:
""" Converts a web locale or language tag into a Babel Locale instance.
.. note:: If `value` already is a Locale, it is returned as-is.
:param value: A string containing a (web) locale (e.g. 'fr-CH')
or language tag (e.g. 'de').
:param silent: If True (default = False), no errors will be raised
when parsing failed. Instead, `None` will be returned.
:returns: babel.core.Locale or None
:raises: LocaleError
"""
if isinstance(value, Locale):
return value
loc = _lc_cache.get(value)
if loc:
# Value has been converted before: return cached Locale
return loc
try:
loc = Locale.parse(value.strip().replace('-', '_'))
except (ValueError, AttributeError):
if not silent:
raise LocaleError(f"invalid locale '{value}'")
except _UnknownLocaleError as err:
if not silent:
raise LocaleError(err)
else:
# Add to Locale cache
_lc_cache[value] = loc
return loc
def locale2str(value: Locale) -> str:
""" Converts a Babel Locale instance into a web locale string.
:param value: babel.core.Locale
:returns: A string containing a web locale (e.g. 'fr-CH')
or language tag (e.g. 'de').
:raises: LocaleError
"""
if not isinstance(value, Locale):
raise LocaleError(f"'{value}' is not of type {Locale.__name__}")
return str(value).replace('_', '-')
def best_match(accept_languages, available_locales) -> Locale:
""" Takes an Accept-Languages string (from header or request query params)
and finds the best matching locale from a list of available locales.
This function provides a framework-independent alternative to the
`best_match()` function available in Flask/Werkzeug.
If no match can be found for the Accept-Languages,
the first available locale is returned.
This function always returns a Babel Locale instance. If you require the
web locale string, please use the :func:`locale2str` function.
If you only ever need the language part of the locale, use the `language`
property of the returned locale.
.. note:: Any tag in the `accept_languages` string that is an invalid
or unknown locale is ignored. However, if no
`available_locales` are specified, a `LocaleError` is raised.
:param accept_languages: A Locale or string with one or more languages.
This can be as simple as "de" for example,
but it's also possible to include a territory
(e.g. "en-US" or "fr_BE") or even a complex
string with quality values, e.g.
"fr-CH, fr;q=0.9, en;q=0.8, de;q=0.7, *;q=0.5".
:param available_locales: A list containing the available locales.
For example, a pygeoapi provider might only
support ["de", "en"].
Locales in the list can be specified as strings
(e.g. "nl-NL") or `Locale` instances.
:returns: babel.core.Locale
:raises: LocaleError
"""
def get_match(locale_, available_locales_):
""" Finds the first match of `locale_` in `available_locales_`. """
if not locale_:
return None
territories_ = available_locales_.get(locale_.language, {})
if locale_.territory in territories_:
# Full match on language and territory
return locale_
if None in territories_:
# Match on language only (generic, no territory)
return Locale(locale_.language)
if territories_:
# Match on language but another territory (use first)
return Locale(locale_.language, territory=territories_[0])
# No match at all
return None
if not available_locales:
raise LocaleError('No available locales specified')
if isinstance(accept_languages, Locale):
# If a Babel Locale was used as input, transform back into a string
accept_languages = locale2str(accept_languages)
if not isinstance(accept_languages, str):
# If `accept_languages` is not a string, ignore it
LOGGER.debug(f"ignoring invalid accept-languages '{accept_languages}'")
accept_languages = ''
tags = accept_languages.split(',')
num_tags = len(tags)
req_locales = {}
for i, lang in enumerate(tags):
q_raw = None
q_out = None
if not lang:
continue
# Check if complex (i.e. with quality weights)
try:
lang, q_raw = (v.strip() for v in lang.split(';'))
except ValueError:
# Tuple unpacking failed: tag is not complex (or too complex :))
pass
# Validate locale tag
loc = str2locale(lang, True)
if not loc:
LOGGER.debug(f"ignoring invalid accept-language '{lang}'")
continue
# Validate quality weight (e.g. "q=0.7")
if q_raw:
try:
q_out = float([v.strip() for v in q_raw.split('=')][1])
except (ValueError, IndexError):
# Tuple unpacking failed: not a valid q tag
pass
# If there's no actual q, set one based on the language order
if not q_out:
q_out = num_tags - i
# Store locale
req_locales[q_out] = loc
# Process supported locales
prv_locales = OrderedDict()
for a in available_locales:
loc = str2locale(a)
prv_locales.setdefault(loc.language, []).append(loc.territory)
# Return best match from accepted languages
for _, loc in sorted(req_locales.items(), reverse=True):
match = get_match(loc, prv_locales)
if match:
LOGGER.debug(f"'{match}' matches requested '{accept_languages}'")
return match
# Nothing matched: return the first available locale
for lang, territories in prv_locales.items():
match = Locale(lang, territory=territories[0])
LOGGER.debug(f"No match found for language '{accept_languages}'; "
f"returning default locale '{match}'")
return match
def translate(value, language: Union[Locale, str]):
"""
If `value` is a language struct (where its keys are language codes
and its values are translations for each language), this function tries to
find and return the translation for the given `language`.
If the given `value` is not a dict, the original value is returned.
If the requested language does not exist in the struct,
the first language value is returned. If there are no valid language keys
in the struct, the original value is returned as well.
If `language` is not a string or Locale, a LocaleError is raised.
:param value: A value to translate. Typically either a string or
a language struct dictionary.
:param language: A locale string (e.g. "en-US" or "en") or Babel Locale.
:returns: A translated string or the original value.
:raises: LocaleError
"""
nested_dicts = isinstance(value, dict) and any(isinstance(v, dict)
for v in value.values())
if not isinstance(value, dict) or nested_dicts:
# Return non-dicts or dicts with nested dicts as-is
return value
# Validate language key by type (do not check if parsable)
if not isinstance(language, (str, Locale)):
raise LocaleError('language is not a str or Locale')
# First try fast approach: directly fetch expected language key
translation = value.get(locale2str(language)
if hasattr(language, 'language') else language)
if translation:
return translation
# Find valid locale keys in language struct
# Also maps Locale instances to actual key names
loc_items = OrderedDict()
for k in value.keys():
loc = str2locale(k, True)
if loc:
loc_items[loc] = k
if not loc_items:
# No valid locale keys found: return as-is
return value
# Find best language match and return value by its key
out_locale = best_match(language, loc_items)
return value[loc_items[out_locale]]
def translate_struct(struct, locale_: Locale, is_config: bool = False):
""" Returns a copy of a given dict or list, where all language structs
are filtered on the given locale, i.e. all language structs are replaced
by translated values for the best matching locale.
:param struct: A dict or list (of dicts) to filter/translate.
:param locale_: The Babel Locale to filter on.
:param is_config: If True, the struct is treated as a pygeoapi config.
This means that the first 2 levels won't be translated
and the translated struct is cached for speed.
:returns: A translated dict or list
"""
def _translate_dict(obj, level: int = 0):
""" Recursive function to walk and translate a struct. """
items = obj.items() if isinstance(obj, dict) else enumerate(obj)
for k, v in items:
if 0 <= level <= max_level and isinstance(v, (dict, list)):
# Skip first 2 levels (don't translate)
_translate_dict(v, level + 1)
continue
if isinstance(v, list):
_translate_dict(v, level + 1) # noqa
continue
tr = translate(v, locale_)
if isinstance(tr, dict):
# Look for language structs in next level
_translate_dict(tr, level + 1)
else:
# Overwrite level with translated value
obj[k] = tr
max_level = 1 if is_config else -1
result = {}
if not struct:
return result
if not locale_:
return struct
# Check if we already translated the dict before
result = _cfg_cache.get(locale_) if is_config else result
if not result:
# Create deep copy of config and translate/filter values
result = deepcopy(struct)
_translate_dict(result)
# Cache translated pygeoapi configs for faster retrieval next time
if is_config:
_cfg_cache[locale_] = result
return result
def locale_from_headers(headers) -> str:
"""
Gets a valid Locale from a request headers dictionary.
Supported are complex strings (e.g. "fr-CH, fr;q=0.9, en;q=0.8"),
web locales (e.g. "en-US") or basic language tags (e.g. "en").
A value of `None` is returned if the locale was not found or invalid.
:param headers: Mapping of request headers.
:returns: locale string or None
"""
lang = {k.lower(): v for k, v in headers.items()}.get('accept-language')
if lang:
LOGGER.debug(f"Got locale '{lang}' from 'Accept-Language' header")
return lang
def locale_from_params(params) -> str:
"""
Gets a valid Locale from a request query parameters dictionary.
Supported are complex strings (e.g. "fr-CH, fr;q=0.9, en;q=0.8"),
web locales (e.g. "en-US") or basic language tags (e.g. "en").
A value of `None` is returned if the locale was not found or invalid.
:param params: Mapping of request query parameters.
:returns: locale string or None
"""
lang = params.get(QUERY_PARAM)
if lang:
LOGGER.debug(f"Got locale '{lang}' from query parameter '{QUERY_PARAM}'") # noqa
return lang
def set_response_language(headers: dict, *locale_: Locale):
""" Sets the Content-Language on the given HTTP response headers dict.
:param headers: A dict of HTTP response headers.
:param locale_: The Babel Locale(s) to which to set the
Content-Language header.
Multiple locales can be set for this header.
Note that duplicates will be removed.
:raises: LocaleError if no valid Babel Locale was found.
"""
if not hasattr(headers, '__setitem__'):
LOGGER.warning(f"Cannot set headers on object '{headers}'")
return
locales = []
for loc in locale_:
try:
loc_str = locale2str(loc)
except LocaleError:
if len(locale_) == 1:
raise
else:
if loc_str not in locales:
locales.append(loc_str)
if not locales:
raise LocaleError('no valid locales set')
loc_str = ', '.join(locales)
LOGGER.debug(f'Setting Content-Language to {loc_str}')
headers['Content-Language'] = loc_str
def add_locale(url, locale_):
""" Adds a locale query parameter (e.g. 'lang=en-US') to a URL.
If `locale_` is None or an empty string, the URL will be returned as-is.
:param url: The web page URL (may contain query string).
:param locale_: The web locale or language tag to append to the query.
:returns: A new URL with a 'lang=<locale>' query parameter.
:raises: requests.exceptions.MissingSchema
"""
loc = str2locale(locale_, True)
if not loc:
# Validation of locale failed
LOGGER.warning(
f"Invalid locale '{locale_}': returning URL as-is")
return url
try:
url_comp = parse.urlparse(url)
params = dict(parse.parse_qsl(url_comp.query))
params[QUERY_PARAM] = locale2str(loc)
qstr = parse.urlencode(params, quote_via=parse.quote, safe='/')
return parse.urlunparse((
url_comp.scheme,
url_comp.netloc,
url_comp.path,
url_comp.params,
qstr,
url_comp.fragment
))
except (TypeError, ValueError):
LOGGER.warning(
f"Failed to append '{QUERY_PARAM}={loc}': returning URL as-is") # noqa
return url
def get_locales(config: dict) -> list:
""" Reads the configured locales/languages from the given configuration.
The first Locale in the returned list should be the default locale.
:param config: A pygeaapi configuration dict
:returns: A list of supported Locale instances
"""
srv_cfg = config.get('server', {})
lang = srv_cfg.get('languages', srv_cfg.get('language', []))
if isinstance(lang, str):
LOGGER.info(f"pygeoapi only supports 1 language: {lang}")
lang = [lang]
if not isinstance(lang, list) or len(lang) == 0:
LOGGER.error("Missing 'language(s)' key in config or bad value(s)")
raise LocaleError('No languages have been configured')
try:
return [str2locale(loc) for loc in lang]
except LocaleError as err:
LOGGER.debug(err)
raise LocaleError('Bad value in supported server language(s)')
def get_plugin_locale(config: dict, requested_locale: Union[str, None]) -> Union[Locale, None]: # noqa
""" Returns the supported locale (best match) for a plugin
based on the requested raw locale string.
Returns None if the plugin does not support any locales.
Returns the default (= first) locale that the plugin supports
if no match for the requested locale could be found.
:param config: The plugin definition
:param requested_locale: The requested locale string (or None)
"""
plugin_name = f"{config.get('name', '')} plugin".strip()
if not requested_locale:
LOGGER.debug(f'No requested locale for {plugin_name}')
requested_locale = ''
LOGGER.debug(f'Requested {plugin_name} locale: {requested_locale}')
locales = config.get('languages', config.get('language', []))
if locales:
if not isinstance(locales, list):
locales = [locales]
locale = best_match(requested_locale, locales)
LOGGER.info(f'{plugin_name} locale set to {locale}')
return locale
LOGGER.info(f'{plugin_name} has no locale support')
return None
pygeoapi/linked_data.py
+26 -15
View File
@@ -33,6 +33,7 @@ Returns content as linked data representations
import logging
from pygeoapi.util import is_url
from pygeoapi import l10n
LOGGER = logging.getLogger(__name__)
@@ -48,9 +49,12 @@ def jsonldify(func):
"""
def inner(*args, **kwargs):
format_ = args[2]
apireq = args[1]
format_ = getattr(apireq, 'format', None)
if not format_ == 'jsonld':
return func(*args, **kwargs)
# Function args have been pre-processed, so get locale from APIRequest
locale_ = getattr(apireq, 'locale', None)
LOGGER.debug('Creating JSON-LD representation')
cls = args[0]
cfg = cls.config
@@ -63,14 +67,17 @@ def jsonldify(func):
"@type": "DataCatalog",
"@id": cfg.get('server', {}).get('url', None),
"url": cfg.get('server', {}).get('url', None),
"name": ident.get('title', None),
"description": ident.get('description', None),
"keywords": ident.get('keywords', None),
"termsOfService": ident.get('terms_of_service', None),
"name": l10n.translate(ident.get('title', None), locale_),
"description": l10n.translate(
ident.get('description', None), locale_),
"keywords": l10n.translate(
ident.get('keywords', None), locale_),
"termsOfService": l10n.translate(
ident.get('terms_of_service', None), locale_),
"license": meta.get('license', {}).get('url', None),
"provider": {
"@type": "Organization",
"name": provider.get('name', None),
"name": l10n.translate(provider.get('name', None), locale_),
"url": provider.get('url', None),
"address": {
"@type": "PostalAddress",
@@ -88,10 +95,13 @@ def jsonldify(func):
"url": contact.get('url', None),
"hoursAvailable": {
"opens": contact.get('hours', None),
"description": contact.get('instructions', None)
"description": l10n.translate(
contact.get('instructions', None), locale_)
},
"contactType": contact.get('role', None),
"description": contact.get('position', None)
"contactType": l10n.translate(
contact.get('role', None), locale_),
"description": l10n.translate(
contact.get('position', None), locale_)
}
}
}
@@ -100,13 +110,14 @@ def jsonldify(func):
return inner
def jsonldify_collection(cls, collection):
def jsonldify_collection(cls, collection, locale_):
"""
Transforms collection into a JSON-LD representation
:param cls: API object
:param collection: `collection` as prepared for non-LD JSON
representation
:param locale_: The locale to use for translations (if supported)
:returns: `collection` a dictionary, mapped into JSON-LD, of
type schema:Dataset
@@ -125,10 +136,10 @@ def jsonldify_collection(cls, collection):
cls.config['server']['url'],
collection['id']
),
"name": collection['title'],
"description": collection['description'],
"name": l10n.translate(collection['title'], locale_),
"description": l10n.translate(collection['description'], locale_),
"license": cls.fcmld['license'],
"keywords": collection.get('keywords', None),
"keywords": l10n.translate(collection.get('keywords', None), locale_),
"spatial": None if (not hascrs84 or not bbox) else [{
"@type": "Place",
"geo": {
@@ -148,9 +159,9 @@ def jsonldify_collection(cls, collection):
"@type": "DataDownload",
"contentURL": link['href'],
"encodingFormat": link['type'],
"description": link['title'],
"description": l10n.translate(link['title'], locale_),
"inLanguage": link.get(
'hreflang', cls.config.get('server', {}).get('language', None)
'hreflang', l10n.locale2str(cls.default_locale)
),
"author": link['rel'] if link.get(
'rel', None
pygeoapi/openapi.py
+104 -70
View File
@@ -35,6 +35,7 @@ import click
import yaml
from pygeoapi import __version__
from pygeoapi import l10n
from pygeoapi.plugin import load_plugin
from pygeoapi.provider.base import ProviderTypeError
from pygeoapi.util import (filter_dict_by_key_value, get_provider_by_type,
@@ -125,6 +126,10 @@ def get_oas_30(cfg):
paths = {}
# TODO: make openapi multilingual (default language only for now)
server_locales = l10n.get_locales(cfg)
locale_ = server_locales[0]
osl = get_ogc_schemas_location(cfg['server'])
OPENAPI_YAML['oapif'] = os.path.join(osl, 'ogcapi/features/part1/1.0/openapi/ogcapi-features-1.yaml') # noqa
@@ -134,9 +139,9 @@ def get_oas_30(cfg):
'tags': []
}
info = {
'title': cfg['metadata']['identification']['title'],
'description': cfg['metadata']['identification']['description'],
'x-keywords': cfg['metadata']['identification']['keywords'],
'title': l10n.translate(cfg['metadata']['identification']['title'], locale_), # noqa
'description': l10n.translate(cfg['metadata']['identification']['description'], locale_), # noqa
'x-keywords': l10n.translate(cfg['metadata']['identification']['keywords'], locale_), # noqa
'termsOfService':
cfg['metadata']['identification']['terms_of_service'],
'contact': {
@@ -154,7 +159,7 @@ def get_oas_30(cfg):
oas['servers'] = [{
'url': cfg['server']['url'],
'description': cfg['metadata']['identification']['description']
'description': l10n.translate(cfg['metadata']['identification']['description'], locale_) # noqa
}]
paths['/'] = {
@@ -164,7 +169,8 @@ def get_oas_30(cfg):
'tags': ['server'],
'operationId': 'getLandingPage',
'parameters': [
{'$ref': '#/components/parameters/f'}
{'$ref': '#/components/parameters/f'},
{'$ref': '#/components/parameters/lang'}
],
'responses': {
'200': {'$ref': '{}#/components/responses/LandingPage'.format(OPENAPI_YAML['oapif'])}, # noqa
@@ -181,7 +187,8 @@ def get_oas_30(cfg):
'tags': ['server'],
'operationId': 'getOpenapi',
'parameters': [
{'$ref': '#/components/parameters/f'}
{'$ref': '#/components/parameters/f'},
{'$ref': '#/components/parameters/lang'}
],
'responses': {
'200': {'$ref': '#/components/responses/200'},
@@ -198,7 +205,8 @@ def get_oas_30(cfg):
'tags': ['server'],
'operationId': 'getConformanceDeclaration',
'parameters': [
{'$ref': '#/components/parameters/f'}
{'$ref': '#/components/parameters/f'},
{'$ref': '#/components/parameters/lang'}
],
'responses': {
'200': {'$ref': '{}#/components/responses/ConformanceDeclaration'.format(OPENAPI_YAML['oapif'])}, # noqa
@@ -215,7 +223,8 @@ def get_oas_30(cfg):
'tags': ['server'],
'operationId': 'getCollections',
'parameters': [
{'$ref': '#/components/parameters/f'}
{'$ref': '#/components/parameters/f'},
{'$ref': '#/components/parameters/lang'}
],
'responses': {
'200': {'$ref': '{}#/components/responses/Collections'.format(OPENAPI_YAML['oapif'])}, # noqa
@@ -227,7 +236,7 @@ def get_oas_30(cfg):
oas['tags'].append({
'name': 'server',
'description': cfg['metadata']['identification']['description'],
'description': l10n.translate(cfg['metadata']['identification']['description'], locale_), # noqa
'externalDocs': {
'description': 'information',
'url': cfg['metadata']['identification']['url']}
@@ -271,6 +280,17 @@ def get_oas_30(cfg):
'style': 'form',
'explode': False
},
'lang': {
'name': 'lang',
'in': 'query',
'description': 'The optional lang parameter instructs the server return a response in a certain language, if supported. If the language is not among the available values, the Accept-Language header language will be used if it is supported. If the header is missing, the default server language is used. Note that providers may only support a single language (or often no language at all), that can be different from the server language. Language strings can be written in a complex (e.g. "fr-CA,fr;q=0.9,en-US;q=0.8,en;q=0.7"), simple (e.g. "de") or locale-like (e.g. "de-CH" or "fr_BE") fashion.', # noqa
'required': False,
'schema': {
'type': 'string',
'enum': [l10n.locale2str(sl) for sl in server_locales],
'default': l10n.locale2str(locale_)
}
},
'properties': {
'name': 'properties',
'in': 'query',
@@ -367,19 +387,23 @@ def get_oas_30(cfg):
items_f = deepcopy(oas['components']['parameters']['f'])
items_f['schema']['enum'].append('csv')
items_l = deepcopy(oas['components']['parameters']['lang'])
LOGGER.debug('setting up datasets')
collections = filter_dict_by_key_value(cfg['resources'],
'type', 'collection')
for k, v in collections.items():
name = l10n.translate(k, locale_)
title = l10n.translate(v['title'], locale_)
desc = l10n.translate(v['description'], locale_)
collection_name_path = '/collections/{}'.format(k)
tag = {
'name': k,
'description': v['description'],
'name': name,
'description': desc,
'externalDocs': {}
}
for link in v['links']:
for link in l10n.translate(v['links'], locale_):
if link['type'] == 'information':
tag['externalDocs']['description'] = link['type']
tag['externalDocs']['url'] = link['url']
@@ -391,12 +415,13 @@ def get_oas_30(cfg):
paths[collection_name_path] = {
'get': {
'summary': 'Get collection metadata'.format(v['title']), # noqa
'description': v['description'],
'tags': [k],
'operationId': 'describe{}Collection'.format(k.capitalize()),
'summary': 'Get {} metadata'.format(title),
'description': desc,
'tags': name,
'operationId': 'describe{}Collection'.format(name.capitalize()), # noqa
'parameters': [
{'$ref': '#/components/parameters/f'}
{'$ref': '#/components/parameters/f'},
{'$ref': '#/components/parameters/lang'}
],
'responses': {
'200': {'$ref': '{}#/components/responses/Collection'.format(OPENAPI_YAML['oapif'])}, # noqa
@@ -430,12 +455,13 @@ def get_oas_30(cfg):
paths[items_path] = {
'get': {
'summary': 'Get {} items'.format(v['title']),
'description': v['description'],
'tags': [k],
'operationId': 'get{}Features'.format(k.capitalize()),
'summary': 'Get {} items'.format(title), # noqa
'description': desc,
'tags': [name],
'operationId': 'get{}Features'.format(name.capitalize()),
'parameters': [
items_f,
items_l,
{'$ref': '{}#/components/parameters/bbox'.format(OPENAPI_YAML['oapif'])}, # noqa
{'$ref': '{}#/components/parameters/limit'.format(OPENAPI_YAML['oapif'])}, # noqa
coll_properties,
@@ -460,13 +486,14 @@ def get_oas_30(cfg):
paths[queryables_path] = {
'get': {
'summary': 'Get {} queryables'.format(v['title']),
'description': v['description'],
'tags': [k],
'summary': 'Get {} queryables'.format(title),
'description': desc,
'tags': [name],
'operationId': 'get{}Queryables'.format(
k.capitalize()),
name.capitalize()),
'parameters': [
items_f,
items_l
],
'responses': {
'200': {'$ref': '#/components/responses/Queryables'}, # noqa
@@ -523,13 +550,14 @@ def get_oas_30(cfg):
paths['{}/items/{{featureId}}'.format(collection_name_path)] = {
'get': {
'summary': 'Get {} item by id'.format(v['title']),
'description': v['description'],
'tags': [k],
'operationId': 'get{}Feature'.format(k.capitalize()),
'summary': 'Get {} item by id'.format(title),
'description': desc,
'tags': [name],
'operationId': 'get{}Feature'.format(name.capitalize()),
'parameters': [
{'$ref': '{}#/components/parameters/featureId'.format(OPENAPI_YAML['oapif'])}, # noqa
{'$ref': '#/components/parameters/f'}
{'$ref': '#/components/parameters/f'},
{'$ref': '#/components/parameters/lang'}
],
'responses': {
'200': {'$ref': '{}#/components/responses/Feature'.format(OPENAPI_YAML['oapif'])}, # noqa
@@ -551,12 +579,13 @@ def get_oas_30(cfg):
paths[coverage_path] = {
'get': {
'summary': 'Get {} coverage'.format(v['title']),
'description': v['description'],
'tags': [k],
'operationId': 'get{}Coverage'.format(k.capitalize()),
'summary': 'Get {} coverage'.format(title),
'description': desc,
'tags': [name],
'operationId': 'get{}Coverage'.format(name.capitalize()),
'parameters': [
items_f,
items_l
],
'responses': {
'200': {'$ref': '{}#/components/responses/Features'.format(OPENAPI_YAML['oapif'])}, # noqa
@@ -572,13 +601,14 @@ def get_oas_30(cfg):
paths[coverage_domainset_path] = {
'get': {
'summary': 'Get {} coverage domain set'.format(v['title']),
'description': v['description'],
'tags': [k],
'summary': 'Get {} coverage domain set'.format(title),
'description': desc,
'tags': [name],
'operationId': 'get{}CoverageDomainSet'.format(
k.capitalize()),
name.capitalize()),
'parameters': [
items_f,
items_l
],
'responses': {
'200': {'$ref': '{}/schemas/cis_1.1/domainSet.yaml'.format(OPENAPI_YAML['oacov'])}, # noqa
@@ -594,13 +624,14 @@ def get_oas_30(cfg):
paths[coverage_rangetype_path] = {
'get': {
'summary': 'Get {} coverage range type'.format(v['title']),
'description': v['description'],
'tags': [k],
'summary': 'Get {} coverage range type'.format(title),
'description': desc,
'tags': [name],
'operationId': 'get{}CoverageRangeType'.format(
k.capitalize()),
name.capitalize()),
'parameters': [
items_f,
items_l
],
'responses': {
'200': {'$ref': '{}/schemas/cis_1.1/rangeType.yaml'.format(OPENAPI_YAML['oacov'])}, # noqa
@@ -672,12 +703,13 @@ def get_oas_30(cfg):
paths[tiles_path] = {
'get': {
'summary': 'Fetch a {} tiles description'.format(v['title']), # noqa
'description': v['description'],
'tags': [k],
'operationId': 'describe{}Tiles'.format(k.capitalize()),
'summary': 'Fetch a {} tiles description'.format(title), # noqa
'description': desc,
'tags': [name],
'operationId': 'describe{}Tiles'.format(name.capitalize()),
'parameters': [
items_f,
# items_l TODO: is this useful?
],
'responses': {
'200': {'$ref': '#/components/responses/Tiles'},
@@ -692,10 +724,10 @@ def get_oas_30(cfg):
paths[tiles_data_path] = {
'get': {
'summary': 'Get a {} tile'.format(v['title']),
'description': v['description'],
'tags': [k],
'operationId': 'get{}Tiles'.format(k.capitalize()),
'summary': 'Get a {} tile'.format(title),
'description': desc,
'tags': [name],
'operationId': 'get{}Tiles'.format(name.capitalize()),
'parameters': [
{'$ref': '{}#/components/parameters/tileMatrixSetId'.format(OPENAPI_YAML['oat'])}, # noqa
{'$ref': '{}#/components/parameters/tileMatrix'.format(OPENAPI_YAML['oat'])}, # noqa
@@ -825,15 +857,17 @@ def get_oas_30(cfg):
LOGGER.debug('setting up processes')
for k, v in processes.items():
name = l10n.translate(k, locale_)
p = load_plugin('process', v['processor'])
process_name_path = '/processes/{}'.format(k)
md_desc = l10n.translate(p.metadata['description'], locale_)
process_name_path = '/processes/{}'.format(name)
tag = {
'name': k,
'description': p.metadata['description'],
'name': name,
'description': md_desc, # noqa
'externalDocs': {}
}
for link in p.metadata['links']:
for link in l10n.translate(p.metadata['links'], locale_):
if link['type'] == 'information':
tag['externalDocs']['description'] = link['type']
tag['externalDocs']['url'] = link['url']
@@ -846,9 +880,9 @@ def get_oas_30(cfg):
paths[process_name_path] = {
'get': {
'summary': 'Get process metadata',
'description': p.metadata['description'],
'tags': [k],
'operationId': 'describe{}Process'.format(k.capitalize()),
'description': md_desc,
'tags': [name],
'operationId': 'describe{}Process'.format(name.capitalize()), # noqa
'parameters': [
{'$ref': '#/components/parameters/f'}
],
@@ -861,9 +895,9 @@ def get_oas_30(cfg):
paths['{}/jobs'.format(process_name_path)] = {
'get': {
'summary': 'Retrieve job list for process',
'description': p.metadata['description'],
'tags': [k],
'operationId': 'get{}Jobs'.format(k.capitalize()),
'description': md_desc,
'tags': [name],
'operationId': 'get{}Jobs'.format(name.capitalize()),
'responses': {
'200': {'$ref': '#/components/responses/200'},
'404': {'$ref': '{}/responses/NotFound.yaml'.format(OPENAPI_YAML['oapip'])}, # noqa
@@ -872,10 +906,10 @@ def get_oas_30(cfg):
},
'post': {
'summary': 'Process {} execution'.format(
p.metadata['title']),
'description': p.metadata['description'],
'tags': [k],
'operationId': 'execute{}Job'.format(k.capitalize()),
l10n.translate(p.metadata['title'], locale_)),
'description': md_desc,
'tags': [name],
'operationId': 'execute{}Job'.format(name.capitalize()),
'parameters': [{
'name': 'response',
'in': 'query',
@@ -926,12 +960,12 @@ def get_oas_30(cfg):
'get': {
'summary': 'Retrieve job details',
'description': '',
'tags': [k],
'tags': [name],
'parameters': [
name_in_path,
{'$ref': '#/components/parameters/f'}
],
'operationId': f'get{k.capitalize()}Job',
'operationId': f'get{name.capitalize()}Job',
'responses': {
'200': {'$ref': '#/components/responses/200'},
'404': {'$ref': '{}/responses/NotFound.yaml'.format(OPENAPI_YAML['oapip'])}, # noqa
@@ -941,11 +975,11 @@ def get_oas_30(cfg):
'delete': {
'summary': 'Cancel / delete job',
'description': '',
'tags': [k],
'tags': [name],
'parameters': [
name_in_path
],
'operationId': f'delete{k.capitalize()}Job',
'operationId': f'delete{name.capitalize()}Job',
'responses': {
'204': {'$ref': '#/components/responses/204'},
'404': {'$ref': '{}/responses/NotFound.yaml'.format(OPENAPI_YAML['oapip'])}, # noqa
@@ -958,12 +992,12 @@ def get_oas_30(cfg):
'get': {
'summary': 'Retrieve job results',
'description': '',
'tags': [k],
'tags': [name],
'parameters': [
name_in_path,
{'$ref': '#/components/parameters/f'}
],
'operationId': f'get{k.capitalize()}JobResults',
'operationId': f'get{name.capitalize()}JobResults',
'responses': {
'200': {'$ref': '#/components/responses/200'},
'404': {'$ref': '{}/responses/NotFound.yaml'.format(OPENAPI_YAML['oapip'])}, # noqa
pygeoapi/process/hello_world.py
+12 -4
View File
@@ -38,10 +38,18 @@ LOGGER = logging.getLogger(__name__)
PROCESS_METADATA = {
'version': '0.2.0',
'id': 'hello-world',
'title': 'Hello World',
'description': 'An example process that takes a name as input, and echoes'
'it back as output. Intended to demonstrate a simple'
'process with a single literal input.',
'title': {
'en': 'Hello World',
'fr': 'Bonjour le Monde'
},
'description': {
'en': 'An example process that takes a name as input, and echoes '
'it back as output. Intended to demonstrate a simple '
'process with a single literal input.',
'fr': 'Un exemple de processus qui prend un nom en entrée et le '
'renvoie en sortie. Destiné à démontrer un processus '
'simple avec une seule entrée littérale.',
},
'keywords': ['hello world', 'example', 'echo'],
'links': [{
'type': 'text/html',
pygeoapi/provider/csv_.py
+2 -2
View File
@@ -147,7 +147,7 @@ class CSVProvider(BaseProvider):
def query(self, startindex=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
select_properties=[], skip_geometry=False, q=None):
select_properties=[], skip_geometry=False, q=None, **kwargs):
"""
CSV query
@@ -169,7 +169,7 @@ class CSVProvider(BaseProvider):
select_properties=select_properties,
skip_geometry=skip_geometry)
def get(self, identifier):
def get(self, identifier, **kwargs):
"""
query CSV id
pygeoapi/provider/elasticsearch_.py
+2 -2
View File
@@ -144,7 +144,7 @@ class ElasticsearchProvider(BaseProvider):
def query(self, startindex=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
select_properties=[], skip_geometry=False, q=None):
select_properties=[], skip_geometry=False, q=None, **kwargs):
"""
query Elasticsearch index
@@ -344,7 +344,7 @@ class ElasticsearchProvider(BaseProvider):
return feature_collection
def get(self, identifier):
def get(self, identifier, **kwargs):
"""
Get ES document by id
pygeoapi/provider/geojson.py
+2 -2
View File
@@ -116,7 +116,7 @@ class GeoJSONProvider(BaseProvider):
def query(self, startindex=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
select_properties=[], skip_geometry=False, q=None):
select_properties=[], skip_geometry=False, q=None, **kwargs):
"""
query the provider
@@ -148,7 +148,7 @@ class GeoJSONProvider(BaseProvider):
return data
def get(self, identifier):
def get(self, identifier, **kwargs):
"""
query the provider by id
pygeoapi/provider/mongo.py
+2 -2
View File
@@ -98,7 +98,7 @@ class MongoProvider(BaseProvider):
def query(self, startindex=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
select_properties=[], skip_geometry=False, q=None):
select_properties=[], skip_geometry=False, q=None, **kwargs):
"""
query the provider
@@ -143,7 +143,7 @@ class MongoProvider(BaseProvider):
return feature_collection
def get(self, identifier):
def get(self, identifier, **kwargs):
"""
query the provider by id
pygeoapi/provider/mvt.py
+1 -1
View File
@@ -214,7 +214,7 @@ class MVTProvider(BaseTileProvider):
raise ProviderTileNotFoundError(err)
def get_metadata(self, dataset, server_url, layer=None,
tileset=None, tilejson=True):
tileset=None, tilejson=True, **kwargs):
"""
Gets tile metadata
pygeoapi/provider/ogr.py
+2 -2
View File
@@ -291,7 +291,7 @@ class OGRProvider(BaseProvider):
def query(self, startindex=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
select_properties=[], skip_geometry=False, q=None):
select_properties=[], skip_geometry=False, q=None, **kwargs):
"""
Query OGR source
@@ -372,7 +372,7 @@ class OGRProvider(BaseProvider):
return result
def get(self, identifier):
def get(self, identifier, **kwargs):
"""
Get Feature by id
pygeoapi/provider/postgresql.py
+2 -2
View File
@@ -208,7 +208,7 @@ class PostgreSQLProvider(BaseProvider):
def query(self, startindex=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
select_properties=[], skip_geometry=False, q=None):
select_properties=[], skip_geometry=False, q=None, **kwargs):
"""
Query Postgis for all the content.
e,g: http://localhost:5000/collections/hotosm_bdi_waterways/items?
@@ -330,7 +330,7 @@ class PostgreSQLProvider(BaseProvider):
id_ = item[0]['id'] if item else identifier
return id_
def get(self, identifier):
def get(self, identifier, **kwargs):
"""
Query the provider for a specific
feature id e.g: /collections/hotosm_bdi_waterways/items/13990765
pygeoapi/provider/rasterio_.py
+3 -3
View File
@@ -65,7 +65,7 @@ class RasterioProvider(BaseProvider):
LOGGER.warning(err)
raise ProviderConnectionError(err)
def get_coverage_domainset(self):
def get_coverage_domainset(self, *args, **kwargs):
"""
Provide coverage domainset
:returns: CIS JSON object of domainset metadata
@@ -119,7 +119,7 @@ class RasterioProvider(BaseProvider):
return domainset
def get_coverage_rangetype(self):
def get_coverage_rangetype(self, *args, **kwargs):
"""
Provide coverage rangetype
:returns: CIS JSON object of rangetype metadata
@@ -161,7 +161,7 @@ class RasterioProvider(BaseProvider):
return rangetype
def query(self, range_subset=[], subsets={}, bbox=[], datetime_=None,
format_='json'):
format_='json', **kwargs):
"""
Extract data from collection collection
:param range_subset: list of bands
pygeoapi/provider/sqlite.py
+2 -2
View File
@@ -251,7 +251,7 @@ class SQLiteGPKGProvider(BaseProvider):
def query(self, startindex=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
select_properties=[], skip_geometry=False, q=None):
select_properties=[], skip_geometry=False, q=None, **kwargs):
"""
Query SQLite/GPKG for all the content.
e,g: http://localhost:5000/collections/countries/items?
@@ -310,7 +310,7 @@ class SQLiteGPKGProvider(BaseProvider):
return feature_collection
def get(self, identifier):
def get(self, identifier, **kwargs):
"""
Query the provider for a specific
feature id e.g: /collections/countries/items/1
pygeoapi/provider/tinydb_.py
+2 -2
View File
@@ -93,7 +93,7 @@ class TinyDBCatalogueProvider(BaseProvider):
def query(self, startindex=0, limit=10, resulttype='results',
bbox=[], datetime_=None, properties=[], sortby=[],
select_properties=[], skip_geometry=False, q=None):
select_properties=[], skip_geometry=False, q=None, **kwargs):
"""
query TinyDB document store
@@ -203,7 +203,7 @@ class TinyDBCatalogueProvider(BaseProvider):
return feature_collection
def get(self, identifier):
def get(self, identifier, **kwargs):
"""
Get TinyDB document by id
pygeoapi/provider/xarray_.py
+3 -3
View File
@@ -74,7 +74,7 @@ class XarrayProvider(BaseProvider):
LOGGER.warning(err)
raise ProviderConnectionError(err)
def get_coverage_domainset(self):
def get_coverage_domainset(self, *args, **kwargs):
"""
Provide coverage domainset
@@ -140,7 +140,7 @@ class XarrayProvider(BaseProvider):
return domainset
def get_coverage_rangetype(self):
def get_coverage_rangetype(self, *args, **kwargs):
"""
Provide coverage rangetype
@@ -182,7 +182,7 @@ class XarrayProvider(BaseProvider):
return rangetype
def query(self, range_subset=[], subsets={}, bbox=[], datetime_=None,
format_='json'):
format_='json', **kwargs):
"""
Extract data from collection collection
pygeoapi/starlette_app.py
+69 -189
View File
@@ -75,22 +75,30 @@ if (OGC_SCHEMAS_LOCATION is not None and
api_ = API(CONFIG)
def get_response(result: tuple) -> Response:
""" Creates a Starlette Response object and updates matching headers.
:param result: The result of the API call.
This should be a tuple of (headers, status, content).
:returns: A Response instance.
"""
headers, status, content = result
response = Response(content=content, status_code=status)
if headers is not None:
response.headers.update(headers)
return response
@app.route('/')
async def landing_page(request: Request):
"""
OGC API landing page endpoint
:param request: Starlette Request instance
:returns: Starlette HTTP Response
"""
headers, status_code, content = api_.landing_page(
request.headers, request.query_params)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.landing_page(request))
@app.route('/openapi')
@@ -99,24 +107,17 @@ async def openapi(request: Request):
"""
OpenAPI endpoint
:param request: Starlette Request instance
:returns: Starlette HTTP Response
"""
with open(os.environ.get('PYGEOAPI_OPENAPI'), encoding='utf8') as ff:
if os.environ.get('PYGEOAPI_OPENAPI').endswith(('.yaml', '.yml')):
openapi = yaml_load(ff)
openapi_ = yaml_load(ff)
else: # JSON file, do not transform
openapi = ff
openapi_ = ff
headers, status_code, content = api_.openapi(
request.headers, request.query_params, openapi)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.openapi(request, openapi_))
@app.route('/conformance')
@@ -125,17 +126,11 @@ async def conformance(request: Request):
"""
OGC API conformance endpoint
:param request: Starlette Request instance
:returns: Starlette HTTP Response
"""
headers, status_code, content = api_.conformance(
request.headers, request.query_params)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.conformance(request))
@app.route('/collections')
@@ -146,21 +141,14 @@ async def collections(request: Request, collection_id=None):
"""
OGC API collections endpoint
:param request: Starlette Request instance
:param collection_id: collection identifier
:returns: Starlette HTTP Response
"""
if 'collection_id' in request.path_params:
collection_id = request.path_params['collection_id']
headers, status_code, content = api_.describe_collections(
request.headers, request.query_params, collection_id)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.describe_collections(request, collection_id))
@app.route('/collections/{collection_id}/queryables')
@@ -169,21 +157,14 @@ async def collection_queryables(request: Request, collection_id=None):
"""
OGC API collections queryables endpoint
:param request: Starlette Request instance
:param collection_id: collection identifier
:returns: Starlette HTTP Response
"""
if 'collection_id' in request.path_params:
collection_id = request.path_params['collection_id']
headers, status_code, content = api_.get_collection_queryables(
request.headers, request.query_params, collection_id)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_collection_queryables(request, collection_id))
@app.route('/collections/{name}/tiles')
@@ -192,21 +173,14 @@ async def get_collection_tiles(request: Request, name=None):
"""
OGC open api collections tiles access point
:param request: Starlette Request instance
:param name: identifier of collection name
:returns: Starlette HTTP Response
"""
if 'name' in request.path_params:
name = request.path_params['name']
headers, status_code, content = api_.get_collection_tiles(
request.headers, request.query_params, name)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_collection_tiles(request, name))
@app.route('/collections/{name}/tiles/\
@@ -219,6 +193,7 @@ async def get_collection_items_tiles(request: Request, name=None,
"""
OGC open api collection tiles service
:param request: Starlette Request instance
:param name: identifier of collection name
:param tileMatrixSetId: identifier of tile matrix set
:param tile_matrix: identifier of {z} matrix index
@@ -227,7 +202,6 @@ async def get_collection_items_tiles(request: Request, name=None,
:returns: HTTP response
"""
if 'name' in request.path_params:
name = request.path_params['name']
if 'tileMatrixSetId' in request.path_params:
@@ -238,15 +212,8 @@ async def get_collection_items_tiles(request: Request, name=None,
tileRow = request.path_params['tileRow']
if 'tileCol' in request.path_params:
tileCol = request.path_params['tileCol']
headers, status_code, content = api_.get_collection_items_tiles(
request.headers, request.query_params, name, tileMatrixSetId,
tile_matrix, tileRow, tileCol)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_collection_tiles_data(
request, name, tileMatrixSetId, tile_matrix, tileRow, tileCol))
@app.route('/collections/{collection_id}/items')
@@ -257,30 +224,22 @@ async def collection_items(request: Request, collection_id=None, item_id=None):
"""
OGC API collections items endpoint
:param request: Starlette Request instance
:param collection_id: collection identifier
:param item_id: item identifier
:returns: Starlette HTTP Response
"""
if 'collection_id' in request.path_params:
collection_id = request.path_params['collection_id']
if 'item_id' in request.path_params:
item_id = request.path_params['item_id']
if item_id is None:
headers, status_code, content = api_.get_collection_items(
request.headers, request.query_params,
collection_id, pathinfo=request.scope['path'])
return get_response(api_.get_collection_items(
request, collection_id, pathinfo=request.scope['path']))
else:
headers, status_code, content = api_.get_collection_item(
request.headers, request.query_params, collection_id, item_id)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_collection_item(
request, collection_id, item_id))
@app.route('/collections/{collection_id}/coverage')
@@ -288,23 +247,15 @@ async def collection_coverage(request: Request, collection_id):
"""
OGC API - Coverages coverage endpoint
:param request: Starlette Request instance
:param collection_id: collection identifier
:returns: Starlette HTTP Response
"""
if 'collection_id' in request.path_params:
collection_id = request.path_params['collection_id']
headers, status_code, content = api_.get_collection_coverage(
request.headers, request.query_params, collection_id)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_collection_coverage(request, collection_id))
@app.route('/collections/{collection_id}/coverage/domainset')
@@ -312,23 +263,16 @@ async def collection_coverage_domainset(request: Request, collection_id):
"""
OGC API - Coverages coverage domainset endpoint
:param request: Starlette Request instance
:param collection_id: collection identifier
:returns: Starlette HTTP Response
"""
if 'collection_id' in request.path_params:
collection_id = request.path_params['collection_id']
headers, status_code, content = api_.get_collection_coverage_domainset(
request.headers, request.query_params, collection_id)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_collection_coverage_domainset(
request, collection_id))
@app.route('/collections/{collection_id}/coverage/rangetype')
@@ -336,23 +280,16 @@ async def collection_coverage_rangetype(request: Request, collection_id):
"""
OGC API - Coverages coverage rangetype endpoint
:param request: Starlette Request instance
:param collection_id: collection identifier
:returns: Starlette HTTP Response
"""
if 'collection_id' in request.path_params:
collection_id = request.path_params['collection_id']
headers, status_code, content = api_.get_collection_coverage_rangetype(
request.headers, request.query_params, collection_id)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_collection_coverage_rangetype(
request, collection_id))
@app.route('/processes')
@@ -363,23 +300,15 @@ async def get_processes(request: Request, process_id=None):
"""
OGC API - Processes description endpoint
:param request: Starlette Request instance
:param process_id: identifier of process to describe
:returns: Starlette HTTP Response
"""
if 'process_id' in request.path_params:
process_id = request.path_params['process_id']
headers, status_code, content = api_.describe_processes(
request.headers, request.query_params, process_id)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.describe_processes(request, process_id))
@app.route('/processes/{process_id}/jobs', methods=['GET', 'POST'])
@@ -390,6 +319,7 @@ async def get_process_jobs(request: Request, process_id=None, job_id=None):
"""
OGC API - Processes jobs endpoint
:param request: Starlette Request instance
:param process_id: process identifier
:param job_id: job identifier
@@ -403,27 +333,15 @@ async def get_process_jobs(request: Request, process_id=None, job_id=None):
if job_id is None: # list of submit job
if request.method == 'GET':
headers, status_code, content = api_.get_process_jobs(
request.headers, request.query_params, process_id)
return get_response(api_.get_process_jobs(request, process_id))
elif request.method == 'POST':
request_body = await request.body()
headers, status_code, content = api_.execute_process(
request.headers, request.query_params, request_body,
process_id)
return get_response(api_.execute_process(request, process_id))
else: # get or delete job
if request.method == 'DELETE':
headers, status_code, content = api_.delete_process_job(
process_id, job_id)
return get_response(api_.delete_process_job(process_id, job_id))
else: # Return status of a specific job
headers, status_code, content = api_.get_process_job_status(
request.headers, request.args, process_id, job_id)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_process_jobs(
request, process_id, job_id))
@app.route('/processes/{process_id}/jobs/{job_id}/results', methods=['GET'])
@@ -433,6 +351,7 @@ async def get_process_job_result(request: Request, process_id=None,
"""
OGC API - Processes job result endpoint
:param request: Starlette Request instance
:param process_id: process identifier
:param job_id: job identifier
@@ -444,15 +363,8 @@ async def get_process_job_result(request: Request, process_id=None,
if 'job_id' in request.path_params:
job_id = request.path_params['job_id']
headers, status_code, content = api_.get_process_job_result(
request.headers, request.args, process_id, job_id)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_process_job_result(
request, process_id, job_id))
@app.route('/processes/{process_id}/jobs/{job_id}/results/{resource}',
@@ -464,6 +376,7 @@ async def get_process_job_result_resource(request: Request, process_id=None,
"""
OGC API - Processes job result resource endpoint
:param request: Starlette Request instance
:param process_id: process identifier
:param job_id: job identifier
:param resource: job resource
@@ -478,15 +391,8 @@ async def get_process_job_result_resource(request: Request, process_id=None,
if 'resource' in request.path_params:
resource = request.path_params['resource']
headers, status_code, content = api_.get_process_job_result_resource(
request.headers, request.args, process_id, job_id, resource)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_process_job_result_resource(
request, process_id, job_id, resource))
@app.route('/collections/{collection_id}/position')
@@ -515,18 +421,9 @@ async def get_collection_edr_query(request: Request, collection_id=None, instanc
if 'instance_id' in request.path_params:
instance_id = request.path_params['instance_id']
query_type = request.path.split('/')[-1]
headers, status_code, content = api_.get_collection_edr_query(
request.headers, request.query_params, collection_id, instance_id,
query_type)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
query_type = request.path.split('/')[-1] # noqa
return get_response(api_.get_collection_edr_query(request, collection_id,
instance_id, query_type))
@app.route('/stac')
@@ -534,18 +431,11 @@ async def stac_catalog_root(request: Request):
"""
STAC root endpoint
:param request: Starlette Request instance
:returns: Starlette HTTP response
"""
headers, status_code, content = api_.get_stac_root(
request.headers, request.query_params)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_stac_root(request))
@app.route('/stac/{path:path}')
@@ -553,22 +443,12 @@ async def stac_catalog_path(request: Request):
"""
STAC endpoint
:param path: path
:param request: Starlette Request instance
:returns: Starlette HTTP response
"""
path = request.path_params["path"]
headers, status_code, content = api_.get_stac_path(
request.headers, request.query_params, path)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
return get_response(api_.get_stac_path(request, path))
@click.command()
pygeoapi/util.py
+15 -5
View File
@@ -43,11 +43,13 @@ from urllib.request import urlopen
from urllib.parse import urlparse
import dateutil.parser
# from babel.support import Translations
from jinja2 import Environment, FileSystemLoader
from jinja2.exceptions import TemplateNotFound
import yaml
from pygeoapi import __version__
from pygeoapi import l10n
from pygeoapi.provider.base import ProviderTypeError
LOGGER = logging.getLogger(__name__)
@@ -277,6 +279,8 @@ def json_serial(obj):
return base64.b64encode(obj)
elif isinstance(obj, Decimal):
return float(obj)
elif isinstance(obj, l10n.Locale):
return l10n.locale2str(obj)
msg = '{} type {} not serializable'.format(obj, type(obj))
LOGGER.error(msg)
@@ -298,13 +302,14 @@ def is_url(urlstring):
return False
def render_j2_template(config, template, data):
def render_j2_template(config, template, data, locale_=None):
"""
render Jinja2 template
:param config: dict of configuration
:param template: template (relative path)
:param data: dict of data
:param locale_: the requested output Locale
:returns: string of rendered template
"""
@@ -312,11 +317,13 @@ def render_j2_template(config, template, data):
custom_templates = False
try:
templates_path = config['server']['templates']['path']
env = Environment(loader=FileSystemLoader(templates_path))
env = Environment(loader=FileSystemLoader(templates_path),
extensions=['jinja2.ext.i18n'])
custom_templates = True
LOGGER.debug('using custom templates: {}'.format(templates_path))
except (KeyError, TypeError):
env = Environment(loader=FileSystemLoader(TEMPLATES))
env = Environment(loader=FileSystemLoader(TEMPLATES),
extensions=['jinja2.ext.i18n'])
LOGGER.debug('using default templates: {}'.format(TEMPLATES))
env.filters['to_json'] = to_json
@@ -334,18 +341,21 @@ def render_j2_template(config, template, data):
env.filters['filter_dict_by_key_value'] = filter_dict_by_key_value
env.globals.update(filter_dict_by_key_value=filter_dict_by_key_value)
# TODO: insert Babel Translation stuff here
try:
template = env.get_template(template)
except TemplateNotFound as err:
if custom_templates:
LOGGER.debug(err)
LOGGER.debug('Custom template not found; using default')
env = Environment(loader=FileSystemLoader(TEMPLATES))
env = Environment(loader=FileSystemLoader(TEMPLATES),
extensions=['jinja2.ext.i18n'])
template = env.get_template(template)
else:
raise
return template.render(config=config, data=data, version=__version__)
return template.render(config=l10n.translate_struct(config, locale_, True),
data=data, version=__version__)
def get_mimetype(filename):
requirements.txt
+1
View File
@@ -8,3 +8,4 @@ rasterio
shapely
tinydb
unicodecsv
Babel
tests/pygeoapi-test-config.yml
+30 -10
View File
@@ -34,7 +34,10 @@ server:
url: http://localhost:5000/
mimetype: application/json; charset=UTF-8
encoding: utf-8
language: en-US
languages:
# First language is the default language
- en-US
- fr-CA
cors: true
pretty_print: true
limit: 10
@@ -52,12 +55,21 @@ logging:
metadata:
identification:
title: pygeoapi default instance
description: pygeoapi provides an API to geospatial data
title:
en: pygeoapi default instance
fr: instance par défaut de pygeoapi
description:
en: pygeoapi provides an API to geospatial data
fr: pygeoapi fournit une API aux données géospatiales
keywords:
- geospatial
- data
- api
en:
- geospatial
- data
- api
fr:
- géospatiale
- données
- api
keywords_type: theme
terms_of_service: https://creativecommons.org/licenses/by/4.0/
url: http://example.org
@@ -86,8 +98,12 @@ metadata:
resources:
obs:
type: collection
title: Observations
description: My cool observations
title:
en: Observations
fr: Observations
description:
en: My cool observations
fr: Mes belles observations
keywords:
- observations
- monitoring
@@ -160,8 +176,12 @@ resources:
lakes:
type: collection
title: Large Lakes
description: lakes of the world, public domain
title:
en: Large Lakes
fr: Grands Lacs
description:
en: lakes of the world, public domain
fr: lacs du monde, domaine public
keywords:
- lakes
links:
tests/test_api.py
+483 -362
View File
File diff suppressed because it is too large Load Diff
tests/test_l10n.py
+307
View File
@@ -0,0 +1,307 @@
# =================================================================
#
# Authors: Sander Schaminee <sander.schaminee@geocat.net>
#
# Copyright (c) 2021 GeoCat BV
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
# files (the "Software"), to deal in the Software without
# restriction, including without limitation the rights to use,
# copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following
# conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
# =================================================================
import os
from babel import Locale
from pygeoapi import l10n
from pygeoapi.util import yaml_load
import pytest
def test_str2locale():
us_locale = Locale.parse('en_US')
assert l10n.str2locale('en') == Locale.parse('en')
assert l10n.str2locale('en_US') == us_locale
assert l10n.str2locale('en-US') == us_locale
assert l10n.str2locale('eng_CA') == Locale.parse('en_CA')
assert l10n.str2locale(' fr-CH ') == Locale.parse('fr_CH')
assert l10n.str2locale(us_locale) is us_locale
assert l10n.str2locale(None, True) is None
assert l10n.str2locale(42, True) is None
assert l10n.str2locale('is_BS', True) is None
with pytest.raises(l10n.LocaleError):
for v in ('', None, 1, 42.0, 'is_BS', 'eng;CAN'):
l10n.str2locale(v)
def test_locale2str():
assert l10n.locale2str(Locale.parse('en_US')) == 'en-US'
assert l10n.locale2str(Locale.parse('fr')) == 'fr'
with pytest.raises(l10n.LocaleError):
for v in (None, 1, 42.0, 'is_BS', object()):
l10n.locale2str(v) # noqa
def test_bestmatch():
assert l10n.best_match('de', ('en',)) == Locale('en')
assert l10n.best_match(None, ['en', 'de']) == Locale('en') # noqa
assert l10n.best_match('', ['en', 'de']) == Locale('en')
assert l10n.best_match('de-DE', ['en', 'de']) == Locale('de')
assert l10n.best_match('de-DE, en', ['en', 'de']) == Locale('de')
assert l10n.best_match('de, en', ['en_US', 'de-DE']) == Locale.parse('de_DE') # noqa
assert l10n.best_match(Locale('de'), ['nl', 'de']) == Locale('de')
accept = "fr-CH, fr;q=0.9, en;q=0.8, de;q=0.7, *;q=0.5"
assert l10n.best_match(accept, ['fr', 'en']) == Locale('fr')
assert l10n.best_match(accept, ['it', 'de']) == Locale('de')
assert l10n.best_match(accept, ['fr-BE', 'fr']) == Locale('fr')
assert l10n.best_match(accept, ['fr-BE', 'fr-FR']) == Locale.parse('fr_BE')
assert l10n.best_match(accept, ['fr-BE', 'fr-FR']) == Locale.parse('fr_BE')
assert l10n.best_match(accept, ['it', 'es']) == Locale('it')
assert l10n.best_match(accept, ['it', 'es']) == Locale('it')
assert l10n.best_match(accept, ('it', 'es')) == Locale('it')
with pytest.raises(l10n.LocaleError):
l10n.best_match(accept, [])
l10n.best_match(accept, None)
l10n.best_match(accept, 42)
l10n.best_match(accept, ['is_BS'])
@pytest.fixture()
def language_struct():
return {k: Locale.parse(k).display_name for k in (
'en', 'fr', 'en_US', 'fr_BE', 'alb', 'nl_BE'
)}
@pytest.fixture()
def nonlanguage_struct():
return {
None: 'empty key',
42: 'numeric key',
'fla': 'non-language key'
}
def test_translate(language_struct, nonlanguage_struct):
assert l10n.translate({}, 'en-US') == {}
assert l10n.translate(42, 'fr') == 42
assert l10n.translate(None, 'de') is None
assert l10n.translate(['list item'], Locale('en')) == ['list item']
assert l10n.translate({'nested dict': {'en': 1, 'fr': 2}}, 'en') == {'nested dict': {'en': 1, 'fr': 2}} # noqa
assert l10n.translate(nonlanguage_struct, 'fr') == nonlanguage_struct
assert l10n.translate(nonlanguage_struct, 'fla') == 'non-language key'
assert l10n.translate(language_struct, 'en') == 'English'
assert l10n.translate(language_struct, 'en-US') == 'English (United States)' # noqa
assert l10n.translate(language_struct, 'sq_AL') == Locale.parse('alb').display_name # noqa
assert l10n.translate(language_struct, 'fr_CH') == Locale.parse('fr').display_name # noqa
assert l10n.translate(language_struct, 'nl') == Locale.parse('nl_BE').display_name # noqa
assert l10n.translate(language_struct, 'de') == 'English'
assert l10n.translate(language_struct, Locale('en')) == 'English'
assert l10n.translate(language_struct, Locale.parse('en_US')) == 'English (United States)' # noqa
with pytest.raises(l10n.LocaleError):
l10n.translate(language_struct, None) # noqa
l10n.translate(language_struct, 42) # noqa
def test_localefromheaders():
assert l10n.locale_from_headers({}) is None
assert l10n.locale_from_headers({'Accept-Language': 'de'}) == 'de'
assert l10n.locale_from_headers({'accept-language': 'en_US'}) == 'en_US'
def test_localefromparams():
assert l10n.locale_from_params({}) is None
assert l10n.locale_from_params({'lang': 'de'}) == 'de'
assert l10n.locale_from_params({'language': 'en_US'}) is None
assert l10n.locale_from_params({'lang': 'en_US'}) == 'en_US'
def test_addlocale():
assert l10n.add_locale('http://a.pi/', None) == 'http://a.pi/'
assert l10n.add_locale('http://a.pi/', 'en') == 'http://a.pi/?lang=en'
assert l10n.add_locale('http://a.pi', 'de_CH') == 'http://a.pi?lang=de-CH'
assert l10n.add_locale('http://a.pi', 'zz') == 'http://a.pi'
assert l10n.add_locale('http://a.pi?q=1', 'nl') == 'http://a.pi?q=1&lang=nl' # noqa
assert l10n.add_locale('http://a.pi?lang=de', 'nl') == 'http://a.pi?lang=nl' # noqa
def test_getlocales():
config = {
'server': {
'language': ''
}
}
with pytest.raises(l10n.LocaleError):
l10n.get_locales({})
l10n.get_locales(config)
config['server']['language'] = 'zz'
l10n.get_locales(config)
config['server']['language'] = 'en-US'
assert l10n.get_locales(config) == [Locale.parse('en_US')]
config['server']['language'] = 'de_CH'
assert l10n.get_locales(config) == [Locale.parse('de_CH')]
config['server']['language'] = ['de', 'en-US'] # noqa
assert l10n.get_locales(config) == [Locale.parse('de'), Locale.parse('en_US')] # noqa
config = {
'server': {
'languages': []
}
}
with pytest.raises(l10n.LocaleError):
l10n.get_locales(config)
config['server']['languages'] = [None]
with pytest.raises(l10n.LocaleError):
l10n.get_locales(config)
config['server']['languages'] = ['de', 'en-US']
assert l10n.get_locales(config) == [Locale.parse('de'), Locale.parse('en_US')] # noqa
def test_getpluginlocale():
assert l10n.get_plugin_locale({}, 'de') is None
assert l10n.get_plugin_locale({}, None) is None # noqa
assert l10n.get_plugin_locale({}, '') is None
assert l10n.get_plugin_locale({'language': 'de'}, 'en') == Locale('de')
assert l10n.get_plugin_locale({'language': None}, 'en') is None
assert l10n.get_plugin_locale({'languages': ['en']}, None) == Locale('en') # noqa
assert l10n.get_plugin_locale({'languages': []}, 'nl') is None
assert l10n.get_plugin_locale({'languages': ['en']}, 'fr') == Locale('en')
assert l10n.get_plugin_locale({'languages': ['en', 'de']}, 'de') == Locale('de') # noqa
assert l10n.get_plugin_locale({'languages': ['en', 'de']}, None) == Locale('en') # noqa
def test_setresponselanguage():
# the following should not raise (only logs warning)
l10n.set_response_language(None, None) # noqa
headers = {}
with pytest.raises(l10n.LocaleError):
l10n.set_response_language(headers, None) # noqa
l10n.set_response_language(headers, None, None) # noqa
l10n.set_response_language(headers, None, 'rubbish') # noqa
l10n.set_response_language(headers, Locale('en'))
assert headers['Content-Language'] == 'en'
l10n.set_response_language(headers, Locale('de'))
assert headers['Content-Language'] == 'de'
l10n.set_response_language(headers, Locale('de'), Locale('en', 'US'))
assert headers['Content-Language'] == 'de, en-US'
l10n.set_response_language(headers, Locale('en'), Locale('en'))
assert headers['Content-Language'] == 'en'
def get_test_file_path(filename):
"""helper function to open test file safely"""
if os.path.isfile(filename):
return filename
else:
return 'tests/{}'.format(filename)
@pytest.fixture()
def config():
with open(get_test_file_path('pygeoapi-test-config.yml')) as fh:
return yaml_load(fh)
@pytest.fixture()
def locale_():
return Locale.parse('en_US')
def test_translatedict(config, locale_):
cfg = l10n.translate_struct(config, locale_, True)
assert cfg['metadata']['identification']['title'] == 'pygeoapi default instance' # noqa
assert cfg['metadata']['identification']['keywords'] == ['geospatial', 'data', 'api'] # noqa
# test full equality (must come from cache)
cfg2 = l10n.translate_struct(config, locale_, True)
assert cfg is cfg2
# missing locale_ should return the same dict
assert l10n.translate_struct(config, None) is config # noqa
# missing or empty dict should return an empty dict
assert l10n.translate_struct(None, locale_) == {} # noqa
# test custom dict (translate from level 0, do not cache)
test_dict = {
'level0': {
'en': 'test value',
'fr': 'valeur de test'
}
}
tr_dict = l10n.translate_struct(test_dict, locale_)
assert tr_dict['level0'] == 'test value'
tr_dict2 = l10n.translate_struct(test_dict, locale_)
assert tr_dict == tr_dict2
assert tr_dict is not tr_dict2
# test mixed structure
test_input = [
{'test': {
'en': 'test value',
'fr': 'valeur de test'
}},
'some string',
{'item1': 1},
{'item2a': [
'list_item1',
'list_item2',
{
'en': 'list value',
'fr': 'valeur de liste'
}
],
'item2b': {
'en': 'test value',
'fr': 'valeur de test'
}}
]
test_output = [
{'test': 'test value'},
'some string',
{'item1': 1},
{'item2a': [
'list_item1',
'list_item2',
'list value'
],
'item2b': 'test value'
}
]
assert l10n.translate_struct(test_input, locale_) == test_output