huanld/pygeoapi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add support for Starlette (#223)

Browse Source 
* Add starlette app with base route

* Add api route

* Add conformance route

* Add route for collections

* Fix flake8 errors

* Add route for collection items

* Add route for processes

* Add docs for starlette

* Add support for starlette

Add api route


Add conformance route


Add route for collections


Fix flake8 errors


Add route for collection items


Add route for processes


Add docs for starlette

* Make starlette deps into a separate requirements file 


Move imports

* Add comment for starlette requirements

* Fix missing docstring param

* Improve installation docs
This commit is contained in:
Francesco Bartoli
2019-09-06 20:25:28 +02:00
committed by Tom Kralidis
parent 4bd884e564
commit ab12cbfc92
11 changed files with 340 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1
View File
@@ -103,6 +103,7 @@ ENV/
# pygeoapi artifacts
local.config.yml
local.openapi.yml
openapi.yml
# misc
*.swp
README.md
+1
View File
@@ -15,6 +15,7 @@ cd pygeoapi
pip install -r requirements.txt
pip install -r requirements-dev.txt
# install provider requirements accordingly from requirements-provider.txt
# install starlette requirements accordingly from requirements-starlette.txt
pip install -e .
cp pygeoapi-config.yml local.config.yml
vi local.config.yml
docs/source/asgi.rst
+42
View File
@@ -0,0 +1,42 @@
.. _asgi:
ASGI
====
Asynchronous Server Gateway Interface (ASGI) is standard interface between async-capable web servers, frameworks, and applications written on Python language. pygeoapi itself
doesn't implement ASGI since it is an API, therefore it is required a webframework to access HTTP requests and pass the information to pygeoapi
.. code-block:: console
HTTP request --> Starlette (starlette_app.py) --> pygeoapi API
the pygeoapi package integrates `starlette_app <https://www.starlette.io/>`_ as webframework for defining the API routes/end points and WSGI support.
The starlette ASGI server can be easily run as a pygeoapi command with the option `--starlette`:
.. code-block:: console
pygeoapi serve --starlette
Running a Uvicorn server is not advisable, the preferred option is as follows:
.. code-block:: console
HTTP request --> ASGI server (gunicorn) --> Starlette (starlette_app.py) --> pygeoapi API
By having a specific ASGI server, the HTTPS are efficiently processed into threads/processes. The current docker pygeoapi
implement such strategy (see section: :ref:`docker`), it is prefered to implement pygeopai using docker solutions than running host native ASGI servers.
Running gunicorn
----------------
Uvicorn includes a Gunicorn worker class allowing you to run ASGI applications, with all of Uvicorn's performance benefits, while also giving you Gunicorn's fully-featured process management. This server
is simple to run from the command, e.g:
.. code-block:: console
gunicorn pygeoapi.starlette_app:app -w 4 -k uvicorn.workers.UvicornWorker
For extra configuration parameters like port binding, workers please consult the gunicorn `settings <http://docs.gunicorn.org/en/stable/settings.html>`_
docs/source/index.rst
+1
View File
@@ -17,6 +17,7 @@ Welcome to pygeoapi's documentation!
openapi
docker
wsgi
asgi
configuration
plugins
code
docs/source/install.rst
+17 -12
View File
@@ -3,18 +3,18 @@
Install
=======
pygeoapi is nativally run as a Flask app (the code struct is an API and Flask is used as a wrapper).
pygeoapi, by default, is natively run as a Flask app (the code struct is an API and Flask is used as a wrapper). Optionally it can be run as a Starlette app.
pygeoapi uses two configuration files: **pygeoapi-config.yml** and **openapi.yml**. First configuration contains all the information and setup to run pygeoapi while the second is exclusivally for the openapi.
pygeoapi requires setting ``PYGEOAPI_CONFIG`` and ``PYGEOAPI_OPENAPI`` env variable. ``PYGEOAPI_CONFIG`` points to the yaml file containing the configuration, in the example
bellow we copy the ``local.config.yml`` default configuration to ``pygeoapi-config.yml`` and use this configuration file.
below we copy the ``local.config.yml`` default configuration to ``pygeoapi-config.yml`` and use this configuration file.
``PYGEOAPI_OPENAPI`` variable is the path to openapi file configuration, this file **needs to be autogenerated** using the ``pygeoapi generate-openapi-document`` command and
the pygeoapi config files e.g: ``pygeoapi generate-openapi-document -c local.config.yml > openapi.yml``. And then setting the env variable to the path:
the pygeoapi config files e.g.: ``pygeoapi generate-openapi-document -c local.config.yml > openapi.yml``. And then setting the env variable to the path:
``export PYGEOAPI_OPENAPI=/path/to/openapi.yml``
For production environments it is recommended to use :ref:`docker` or a specialized WSGI HTTP server :ref:`wsgi`
For production environments it is recommended to use :ref:`docker` or specialized servers like WSGI HTTP server :ref:`wsgi` or ASGI HTTP server :ref:`asgi`
Copy/paste install
------------------
@@ -23,32 +23,37 @@ It is advisable to run pygeoapi inside an isolated enviroment either *virtualenv
.. code-block:: console
#create virtualenv
# create virtualenv
virtualenv -p python pygeoapi
cd pygeoapi
. bin/activate
git clone https://github.com/geopython/pygeoapi.git
cd pygeoapi
#install requirements
# install requirements
pip install -r requirements.txt
pip install -r requirements-dev.txt
# optionally install requirements for starlette
pip install -r requirements-starlette.txt
# install source in current directory
pip install -e .
cp pygeoapi-config.yml local.config.yml
#edit configuration file
# edit configuration file
nano local.config.yml
export PYGEOAPI_CONFIG=/path/to/local.config.yml
# generate OpenAPI Document
pygeoapi generate-openapi-document -c local.config.yml > openapi.yml
export PYGEOAPI_OPENAPI=/path/to/openapi.yml
#Run pygeoapi
# run pygeoapi
pygeoapi serve
# optionally run pygeoapi with starlette
pygeoapi serve --starlette
If the default configuration was used then we should have pygeoapi running on
If the default configuration was used then we should have pygeoapi running on
.. image:: /_static/openapi_intro_page.png
docs/source/wsgi.rst
+5 -5
View File
@@ -5,7 +5,7 @@ WSGI
Web Server Gateway Interface (WSGI) is standard for forwarding request to web applications written on Python language. pygeoapi it self
doesn't implement WSGI since it is an API,
therefore it is required a webframework to access HTTP requests and pass the information to pygeopai
therefore it is required a webframework to access HTTP requests and pass the information to pygeoapi
.. code-block:: console
@@ -14,17 +14,17 @@ WSGI
the pygeoapi package integrates `Flask <https://flask.palletsprojects.com/en/1.1.x/>`_ as webframework for defining the API routes/end points and WSGI support.
The flask WSGI server can be easely run as a pygeoapi command:
The flask WSGI server can be easily run as a pygeoapi command with the option `--flask`:
.. code-block:: console
pygeoapi serve
pygeoapi serve --flask
Running a native Flask server is not adivsable, the prefered option is as follows:
Running a native Flask server is not advisable, the prefered option is as follows:
.. code-block:: console
HTTP request --> WSGI server (gunicorn) --> Flask (flask_app.py) --> pygeopai API
HTTP request --> WSGI server (gunicorn) --> Flask (flask_app.py) --> pygeoapi API
By having a specific WSGI server, the HTTPS are efficiently processed into threads/processes. The current docker pygeoapi
implement such strategy (see section: :ref:`docker`), it is prefered to implement pygeopai using docker solutions than running host native WSGI servers.
pygeoapi/__init__.py
+26 -6
View File
@@ -30,15 +30,35 @@
__version__ = '0.6.0'
import click
from pygeoapi.flask_app import serve
from pygeoapi.openapi import generate_openapi_document
@click.group()
@click.version_option(version=__version__)
def cli():
pass
cli = click.Group()
cli.version = __version__
@cli.command()
@click.option('--flask', 'server', flag_value="flask", default=True)
@click.option('--starlette', 'server', flag_value="starlette")
@click.pass_context
def serve(ctx, server):
"""Run the server with different daemon type
:param server: `string` of server type
:returns void
"""
if server == "flask":
from pygeoapi.flask_app import serve as serve_flask
ctx.forward(serve_flask)
ctx.invoke(serve_flask)
elif server == "starlette":
from pygeoapi.starlette_app import serve as serve_starlette
ctx.forward(serve_starlette)
ctx.invoke(serve_starlette)
else:
raise click.ClickException('--flask/--starlette is required')
cli.add_command(serve)
cli.add_command(generate_openapi_document)
pygeoapi/api.py
+10 -4
View File
@@ -368,13 +368,14 @@ class API(object):
return headers_, 200, json.dumps(fcm)
def get_features(self, headers, args, dataset):
def get_features(self, headers, args, dataset, pathinfo=None):
"""
Queries feature collection
:param headers: dict of HTTP headers
:param args: dict of HTTP request parameters
:param dataset: dataset name
:param pathinfo: path location
:returns: tuple of headers, status code, content
"""
@@ -559,9 +560,14 @@ class API(object):
content['links'][1]['rel'] = 'self'
# For constructing proper URIs to items
path_info = '/'.join([
self.config['server']['url'].rstrip('/'),
headers.environ['PATH_INFO'].strip('/')])
if pathinfo:
path_info = '/'.join([
self.config['server']['url'].rstrip('/'),
pathinfo.strip('/')])
else:
path_info = '/'.join([
self.config['server']['url'].rstrip('/'),
headers.environ['PATH_INFO'].strip('/')])
content['items_path'] = path_info
content['dataset_path'] = '/'.join(path_info.split('/')[:-1])
pygeoapi/flask_app.py
+3 -2
View File
@@ -64,7 +64,7 @@ api_ = API(CONFIG)
@APP.route('/')
def root():
"""
HTTP root content of pygeopai. Intro page access point
HTTP root content of pygeoapi. Intro page access point
:returns: HTTP response
"""
@@ -205,11 +205,12 @@ def execute_process(name=None):
@click.command()
@click.pass_context
@click.option('--debug', '-d', default=False, is_flag=True, help='debug')
def serve(ctx, debug=False):
def serve(ctx, server, debug=False):
"""
Serve pygeoapi via Flask. Runs pygeoapi
as a flask server. Not recommend for production.
:param server: `string` of server type
:param debug: `bool` of whether to run in debug mode
:returns void
pygeoapi/starlette_app.py
+232
View File
@@ -0,0 +1,232 @@
# =================================================================
#
# Authors: Francesco Bartoli <xbartolone@gmail.com>
#
#
# Copyright (c) 2019 Francesco Bartoli
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
# files (the "Software"), to deal in the Software without
# restriction, including without limitation the rights to use,
# copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following
# conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
# =================================================================
""" Starlette module providing the route paths to the api"""
import os
import click
from starlette.applications import Starlette
from starlette.requests import Request
from starlette.responses import Response
import uvicorn
from pygeoapi.api import API
from pygeoapi.util import yaml_load
app = Starlette()
CONFIG = None
if 'PYGEOAPI_CONFIG' not in os.environ:
raise RuntimeError('PYGEOAPI_CONFIG environment variable not set')
with open(os.environ.get('PYGEOAPI_CONFIG')) as fh:
CONFIG = yaml_load(fh)
# CORS: optionally enable from config.
if CONFIG['server'].get('cors', False):
from starlette.middleware.cors import CORSMiddleware
app.add_middleware(CORSMiddleware, allow_origins=['*'])
api_ = API(CONFIG)
@app.route('/')
async def root(request: Request):
"""
HTTP root content of pygeoapi. Intro page access point
:returns: Starlette HTTP Response
"""
headers, status_code, content = api_.root(
request.headers, request.query_params)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
@app.route('/api')
async def api(request: Request):
"""
OpenAPI access point
:returns: Starlette HTTP Response
"""
with open(os.environ.get('PYGEOAPI_OPENAPI')) as ff:
openapi = yaml_load(ff)
headers, status_code, content = api_.api(
request.headers, request.query_params, openapi)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
@app.route('/conformance')
async def api_conformance(request: Request):
"""
OGC open api conformance access point
:returns: Starlette HTTP Response
"""
headers, status_code, content = api_.api_conformance(
request.headers, request.query_params)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
@app.route('/collections')
@app.route('/collections/{name}')
async def describe_collections(request: Request, name=None):
"""
OGC open api collections access point
:param name: identifier of collection name
:returns: Starlette HTTP Response
"""
if 'name' in request.path_params:
name = request.path_params['name']
headers, status_code, content = api_.describe_collections(
request.headers, request.query_params, name)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
@app.route('/collections/{feature_collection}/items')
@app.route('/collections/{feature_collection}/items/{feature}')
async def dataset(request: Request, feature_collection=None, feature=None):
"""
OGC open api collections/{dataset}/items/{feature} access point
:returns: Starlette HTTP Response
"""
if 'feature_collection' in request.path_params:
feature_collection = request.path_params['feature_collection']
if 'feature' in request.path_params:
feature = request.path_params['feature']
if feature is None:
headers, status_code, content = api_.get_features(
request.headers, request.query_params,
feature_collection, pathinfo=request.scope['path'])
else:
headers, status_code, content = api_.get_feature(
request.headers, request.query_params, feature_collection, feature)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
@app.route('/processes')
@app.route('/processes/{name}')
async def describe_processes(request: Request, name=None):
"""
OGC open api processes access point (experimental)
:param name: identifier of process to describe
:returns: Starlette HTTP Response
"""
headers, status_code, content = api_.describe_processes(
request.headers, request.query_params, name)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
@app.route('/processes/{name}/jobs', methods=['GET', 'POST'])
async def execute_process(request: Request, name=None):
"""
OGC open api jobs from processes access point (experimental)
:param name: identifier of process to execute
:returns: Starlette HTTP Response
"""
if request.method == 'GET':
headers, status_code, content = ({}, 200, "[]")
elif request.method == 'POST':
headers, status_code, content = api_.execute_process(
request.headers, request.query_params, request.data, name)
response = Response(content=content, status_code=status_code)
if headers:
response.headers.update(headers)
return response
@click.command()
@click.pass_context
@click.option('--debug', '-d', default=False, is_flag=True, help='debug')
def serve(ctx, server, debug=False):
"""
Serve pygeoapi via Starlette. Runs pygeoapi
as a uvicorn server. Not recommend for production.
:param server: `string` of server type
:param debug: `bool` of whether to run in debug mode
:returns void
"""
# setup_logger(CONFIG['logging'])
uvicorn.run(
app, debug=True,
host=api_.config['server']['bind']['host'],
port=api_.config['server']['bind']['port'])
if __name__ == "__main__": # run locally, for testing
serve()
requirements-starlette.txt
+2
View File
@@ -0,0 +1,2 @@
starlette
uvicorn