huanld/pygeoapi
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Merge pull request #15 from geopython/openapi-gen

Browse Source 
implement OpenAPI from single config
This commit is contained in:
Jorge Samuel Mendes de Jesus
2018-04-01 09:53:45 +02:00
committed by GitHub
parent 43b44574f5 d63c23850a
commit 9bd3075023
15 changed files with 744 additions and 578 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+3
View File
@@ -103,3 +103,6 @@ ENV/
# pygeoapi artifacts
local.config.yml
local.swagger.yml
# misc
*.swp
README.md
+16 -8
View File
@@ -5,22 +5,21 @@ pygeoapi provides an API to geospatial data
## Installation
```bash
virtualenv -p python3 pygeoapi
virtualenv -p python pygeoapi
cd pygeoapi
. bin/activate
git clone https://github.com/geopython/pygeoapi.git
cd pygeoapi
pip3 install -r requirements.txt
pip3 install -r requirements-dev.txt
pip3 install -e .
cp openapi/wfs/0.0.1/pygeoapi-openapi.yml local.swagger.yml
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install -e .
cp pygeoapi-config.yml local.config.yml
vi local.config.yml
# TODO: what is most important to edit?
vi local.swagger.yml
# TODO: what is most important to edit?
export PYGEOAPI_CONFIG=/path/to/local.config.yml
export PYGEOAPI_SWAGGER=/path/to/local.swagger.yml
# generate OpenAPI Document
pygeoapi generate_openapi_document -c local.config.yml > openapi.yml
export PYGEOAPI_SWAGGER=/path/to/openapi.yml
pygeoapi serve
```
@@ -40,3 +39,12 @@ curl http://localhost:5000/obs
# feature
curl http://localhost:5000/obs/371
```
## Testing against Swagger UI
```bash
docker pull swaggerapi/swagger-ui
docker run -p 80:8080 swaggerapi/swagger-ui
# go to http://localhost
# enter http://localhost:5000/api and click 'Explore'
```
openapi/wfs/0.0.1/pygeoapi-openapi.yml
-411
View File
@@ -1,411 +0,0 @@
##############################################################################
#
# The MIT License (MIT)
# Copyright (c) 2018 Tom Kralidis
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
# DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE
# OR OTHER DEALINGS IN THE SOFTWARE.
#
##############################################################################
swagger: '2.0'
info:
title: A sample API conforming to the OGC Web Feature Service standard
version: 0.0.1
description: |-
WARNING - THIS IS WORK IN PROGRESS\
WARNING - This is a Swagger / OpenAPI 2.0 variant. WFS 3.0 is expected to use OAS 3.0, not 2.0.\
TODO - Add a description of the sample service.\
TODO - Add security elements in a separate example.\
TODO - Connect to a live service.
contact:
name: Acme Corporation
email: info@example.org
url: 'http://example.org/'
license:
name: CC-BY 4.0 license
url: 'https://creativecommons.org/licenses/by/4.0/'
schemes:
- http
host: {{host}}
basePath: /
paths:
/:
get:
summary: describe the feature collections in the dataset
operationId: pygeoapi.views.describe_collections
tags:
- Capabilities
parameters:
- $ref: '#/parameters/f'
produces:
- application/json
- text/html
responses:
'200':
description: The feature collections shared by this API.
schema:
$ref: '#/definitions/content'
default:
description: An error occured.
schema:
$ref: '#/definitions/exception'
/api:
get:
summary: the API description - this document
operationId: pygeoapi.views.get_specification
tags:
- Capabilities
parameters:
- $ref: '#/parameters/f'
produces:
- application/openapi+json;version=2.0
- text/html
responses:
'200':
description: >-
The formal documentation of this API according to the OpenAPI
specification, version 3.0. I.e., this document.
schema:
type: object
default:
description: An error occured.
schema:
$ref: '#/definitions/exception'
## START OF DATASET DEFINITION - TO BE REPLACED BY CLI TOOLING in the local swagger file
/obs:
get:
summary: retrieve observation features
description: >-
Sample observations
operationId: pygeoapi.views.get_features
tags:
- Features
parameters:
- name: dataset # constant parameter
in: query
type: string
enum:
- obs
default: obs
- $ref: '#/parameters/startIndex'
- $ref: '#/parameters/count'
- $ref: '#/parameters/resultType'
- $ref: '#/parameters/bbox'
- $ref: '#/parameters/f'
produces:
- application/geo+json
- text/html
responses:
'200':
description: >-
Information about the feature collection plus the first features
matching the selection parameters.
schema:
$ref: '#/definitions/featureCollection'
default:
description: An error occured.
schema:
$ref: '#/definitions/exception'
'/obs/{id}':
get:
summary: retrieve a feature
operationId: pygeoapi.views.get_feature
tags:
- Features
parameters:
- name: dataset # constant parameter
in: query
type: string
enum:
- obs
default: obs
- $ref: '#/parameters/id'
- $ref: '#/parameters/f'
produces:
- application/geo+json
- text/html
responses:
'200':
description: A feature.
schema:
$ref: '#/definitions/observation'
'404':
description: 'The feature with id {id} does not exist.'
schema:
$ref: '#/definitions/exception'
default:
description: An error occured.
schema:
$ref: '#/definitions/exception'
## END OF DATASET DEFINITION
parameters:
f:
name: f
in: query
description: >-
The format of the response. If no value is provided, the standard http
rules apply, i.e., the accept header shall be used to determine the
format.\
Pre-defined values are "json" and "html". The response to other values is
determined by the server.
type: string
enum:
- json
- html
startIndex:
name: start_index
in: query
description: >-
The optional startIndex parameter indicates the index within the result
set from which the server shall begin presenting results in the response
document. The first element has an index of 0.\
Minimum = 0.\
Default = 0.
type: integer
minimum: 0
default: 0
count:
name: count
in: query
description: >-
The optional count parameter limits the number of items that are presented
in the response document.\
Only items are counted that are on the first level of the collection in
the response document. Nested objects contained within the explicitly
requested items shall not be counted.\
Minimum = 1.\
Maximum = 10000.\
Default = 10.
type: integer
minimum: 1
maximum: 10000
default: 10
bbox:
name: bbox
in: query
description: >-
Only features that have a geometry that intersects the bounding box are
selected. The bounding box is provided as four numbers:
* Lower corner, coordinate axis 1 (minimum longitude) * Lower corner,
coordinate axis 2 (minimum latitude) * Upper corner, coordinate axis 1
(maximum longitude) * Upper corner, coordinate axis 2 (maximum latitude)
type: array
items:
type: number
minItems: 4
maxItems: 4
minimum: -180
maximum: 180
collectionFormat: csv
resultType:
name: result_type
in: query
description: >-
This service will respond to a query in one of two ways (excluding an
exception response). It may either generate a complete response document
containing resources that satisfy the operation or it may simply generate
an empty response container that indicates the count of the total number
of resources that the operation would return. Which of these two responses
is generated is determined by the value of the optional resultType
parameter.\
The allowed values for this parameter are "results" and "hits".\
If the value of the resultType parameter is set to "results", the server
will generate a complete response document containing resources that
satisfy the operation.\
If the value of the resultType attribute is set to "hits", the server will
generate an empty response document containing no resource instances.\
Default = "results".
type: string
enum:
- hits
- results
default: results
id:
name: id
in: path
description: The id of a feature
required: true
type: string
definitions:
exception:
type: object
required:
- code
properties:
code:
type: string
description:
type: string
bbox:
type: object
required:
- bbox
properties:
crs:
type: string
enum:
- 'http://www.opengis.net/def/crs/OGC/1.3/CRS84'
default: 'http://www.opengis.net/def/crs/OGC/1.3/CRS84'
bbox:
description: >-
minimum longitude, minimum latitude, maximum longitude, maximum
latitude
type: array
items:
minItems: 4
maxItems: 4
type: number
minimum: -180
maximum: 180
example:
- -180
- -90
- 180
- 90
content:
type: object
required:
- collections
properties:
collections:
type: array
items:
$ref: '#/definitions/collectionInfo'
collectionInfo:
type: object
required:
- name
- links
properties:
name:
type: string
example: address
title:
type: string
example: address
description:
type: string
example: An address.
links:
type: array
items:
$ref: '#/definitions/link'
example:
- href: 'http://data.example.com/observations'
rel: item
- href: 'http://example.com/concepts/observations.html'
rel: describedBy
type: text/html
extent:
$ref: '#/definitions/bbox'
crs:
description: TODO - add description ... first crs is the default crs
type: array
items:
type: string
default:
- 'http://www.opengis.net/def/crs/OGC/1.3/CRS84'
example:
- 'http://www.opengis.net/def/crs/OGC/1.3/CRS84'
- 'http://www.opengis.net/def/crs/EPSG/0/4326'
link:
type: object
required:
- href
properties:
href:
type: string
rel:
type: string
example: prev
type:
type: string
example: application/geo+json
hreflang:
type: string
example: en
featureCollection:
type: object
required:
- features
properties:
features:
type: array
items:
$ref: '#/definitions/observation'
Point:
type: object
required:
- type
properties:
type:
type: string
enum:
- Point
coordinates:
type: array
items:
type: number
minItems: 2
observation:
type: object
required:
- id
- type
- geometry
- properties
properties:
type:
type: string
enum:
- Feature
geometry:
$ref: '#/definitions/Point'
id:
type: string
properties:
type: object
properties:
stn_id:
type: string
datetime:
type: string
value:
type: number
tags:
- name: Capabilities
description: >-
Essential characteristics of this API including information about the
data.
- name: Features
description: Access to data (features).
pygeoapi-config.yml
+16 -8
View File
@@ -1,6 +1,7 @@
server:
host: localhost
port: 5000
basepath: /
mimetype: application/json; charset=UTF-8
encoding: utf-8
language: en-US
@@ -14,14 +15,15 @@ logging:
metadata:
identification:
title: pygeoapi
description: pygeoapi provides an API to geospatial data
title: pygeoapi default instance
description: pygeoapi provides an API to geospatial data
keywords:
- geospatial
- data
- api
keywords_type: theme
terms_of_service: None
url: http://example.org
license:
name: CC-BY 4.0 license
url: https://creativecommons.org/licenses/by/4.0/
@@ -57,9 +59,11 @@ datasets:
crs:
- CRS84
links:
- type: information
- type: text/csv
description: data
url: https://github.com/mapserver/mapserver/blob/branch-7-0/msautotest/wxs/data/obs.csv
- type: download
- type: text/csv
description: data
url: https://raw.githubusercontent.com/mapserver/mapserver/branch-7-0/msautotest/wxs/data/obs.csv
extents:
spatial:
@@ -81,9 +85,11 @@ datasets:
crs:
- CRS84
links:
- type: information
- type: text/html
description: information
url: http://example.org/dataset/index.html
- type: download
- type: application/gzip
description: download
url: http://example.org/dataset/data.tgz
extents:
spatial:
@@ -103,9 +109,11 @@ datasets:
crs:
- CRS84
links:
- type: information
- type: text/html
description: information
url: http://www.naturalearthdata.com/
- type: download
- type: text/html
description: download
url: http://www.naturalearthdata.com/
extents:
spatial:
pygeoapi/__init__.py
+2
View File
@@ -30,6 +30,7 @@
import click
from pygeoapi.flask_app import serve
from pygeoapi.openapi import generate_openapi_document
__version__ = '0.1.dev0'
@@ -41,3 +42,4 @@ def cli():
cli.add_command(serve)
cli.add_command(generate_openapi_document)
pygeoapi/flask_app.py
+94 -23
View File
@@ -28,38 +28,109 @@
#
# =================================================================
import click
import connexion
import os
from pygeoapi.log import setup_logger
import yaml
import click
from flask import Flask, make_response, request
from flask_cors import CORS
from pygeoapi import views
from pygeoapi.config import settings
from pygeoapi.log import setup_logger
from pygeoapi.util import get_url
APP = Flask(__name__)
APP.url_map.strict_slashes = False
@APP.route('/')
def root():
headers, status_code, content = views.root(
request.headers, request.args, APP.config['PYGEOAPI_BASEURL'])
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
@APP.route('/api')
def api():
with open(os.environ.get('PYGEOAPI_OPENAPI')) as ff:
openapi = yaml.load(ff)
headers, status_code, content = views.api(request.headers, request.args,
openapi)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
@APP.route('/conformance')
def api_conformance():
headers, status_code, content = views.api_conformance(request.headers,
request.args)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
@APP.route('/collections')
@APP.route('/collections/<name>')
def describe_collections(name=None):
headers, status_code, content = views.describe_collections(
request.headers, request.args, name)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
@APP.route('/collections/<feature_collection>/items')
@APP.route('/collections/<feature_collection>/items/<feature>')
def dataset(feature_collection, feature=None):
if feature is None:
headers, status_code, content = views.get_features(
request.headers, request.args, feature_collection)
else:
headers, status_code, content = views.get_feature(
request.headers, request.args, feature_collection, feature)
response = make_response(content, status_code)
if headers:
response.headers = headers
return response
@click.command()
@click.pass_context
@click.option('--host', '-h', default='localhost', help='Hostname')
@click.option('--port', '-p', default=5000, help='port')
@click.option('--debug', '-d', default=False, is_flag=True, help='debug')
def serve(ctx, host, port, debug=False):
def serve(ctx, debug=False):
"""Serve pygeoapi via Flask"""
if port is not None:
port_ = port
else:
port_ = settings['server']['port']
if host is not None:
host_ = host
else:
host_ = settings['server']['host']
if not settings['server']['pretty_print']:
APP.config['JSONIFY_PRETTYPRINT_REGULAR'] = False
app = connexion.FlaskApp(__name__, port=port_, specification_dir='.')
hostport = '{}:{}'.format(host_, port_)
api = app.add_api(settings['swagger'], debug=debug, strict_validation=True,
arguments={'host': hostport, 'config': settings})
settings['api'] = api.specification
if settings['server']['cors']:
CORS(APP)
setup_logger()
app.run()
# TODO: get scheme
BASEURL = get_url('http', settings['server']['host'],
settings['server']['port'],
settings['server']['basepath'])
APP.config['PYGEOAPI_BASEURL'] = BASEURL
APP.run(debug=True, host='0.0.0.0', port=settings['server']['port'])
pygeoapi/openapi.py
+283
View File
@@ -0,0 +1,283 @@
# =================================================================
#
# Authors: Tom Kralidis <tomkralidis@gmail.com>
#
# Copyright (c) 2018 Tom Kralidis
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
# files (the "Software"), to deal in the Software without
# restriction, including without limitation the rights to use,
# copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following
# conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
# =================================================================
import logging
import click
import yaml
LOGGER = logging.getLogger(__name__)
def get_oas_30(cfg):
"""
Generates an OpenAPI 3.0 Document
:param cfg: configuration object
:returns: OpenAPI definition YAML dict
"""
paths = {}
LOGGER.debug('setting up server info')
oas = {
'openapi': '3.0.1',
'tags': []
}
info = {
'title': cfg['metadata']['identification']['title'],
'description': cfg['metadata']['identification']['description'],
'x-keywords': cfg['metadata']['identification']['keywords'],
'termsOfService':
cfg['metadata']['identification']['terms_of_service'],
'contact': {
'name': cfg['metadata']['provider']['name'],
'url': cfg['metadata']['provider']['url'],
'email': cfg['metadata']['contact']['email']
},
'license': {
'name': cfg['metadata']['license']['name'],
'url': cfg['metadata']['license']['url']
},
'version': '3.0.1'
}
oas['info'] = info
url = 'http://{}'.format(cfg['server']['host'])
if cfg['server']['port'] not in [80, 443]:
url = '{}:{}'.format(url, cfg['server']['port'])
oas['servers'] = [{
'url': url,
'description': cfg['metadata']['identification']['description']
}]
paths['/'] = {
'get': {
'summary': 'API',
'description': 'API',
'tags': ['server'],
'responses': {
200: {
'description': 'successful operation'
}
}
}
}
paths['/api'] = {
'get': {
'summary': 'This document',
'description': 'This document',
'tags': ['server'],
'responses': {
200: {
'description': 'successful operation'
}
}
}
}
paths['/conformance'] = {
'get': {
'summary': 'API conformance definition',
'description': 'API conformance definition',
'tags': ['server'],
'responses': {
200: {
'description': 'successful operation'
}
}
}
}
paths['/collections'] = {
'get': {
'summary': 'Feature Collections',
'descriptions': 'Feature Collections',
'tags': ['server'],
'responses': {
200: {
'description': 'successful operation'
}
}
}
}
oas['tags'].append({
'name': 'server',
'description': cfg['metadata']['identification']['description'],
'externalDocs': {
'description': 'information',
'url': cfg['metadata']['identification']['url']}
}
)
LOGGER.debug('setting up datasets')
for k, v in cfg['datasets'].items():
tag = {
'name': k,
'description': v['description'],
'externalDocs': {}
}
for link in v['links']:
if link['type'] == 'information':
tag['externalDocs']['description'] = link['type']
tag['externalDocs']['url'] = link['url']
break
oas['tags'].append(tag)
paths['/collections/{}'.format(k)] = {
'get': {
'summary': 'Get feature collection metadata'.format(v['title']), # noqa
'description': v['description'],
'tags': [k],
'responses': {
200: {
'description': 'successful operation'
},
400: {
'description': 'Invalid ID supplied'
},
404: {
'description': 'not found'
}
}
}
}
paths['/collections/{}/items'.format(k)] = {
'get': {
'summary': 'Get {} features'.format(v['title']),
'description': v['description'],
'tags': [k],
'parameters': [
{'$ref': '#/components/parameters/limit'}
],
'responses': {
200: {
'description': 'successful operation'
},
400: {
'description': 'Invalid ID supplied'
},
404: {
'description': 'not found'
}
}
}
}
paths['/collections/{}/items/{{id}}'.format(k)] = {
'get': {
'summary': 'Get {} feature by ID'.format(v['title']),
'description': v['description'],
'tags': [k],
'parameters': [
{'$ref': '#/components/parameters/id'}
],
'responses': {
200: {
'description': 'successful operation'
},
400: {
'description': 'Invalid ID supplied'
},
404: {
'description': 'not found'
}
}
}
}
oas['paths'] = paths
oas['components'] = {
'parameters': {
'id': {
'name': 'id',
'in': 'path',
'description': 'The id of a feature',
'required': True,
'type': 'string'
},
'limit': {
'name': 'limit',
'in': 'query',
'description': ('The optional limit parameter limits the',
' number of items that are presented in the',
' response document. Only items are counted',
' that are on the first level of the',
' collection in the response document. Nested',
' objects contained within the explicitly',
' requested items shall not be counted.',
' Minimum = 1. Maximum = 10000.',
' Default = {}.'.format(
cfg['server']['limit'])),
'required': False,
'schema': {
'type': 'integer',
'minimum': 1,
'maximum': 10000,
'default': cfg['server']['limit']
},
'style': 'form',
'explode': False
}
}
}
return oas
def get_oas(cfg, version='3.0'):
"""
Stub to generate OpenAPI Document
:param cfg: configuration object
:param version: version of OpenAPI (default 3.0)
:returns: OpenAPI definition YAML dict
"""
if version == '3.0':
return get_oas_30(cfg)
else:
raise RuntimeError('OpenAPI version not supported')
@click.command()
@click.pass_context
@click.option('--config', '-c', 'config_file', help='configuration file')
def generate_openapi_document(ctx, config_file):
"""Generate OpenAPI Document"""
if config_file is None:
raise click.ClickException('--config/-c required')
with open(config_file) as ff:
s = yaml.load(ff)
click.echo(yaml.dump(get_oas(s), default_flow_style=False))
pygeoapi/provider/csv_.py
+18 -10
View File
@@ -55,53 +55,61 @@ class CSVProvider(BaseProvider):
BaseProvider.__init__(self, name, data, id_field)
def _load(self, startindex=0, count=10, resulttype='results',
def _load(self, startindex=0, limit=10, resulttype='results',
identifier=None):
"""
Load CSV data
:param startindex: starting record to return (default 0)
:param count: number of records to return (default 10)
:param resulttype: return results or hit count (default results)
:param limit: number of records to return (default 10)
:param resulttype: return results or hit limit (default results)
:returns: dict of GeoJSON FeatureCollection
"""
found = False
result = None
feature_collection = {
'type': 'FeatureCollection',
'features': []
}
with open(self.data) as ff:
LOGGER.debug('Serializing DictReader')
data_ = csv.DictReader(ff)
if resulttype == 'hits':
LOGGER('Retrurning hits only')
LOGGER('Returning hits only')
feature_collection['numberMatched'] = len(list(data_))
return feature_collection
LOGGER.debug('Slicing CSV rows')
for row in itertools.islice(data_, startindex, startindex+count):
for row in itertools.islice(data_, startindex, startindex+limit):
feature = {'type': 'Feature'}
feature['ID'] = row.pop('id')
feature['geometry'] = mapping(wkt.loads(row.pop('geom')))
feature['properties'] = row
if identifier is not None and feature['ID'] == identifier:
return feature
found = True
result = feature
feature_collection['features'].append(feature)
if identifier is not None and not found:
return None
elif identifier is not None and found:
return result
return feature_collection
def query(self, startindex=0, count=10, resulttype='results'):
def query(self, startindex=0, limit=10, resulttype='results'):
"""
CSV query
:param startindex: starting record to return (default 0)
:param count: number of records to return (default 10)
:param resulttype: return results or hit count (default results)
:param limit: number of records to return (default 10)
:param resulttype: return results or hit limit (default results)
:returns: dict of GeoJSON FeatureCollection
"""
return self._load(startindex, count, resulttype)
return self._load(startindex, limit, resulttype)
def get(self, identifier):
"""
pygeoapi/provider/elasticsearch_.py
+7 -7
View File
@@ -65,13 +65,13 @@ class ElasticsearchProvider(BaseProvider):
LOGGER.debug('Connecting to Elasticsearch')
self.es = Elasticsearch(self.es_host)
def query(self, startindex=0, count=10, resulttype='results'):
def query(self, startindex=0, limit=10, resulttype='results'):
"""
query Elasticsearch index
:param startindex: starting record to return (default 0)
:param count: number of records to return (default 10)
:param resulttype: return results or hit count (default results)
:param limit: number of records to return (default 10)
:param resulttype: return results or hit limit (default results)
:returns: dict of 0..n GeoJSON features
"""
@@ -84,16 +84,16 @@ class ElasticsearchProvider(BaseProvider):
LOGGER.debug('Querying Elasticsearch')
if resulttype == 'hits':
LOGGER.debug('hits only specified')
count = 0
limit = 0
results = self.es.search(index=self.index_name, from_=startindex,
size=count)
size=limit)
if resulttype == 'hits':
feature_collection['numberMatched'] = results['hits']['total']
return feature_collection
LOGGER.debug('serializing features')
for feature in results['hits']['hits']:
id_ = feature['_source']['properties']['identifier']
id_ = feature['_source']['properties'][self.id_field]
LOGGER.debug('serializing id {}'.format(id_))
feature['_source']['ID'] = id_
feature_collection['features'].append(feature['_source'])
@@ -114,7 +114,7 @@ class ElasticsearchProvider(BaseProvider):
result = self.es.get(self.index_name, doc_type=self.type_name,
id=identifier)
LOGGER.debug('Serializing feature')
id_ = result['_source']['properties']['identifier']
id_ = result['_source']['properties'][self.id_field]
result['_source']['ID'] = id_
except Exception as err:
LOGGER.error(err)
pygeoapi/provider/geojson.py
+4 -4
View File
@@ -70,8 +70,8 @@ class GeoJSONProvider(BaseProvider):
Yes loading from disk, deserializing and validation
happens on every request. This is not efficient.
"""
if os.path.exists(self.path):
with open(self.path) as src:
if os.path.exists(self.data):
with open(self.data) as src:
data = json.loads(src.read())
else:
data = {
@@ -85,7 +85,7 @@ class GeoJSONProvider(BaseProvider):
return data
def query(self, startindex=0, count=10, resulttype='results'):
def query(self, startindex=0, limit=10, resulttype='results'):
"""
query the provider
@@ -99,7 +99,7 @@ class GeoJSONProvider(BaseProvider):
data['numberMatched'] = len(data['features'])
data['features'] = []
else:
data['features'] = data['features'][startindex:startindex + count]
data['features'] = data['features'][startindex:startindex+limit]
return data
pygeoapi/static/css/default.css
+1 -1
View File
@@ -36,4 +36,4 @@ table {
float: left;
margin: 0px 15px 15px 0px;
border: 0;
}
}
pygeoapi/templates/service.html
+4 -2
View File
@@ -8,7 +8,7 @@
<meta name="description" content="{{ config['metadata']['identification']['title'] }}">
<meta name="keywords" content="{{ config['metadata']['identification']['keywords']|join(',') }}">
<link rel="stylesheet" href="{{ url_for('static', filename='css/default.css') }}">
<link rel="stylesheet" href="static/css/default.css">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.js"></script>
@@ -30,9 +30,11 @@
</section>
<section id="datasets">
<h2>Datasets</h2>
<ul>
{% for k, v in config['datasets'].items()%}
<span><a title="{{ v['title'] }}" href="{{ config['server']['url'] }}{{ url_for('dataset', dataset_name=k) }}">{{ v['title'] }}</a></span>
<li><a title="{{ v['title'] }}" href="{{ config['server']['url'] }}{{ k }}">{{ v['title'] }}</a></li>
{% endfor %}
</ul>
</section>
<hr/>
<footer>Powered by <a title="pygeoapi" href="https://github.com/geopython/pygeoapi">pygeoapi</a> {{ version }}</footer>
pygeoapi/util.py
+49
View File
@@ -0,0 +1,49 @@
# =================================================================
#
# Authors: Tom Kralidis <tomkralidis@gmail.com>
#
# Copyright (c) 2018 Tom Kralidis
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
# files (the "Software"), to deal in the Software without
# restriction, including without limitation the rights to use,
# copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following
# conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
# =================================================================
import logging
LOGGER = logging.getLogger(__name__)
def get_url(scheme, host, port, basepath):
"""
Provides URL of instance
:returns: string of complete baseurl
"""
url = '{}://{}'.format(scheme, host)
if port not in [80, 443]:
url = '{}:{}'.format(url, port)
url = '{}{}'.format(url, basepath)
return url
pygeoapi/views.py
+247 -103
View File
@@ -28,27 +28,117 @@
#
# =================================================================
import json
import logging
import os
from flask import request
# from flask import request, url_for
from jinja2 import Environment, FileSystemLoader
from pygeoapi.config import settings
from pygeoapi.provider import load_provider
LOGGER = logging.getLogger(__name__)
TEMPLATES = '{}{}templates'.format(os.path.dirname(
os.path.realpath(__file__)), os.sep)
def api_conformance(f='json'):
def root(headers, args, baseurl):
"""
Provide API
:param headers: dict of HTTP headers
:param args: dict of HTTP request parameters
:param baseurl: baseurl of the server
:returns: tuple of headers, status code, content
"""
headers_ = {
'Content-type': 'application/json'
}
formats = ['json', 'html']
format_ = args.get('f')
if format_ is not None and format_ not in formats:
exception = {
'code': 'InvalidParameterValue',
'description': 'Invalid format'
}
LOGGER.error(exception)
return headers_, 400, json.dumps(exception)
fcm = {
'links': [],
}
LOGGER.debug('Creating links')
fcm['links'] = [{
'rel': 'self',
'type': 'application/json',
'title': 'this document',
'href': baseurl
}, {
'rel': 'self',
'type': 'text/html',
'title': 'this document as HTML',
'href': '{}?f=html'.format(baseurl)
}, {
'rel': 'self',
'type': 'application/openapi+json;version=3.0',
'title': 'the OpenAPI definition as JSON',
'href': '{}api'.format(baseurl)
}, {
'rel': 'self',
'type': 'text/html',
'title': 'the OpenAPI definition as HTML',
'href': '{}?f=html'.format(baseurl)
}
]
if format_ == 'html': # render
headers_['Content-type'] = 'text/html'
content = _render_j2_template(settings, 'service.html', fcm)
return headers_, 200, content
return headers_, 200, json.dumps(fcm)
def api(headers, args, openapi):
"""
Provide OpenAPI document
:param headers: dict of HTTP headers
:param args: dict of HTTP request parameters
:param openapi: dict of OpenAPI definition
:returns: tuple of headers, status code, content
"""
headers_ = {
'Content-type': 'application/json'
}
return headers_, 200, json.dumps(openapi)
def api_conformance(headers, args):
"""
Provide conformance definition
:param f: response format (default JSON)
:param headers: dict of HTTP headers
:param args: dict of HTTP request parameters
:returns: dict of conformance
:returns: tuple of headers, status code, content
"""
return {
headers_ = {
'Content-type': 'application/json'
}
conformance = {
'conformsTo': [
'http://www.opengis.net/spec/wfs-1/3.0/req/core',
'http://www.opengis.net/spec/wfs-1/3.0/req/oas30',
@@ -57,53 +147,37 @@ def api_conformance(f='json'):
]
}
def describe_collections(f='json'):
return headers_, 200, json.dumps(conformance)
def describe_collections(headers, args, name=None):
"""
Provide feature collection metadata
:param f: response format (default JSON)
:param headers: dict of HTTP headers
:param args: dict of HTTP request parameters
:returns: dict of feature collection metadata
:returns: tuple of headers, status code, content
"""
# TODO allow other file return formats
if f.upper() != 'JSON':
msg = 'Unsupported format: {}'.format(f)
LOGGER.error(msg)
return msg, 400
fcm = {
'links': [],
'collections': []
headers_ = {
'Content-type': 'application/json'
}
url = '{}://{}'.format(request.scheme, settings['server']['host'])
if settings['server']['port'] not in [80, 443]:
url = '{}:{}'.format(url, settings['server']['port'])
formats = ['json', 'html']
# LOGGER.debug('Creating links')
# fcm['links'] = [{
# 'rel': 'self',
# 'type': 'application/json',
# 'title': 'this document',
# 'href': '{}{}'.format(url, url_for('index_json')),
# }, {
# 'rel': 'self',
# 'type': 'text/html',
# 'title': 'this document as HTML',
# 'href': '{}{}'.format(url, url_for('index_html')),
# }, {
# 'rel': 'self',
# 'type': 'application/openapi+json;version=3.0',
# 'title': 'the OpenAPI definition as JSON',
# 'href': '{}{}'.format(url, url_for('api_json')),
# }, {
# 'rel': 'self',
# 'type': 'text/html',
# 'title': 'the OpenAPI definition as HTML',
# 'href': '{}{}'.format(url, url_for('api_html')),
# }
# ]
format_ = args.get('f')
if format_ is not None and format_ not in formats:
exception = {
'code': 'InvalidParameterValue',
'description': 'Invalid format'
}
LOGGER.error(exception)
return headers_, 400, json.dumps(exception)
fcm = {
'collections': []
}
LOGGER.debug('Creating collections')
for k, v in settings['datasets'].items():
@@ -117,80 +191,150 @@ def describe_collections(f='json'):
collection['extent'] = v['extents']['spatial']['bbox']
for link in v['links']:
lnk = {'rel': link['type'], 'href': link['url']}
lnk = {
'rel': 'alternate',
'type': link['type'],
'href': link['url']
}
collection['links'].append(lnk)
if name is not None and k == name:
return headers_, 200, json.dumps(collection)
fcm['collections'].append(collection)
return fcm
if format_ == 'html': # render
headers_['Content-type'] = 'text/html'
content = _render_j2_template(settings, 'service.html', fcm)
return headers_, 200, content
return headers_, 200, json.dumps(fcm)
def get_specification(f='json'):
if f.upper() == 'JSON':
return settings['api']
else:
return '{} not supported as a query parameter'.format(f), 400
def get_features(dataset, startindex=0, count=10, resulttype='results',
bbox=None, f='json'):
def get_features(headers, args, dataset):
"""
Queries feature collection
:param dataset: dataset to query
:param startindex: starting record to return (default 0)
:param count: number of records to return (default 10)
:param resulttype: return results or hit count (default results)
:param bbox: list of minx,miny,maxx,maxy
:param f: responase format (default GeoJSON)
:returns: dict of GeoJSON FeatureCollection
:param headers: dict of HTTP headers
:param args: dict of HTTP request parameters
:param dataset: dataset name
:returns: tuple of headers, status code, content
"""
headers_ = {
'Content-type': 'application/json'
}
startindex = args.get('startindex') or 0
limit = args.get('limit') or settings['server']['limit']
resulttype = args.get('resulttype') or 'results'
if dataset not in settings['datasets'].keys():
msg = 'dataset {} not found'.format(dataset)
LOGGER.error(msg)
return msg, 400
else:
LOGGER.debug('Loading provider')
p = load_provider(settings['datasets'][dataset]['provider'],
settings['datasets'][dataset]['data'],
settings['datasets'][dataset]['id_field'])
LOGGER.debug('Querying provider')
LOGGER.debug('startindex: {}'.format(startindex))
LOGGER.debug('count: {}'.format(count))
LOGGER.debug('resulttype: {}'.format(resulttype))
results = p.query(startindex=int(startindex), count=int(count),
resulttype=resulttype)
return results
def get_feature(dataset, id, f='json'):
"""
Get a single feature
:param dataset: dataset to query
:param id: feature identifier
:param f: responase format (default GeoJSON)
:returns: dict of GeoJSON Feature
"""
if dataset not in settings['datasets'].keys():
msg = 'dataset {} not found'.format(dataset)
LOGGER.error(msg)
return msg, 400
exception = {
'code': 'InvalidParameterValue',
'description': 'Invalid feature collection'
}
LOGGER.error(exception)
return headers_, 400, json.dumps(exception)
LOGGER.debug('Loading provider')
p = load_provider(settings['datasets'][dataset]['provider'],
settings['datasets'][dataset]['data'],
settings['datasets'][dataset]['id_field'])
LOGGER.debug('Fetching id {}'.format(id))
feature = p.get(id)
if feature is None:
msg = 'feature {} not found'.format(id)
LOGGER.warning(msg)
return msg, 404
LOGGER.debug('Querying provider')
LOGGER.debug('startindex: {}'.format(startindex))
LOGGER.debug('limit: {}'.format(limit))
LOGGER.debug('resulttype: {}'.format(resulttype))
content = p.query(startindex=int(startindex), limit=int(limit),
resulttype=resulttype)
return feature
next_ = startindex + settings['server']['limit']
content['links'] = [{
'rel': 'self',
'type': 'application/json',
'href': '/collections/{}/items'.format(dataset)
}, {
'rel': 'next',
'type': 'application/json',
'href': '/collections/{}/items/?startindex={}'.format(dataset, next_)
}, {
'rel': 'collection',
'type': 'application/json',
'href': '/collections/{}'.format(dataset)
}
]
return headers_, 200, json.dumps(content)
def get_feature(headers, args, dataset, identifier):
"""
Get a single feature
:param headers: dict of HTTP headers
:param args: dict of HTTP request parameters
:param dataset: dataset name
:param identifier: feature identifier
:returns: tuple of headers, status code, content
"""
headers_ = {
'Content-type': 'application/json'
}
if dataset not in settings['datasets'].keys():
exception = {
'code': 'InvalidParameterValue',
'description': 'Invalid feature collection'
}
LOGGER.error(exception)
return headers_, 400, json.dumps(exception)
LOGGER.debug('Loading provider')
p = load_provider(settings['datasets'][dataset]['provider'],
settings['datasets'][dataset]['data'],
settings['datasets'][dataset]['id_field'])
LOGGER.debug('Fetching id {}'.format(identifier))
content = p.get(identifier)
content['links'] = [{
'rel': 'self',
'type': 'application/json',
'href': '/collections/{}/items/{}'.format(dataset, identifier)
}, {
'rel': 'collection',
'type': 'application/json',
'href': '/collections/{}'.format(dataset)
}
]
if content is None:
exception = {
'code': 'NotFound',
'description': 'identifier not found'
}
LOGGER.error(exception)
return headers_, 404, json.dumps(exception)
return headers_, 200, json.dumps(content)
def _render_j2_template(config, template, data):
"""
render Jinja2 template
:param config: dict of configuration
:param template: template (relative path)
:param data: dict of data
:returns: string of rendered template
"""
env = Environment(loader=FileSystemLoader(TEMPLATES))
template = env.get_template(template)
return template.render(config=config, data=data)
requirements.txt
-1
View File
@@ -1,5 +1,4 @@
click
connexion
elasticsearch
flask
flask_cors