Add enclosure link prefetcher and tests (#1169) (#1173)

docs/source/configuration.rst

@@ -179,7 +179,7 @@ default.
               - type: feature # underlying data geospatial type: (allowed values are: feature, coverage, record, tile, edr)
                 default: true  # optional: if not specified, the first provider definition is considered the default
                 name: CSV
-                # transactions: DO NOT ACTIVATE unless you have setup access contol beyond pygeoapi
+                # transactions: DO NOT ACTIVATE unless you have setup access control beyond pygeoapi
                 editable: true  # optional: if backend is writable, default is false
                 data: tests/data/obs.csv  # required: the data filesystem path or URL, depending on plugin setup
                 id_field: id  # required for vector data, the field corresponding to the ID
@@ -198,7 +198,7 @@ default.
       hello-world:  # name of process
           type: collection  # REQUIRED (collection, process, or stac-collection)
           processor:
-              name: HelloWorld  # Python path of process defition
+              name: HelloWorld  # Python path of process definition
 
 
 .. seealso::
@@ -207,6 +207,60 @@ default.
 .. seealso::
    :ref:`plugins` for more information on plugins
 
+Adding links to collections
+---------------------------
+
+You can add any type of link to a resource of type `collection`.
+pygeoapi does not enforce anything here, as long as the link has a `type`, `rel`, and `href` parameter.
+The `type` parameter defines the MIME type (`Content-Type`) of the linked resource.
+The `rel` parameter tell something about what kind of link it is. You could set this to `license` to
+add a data license link, or to `describedBy` if you wish to add a schema definition for example.
+
+It's also possible to add (bulk) download links to a collection.
+These links should have their `rel` parameter set to `enclosure` and must have a `length` parameter
+that defines the content length (byte size) of the file.
+If you know the content length and it never changes, you can set this and pygeoapi will return the enclosure link(s) as-is.
+
+However, the downloadable resource may be subject to change (e.g. it may grow in size over time).
+In that case, you can omit the `length` and pygeoapi will figure out the actual `Content-Length` header
+by issuing a `HEAD` request on the given URL (`href` parameter).
+Furthermore, if it notices that the defined `type` (MIME type) of the link does not match the actual
+`Content-Type` in the response headers, it will automatically update the `type` accordingly.
+Note that `type` is a mandatory link parameter though, so you must always set it.
+
+So for example, you could define a download link like so:
+
+.. code-block:: yaml
+
+  links
+    - type: application/octet-stream  # must have some MIME type
+      rel: enclosure
+      title: download link
+      href: https://myserver.com/data/file.zip  # URL
+
+And pygeoapi will turn that into:
+
+.. code-block:: json
+
+  {
+    "links": {
+      "type": "application/zip",
+      "rel": "enclosure",
+      "title": "download link",
+      "href": "https://myserver.com/data/file.zip",
+      "length": 46435
+    }
+  }
+
+Note how the MIME type was updated to match the actual `Content-Type` and that the `length` was set
+according to the `Content-Length` header.
+
+.. note::
+
+  If the `length` parameter is omitted and pygeoapi was not able to verify the `Content-Length` within 1 second
+  and/or within 1 URL redirect, the enclosure link will **not** be included in the response.
+  This means that if you want to be sure that the link is always included, you will have to set a `length`.
+
 
 Publishing hidden resources
 ---------------------------
@@ -396,7 +450,7 @@ one with terms defined by schema.org:
   linked-data:
     context:
       - schema: https://schema.org/
-        stn_id: schema:identifer
+        stn_id: schema:identifier
         datetime:
             "@id": schema:observationDate
             "@type": schema:DateTime
@@ -418,7 +472,7 @@ An example of a data provider that includes relationships between items is the S
 SensorThings API, by default, has relationships between entities within its data model.
 Setting the ``intralink`` field of the SensorThings provider to ``true`` sets pygeoapi
 to represent the relationship between configured entities as intra-pygeoapi links or URIs.
-This relationship can further be maintained in the JSON-LD structured data using the appropiate
+This relationship can further be maintained in the JSON-LD structured data using the appropriate
 ``@context`` with the sosa/ssn ontology. For example:
 
 .. code-block:: yaml
@@ -446,7 +500,7 @@ This relationship can further be maintained in the JSON-LD structured data using
             Datastream: sosa:isMemberOf
 
 Sometimes, the JSON-LD desired for an individual feature in a collection is more complicated than can be achieved by
-aliasing properties using a context. In thise case, it is possible to specify a Jinja2 template. When ``item_template``
+aliasing properties using a context. In this case, it is possible to specify a Jinja2 template. When ``item_template``
 is defined for a feature collection, the json-ld prepared by pygeoapi will be used to render the Jinja2 template
 specified by the path. The path specified can be absolute or relative to pygeoapi's template folder. For even more
 deployment flexibility, the path can be specified with string interpolation of environment variables.