Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/source/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,12 @@ default.
begin: 2000-10-30T18:24:39Z # start datetime in RFC3339
end: 2007-10-30T08:57:29Z # end datetime in RFC3339
trs: http://www.opengis.net/def/uom/ISO-8601/0/Gregorian # TRS
# additional extents can be added as desired (1..n)
foo:
url: https://example.org/def # required URL of the extent
range: [0, 10] # required overall range/extent
units: °C # optional units
values: [0, 2, 5, 5, 10] # optional, enumeration of values
providers: # list of 1..n required connections information
- type: feature # underlying data geospatial type. Allowed values are: feature, coverage, record, tile, edr
name: CSV # required: plugin name or import path. See Plugins section for more information.
Expand Down
62 changes: 56 additions & 6 deletions docs/source/plugins.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,15 @@ The core pygeoapi plugin registry can be found in ``pygeoapi.plugin.PLUGINS``.

Each plugin type implements its relevant base class as the API contract:

* data providers: ``pygeoapi.provider.base``
* output formats: ``pygeoapi.formatter.base``
* processes: ``pygeoapi.process.base``
* process_manager: ``pygeoapi.process.manager.base``
* data providers:

* features/records/maps: ``pygeoapi.provider.base.BaseProvider``
* edr: ``pygeoapi.provider.base_edr.BaseEDRProvider``
* tiles: ``pygeoapi.provider.tile.BaseTileProvider``

* output formats: ``pygeoapi.formatter.base.BaseFormatter``
* processes: ``pygeoapi.process.base.BaseProcessor``
* process_manager: ``pygeoapi.process.manager.base.BaseManager``

.. todo:: link PLUGINS to API doc

Expand Down Expand Up @@ -150,7 +155,7 @@ option 2 above).
Example: custom pygeoapi vector data provider
---------------------------------------------

Lets consider the steps for a vector data provider plugin:
Let's consider the steps for a vector data provider plugin:

Python code
^^^^^^^^^^^
Expand Down Expand Up @@ -223,7 +228,7 @@ Each base class documents the functions, arguments and return types required for
Example: custom pygeoapi raster data provider
---------------------------------------------

Lets consider the steps for a raster data provider plugin:
Let's consider the steps for a raster data provider plugin:

Python code
^^^^^^^^^^^
Expand Down Expand Up @@ -278,6 +283,51 @@ Each base class documents the functions, arguments and return types required for

.. _example-custom-pygeoapi-processing-plugin:

Example: custom pygeoapi EDR data provider
------------------------------------------

Let's consider the steps for an EDR data provider plugin:

Python code
^^^^^^^^^^^

The below template provides a minimal example (let's call the file ``mycooledrdata.py``:

.. code-block:: python

from pygeoapi.provider.base_edr import BaseEDRProvider

class MyCoolEDRDataProvider(BaseEDRProvider):

def __init__(self, provider_def):
"""Inherit from the parent class"""

super().__init__(provider_def)

self.covjson = {...}

def get_instances(self):
return ['foo', 'bar']

def get_instance(self, instance):
return instance in get_instances()

def position(self, **kwargs):
return self.covjson

def trajectory(self, **kwargs):
return self.covjson


For brevity, the ``position`` function returns ``self.covjson`` which is a
dictionary of a CoverageJSON representation. ``get_instances`` returns a list
of instances associated with the collection/plugin, and ``get_instance`` returns
a boolean of whether a given instance exists/is valid. EDR query types are subject
to the query functions defined in the plugin. In the example above, the plugin
implements ``position`` and ``trajectory`` queries, which will be advertised as
supported query types.


Example: custom pygeoapi processing plugin
------------------------------------------

Expand Down
10 changes: 10 additions & 0 deletions docs/source/publishing/ogcapi-coverages.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,11 +89,20 @@ The `Xarray`_ provider plugin reads and extracts `NetCDF`_ and `Zarr`_ data.
format:
name: zarr
mimetype: application/zip
options:
zarr:
consolidated: true
squeeze: true


.. note::
`Zarr`_ files are directories with files and subdirectories. Therefore
a zip file is returned upon request for said format.

.. note::

``options.zarr`` is a custom property that can be used to set `Zarr-specific open options`_.

.. note::
When referencing `NetCDF`_ or `Zarr`_ data stored in an S3 bucket,
be sure to provide the full S3 URL. Any parameters required to open the dataset
Expand Down Expand Up @@ -155,3 +164,4 @@ Data access examples
.. _`Zarr`: https://zarr.readthedocs.io/en/stable
.. _`GDAL raster driver short name`: https://gdal.org/drivers/raster/index.html
.. _`pyproj.CRS.from_user_input`: https://pyproj4.github.io/pyproj/stable/api/crs/coordinate_system.html#pyproj.crs.CoordinateSystem.from_user_input
.. _`Zarr-specific open options`: https://docs.xarray.dev/en/stable/generated/xarray.open_zarr.html
7 changes: 6 additions & 1 deletion docs/source/publishing/ogcapi-edr.rst
Original file line number Diff line number Diff line change
Expand Up @@ -90,11 +90,15 @@ The `xarray-edr`_ provider plugin reads and extracts `NetCDF`_ and `Zarr`_ data
a zip file is returned upon request for said format.

.. note::

``options.zarr`` is a custom property that can be used to set `Zarr-specific open options`_.

.. note::

When referencing data stored in an S3 bucket, be sure to provide the full
S3 URL. Any parameters required to open the dataset using fsspec can be added
to the config file under `options` and `s3`, as shown above.


SensorThingsEDR
^^^^^^^^^^^^^^^

Expand Down Expand Up @@ -143,3 +147,4 @@ Data access examples
.. _`NetCDF`: https://en.wikipedia.org/wiki/NetCDF
.. _`Zarr`: https://zarr.readthedocs.io/en/stable
.. _`OGC Environmental Data Retrieval (EDR) (API)`: https://ogcapi.ogc.org/edr
.. _`Zarr-specific open options`: https://docs.xarray.dev/en/stable/generated/xarray.open_zarr.html
4 changes: 4 additions & 0 deletions docs/source/publishing/ogcapi-maps.rst
Original file line number Diff line number Diff line change
Expand Up @@ -136,5 +136,9 @@ Data visualization examples

* http://localhost:5000/collections/foo/map?bbox-crs=http%3A%2F%2Fwww.opengis.net%2Fdef%2Fcrs%2FEPSG%2F0%2F3857&bbox=4.022369384765626%2C50.690447870569436%2C4.681549072265626%2C51.00260125274477&width=800&height=600&transparent

* map with vertical subset (``extents.vertical`` must be set in resource level config)

* http://localhost:5000/collections/foo/map?bbox=-142,42,-52,84&subset=vertical(435)

.. _`OGC API - Maps`: https://ogcapi.ogc.org/maps
.. _`see website`: https://mapserver.org/mapscript/index.html
100 changes: 77 additions & 23 deletions pygeoapi/api/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# Colin Blackburn <colb@bgs.ac.uk>
# Ricardo Garcia Silva <ricardo.garcia.silva@geobeyond.it>
#
# Copyright (c) 2025 Tom Kralidis
# Copyright (c) 2026 Tom Kralidis
# Copyright (c) 2025 Francesco Bartoli
# Copyright (c) 2022 John A Stevenson and Colin Blackburn
# Copyright (c) 2023 Ricardo Garcia Silva
Expand Down Expand Up @@ -588,6 +588,7 @@ def get_exception(self, status: int, headers: dict, format_: str | None,
"""

exception_info = sys.exc_info()

LOGGER.error(
description,
exc_info=exception_info if exception_info[0] is not None else None
Expand Down Expand Up @@ -709,22 +710,22 @@ def landing_page(api: API,
'title': l10n.translate('Collections', request.locale),
'href': api.get_collections_url()
}, {
'rel': 'http://www.opengis.net/def/rel/ogc/1.0/processes',
'rel': f'{OGC_RELTYPES_BASE}/processes',
'type': FORMAT_TYPES[F_JSON],
'title': l10n.translate('Processes', request.locale),
'href': f"{api.base_url}/processes"
}, {
'rel': 'http://www.opengis.net/def/rel/ogc/1.0/job-list',
'rel': f'{OGC_RELTYPES_BASE}/job-list',
'type': FORMAT_TYPES[F_JSON],
'title': l10n.translate('Jobs', request.locale),
'href': f"{api.base_url}/jobs"
}, {
'rel': 'http://www.opengis.net/def/rel/ogc/1.0/tiling-schemes',
'rel': f'{OGC_RELTYPES_BASE}/tiling-schemes',
'type': FORMAT_TYPES[F_JSON],
'title': l10n.translate('The list of supported tiling schemes as JSON', request.locale), # noqa
'href': f"{api.base_url}/TileMatrixSets?f=json"
}, {
'rel': 'http://www.opengis.net/def/rel/ogc/1.0/tiling-schemes',
'rel': f'{OGC_RELTYPES_BASE}/tiling-schemes',
'type': FORMAT_TYPES[F_HTML],
'title': l10n.translate('The list of supported tiling schemes as HTML', request.locale), # noqa
'href': f"{api.base_url}/TileMatrixSets?f=html"
Expand Down Expand Up @@ -897,7 +898,10 @@ def describe_collections(api: API, request: APIRequest,
'links': []
}

bbox = v['extents']['spatial']['bbox']
extents = deepcopy(v['extents'])

bbox = extents['spatial']['bbox']
LOGGER.debug('Setting spatial extents from configuration')
# The output should be an array of bbox, so if the user only
# provided a single bbox, wrap it in a array.
if not isinstance(bbox[0], list):
Expand All @@ -907,12 +911,13 @@ def describe_collections(api: API, request: APIRequest,
'bbox': bbox
}
}
if 'crs' in v['extents']['spatial']:
if 'crs' in extents['spatial']:
collection['extent']['spatial']['crs'] = \
v['extents']['spatial']['crs']
extents['spatial']['crs']

t_ext = v.get('extents', {}).get('temporal', {})
t_ext = extents.get('temporal', {})
if t_ext:
LOGGER.debug('Setting temporal extents from configuration')
begins = dategetter('begin', t_ext)
ends = dategetter('end', t_ext)
collection['extent']['temporal'] = {
Expand All @@ -921,6 +926,24 @@ def describe_collections(api: API, request: APIRequest,
if 'trs' in t_ext:
collection['extent']['temporal']['trs'] = t_ext['trs']

_ = extents.pop('spatial', None)
_ = extents.pop('temporal', None)

for ek, ev in extents.items():
LOGGER.debug(f'Adding extent {ek}')
collection['extent'][ek] = {
'definition': ev['url'],
'interval': [ev['range']]
}
if 'units' in ev:
collection['extent'][ek]['unit'] = ev['units']

if 'values' in ev:
collection['extent'][ek]['grid'] = {
'cellsCount': len(ev['values']),
'coordinates': ev['values']
}

LOGGER.debug('Processing configured collection links')
for link in l10n.translate(v.get('links', []), request.locale):
lnk = {
Expand Down Expand Up @@ -990,13 +1013,13 @@ def describe_collections(api: API, request: APIRequest,
if collection_data_type == 'record':
collection['links'].append({
'type': FORMAT_TYPES[F_JSON],
'rel': 'http://www.opengis.net/def/rel/ogc/1.0/ogc-catalog',
'rel': f'{OGC_RELTYPES_BASE}/ogc-catalog',
'title': l10n.translate('Record catalogue as JSON', request.locale), # noqa
'href': f'{api.get_collections_url()}/{k}?f={F_JSON}'
})
collection['links'].append({
'type': FORMAT_TYPES[F_HTML],
'rel': 'http://www.opengis.net/def/rel/ogc/1.0/ogc-catalog',
'rel': f'{OGC_RELTYPES_BASE}/ogc-catalog',
'title': l10n.translate('Record catalogue as HTML', request.locale), # noqa
'href': f'{api.get_collections_url()}/{k}?f={F_HTML}'
})
Expand All @@ -1021,13 +1044,13 @@ def describe_collections(api: API, request: APIRequest,
LOGGER.debug('Adding feature/record based links')
collection['links'].append({
'type': 'application/schema+json',
'rel': 'http://www.opengis.net/def/rel/ogc/1.0/queryables',
'rel': f'{OGC_RELTYPES_BASE}/queryables',
'title': l10n.translate('Queryables for this collection as JSON', request.locale), # noqa
'href': f'{api.get_collections_url()}/{k}/queryables?f={F_JSON}' # noqa
})
collection['links'].append({
'type': FORMAT_TYPES[F_HTML],
'rel': 'http://www.opengis.net/def/rel/ogc/1.0/queryables',
'rel': f'{OGC_RELTYPES_BASE}/queryables',
'title': l10n.translate('Queryables for this collection as HTML', request.locale), # noqa
'href': f'{api.get_collections_url()}/{k}/queryables?f={F_HTML}' # noqa
})
Expand Down Expand Up @@ -1135,19 +1158,20 @@ def describe_collections(api: API, request: APIRequest,
LOGGER.debug('Adding tile links')
collection['links'].append({
'type': FORMAT_TYPES[F_JSON],
'rel': f'http://www.opengis.net/def/rel/ogc/1.0/tilesets-{p.tile_type}', # noqa
'rel': f'{OGC_RELTYPES_BASE}/tilesets-{p.tile_type}',
'title': l10n.translate('Tiles as JSON', request.locale),
'href': f'{api.get_collections_url()}/{k}/tiles?f={F_JSON}' # noqa
'href': f'{api.get_collections_url()}/{k}/tiles?f={F_JSON}'
})
collection['links'].append({
'type': FORMAT_TYPES[F_HTML],
'rel': f'http://www.opengis.net/def/rel/ogc/1.0/tilesets-{p.tile_type}', # noqa
'rel': f'{OGC_RELTYPES_BASE}/tilesets-{p.tile_type}',
'title': l10n.translate('Tiles as HTML', request.locale),
'href': f'{api.get_collections_url()}/{k}/tiles?f={F_HTML}' # noqa
'href': f'{api.get_collections_url()}/{k}/tiles?f={F_HTML}'
})

try:
map_ = get_provider_by_type(v['providers'], 'map')
p = load_plugin('provider', map_)
except ProviderTypeError:
map_ = None

Expand All @@ -1158,15 +1182,36 @@ def describe_collections(api: API, request: APIRequest,
map_format = map_['format']['name']

title_ = l10n.translate('Map as', request.locale)
title_ = f"{title_} {map_format}"
title_ = f'{title_} {map_format}'

collection['links'].append({
'type': map_mimetype,
'rel': 'http://www.opengis.net/def/rel/ogc/1.0/map',
'rel': f'{OGC_RELTYPES_BASE}/map',
'title': title_,
'href': f"{api.get_collections_url()}/{k}/map?f={map_format}" # noqa
'href': f'{api.get_collections_url()}/{k}/map?f={map_format}'
})

if p._fields:
schema_reltype = f'{OGC_RELTYPES_BASE}/schema',
schema_links = [s for s in collection['links'] if
schema_reltype in s]

if not schema_links:
title_ = l10n.translate('Schema of collection in JSON', request.locale) # noqa
collection['links'].append({
'type': 'application/schema+json',
'rel': f'{OGC_RELTYPES_BASE}/schema',
'title': title_,
'href': f'{api.get_collections_url()}/{k}/schema?f=json' # noqa
})
title_ = l10n.translate('Schema of collection in HTML', request.locale) # noqa
collection['links'].append({
'type': 'text/html',
'rel': f'{OGC_RELTYPES_BASE}/schema',
'title': title_,
'href': f'{api.get_collections_url()}/{k}/schema?f=html' # noqa
})

try:
edr = get_provider_by_type(v['providers'], 'edr')
p = load_plugin('provider', edr)
Expand Down Expand Up @@ -1217,6 +1262,10 @@ def describe_collections(api: API, request: APIRequest,
}
}
}

if request.format is not None and request.format == 'json':
data_query['link']['type'] = 'application/vnd.cov+json'

collection['data_queries'][qt] = data_query

title1 = l10n.translate('query for this collection as JSON', request.locale) # noqa
Expand Down Expand Up @@ -1334,9 +1383,14 @@ def get_collection_schema(api: API, request: Union[APIRequest, Any],
p = load_plugin('provider', get_provider_by_type(
api.config['resources'][dataset]['providers'], 'coverage')) # noqa
except ProviderTypeError:
LOGGER.debug('Loading record provider')
p = load_plugin('provider', get_provider_by_type(
api.config['resources'][dataset]['providers'], 'record'))
try:
LOGGER.debug('Loading record provider')
p = load_plugin('provider', get_provider_by_type(
api.config['resources'][dataset]['providers'], 'record'))
except ProviderTypeError:
LOGGER.debug('Loading edr provider')
p = load_plugin('provider', get_provider_by_type(
api.config['resources'][dataset]['providers'], 'edr'))
except ProviderGenericError as err:
LOGGER.error(err)
return api.get_exception(
Expand Down
Loading