S5P-PAL Data Portal

S5P-PAL product files can be selected and downloaded using the SpatioTemporal Asset Catalog (STAC) browser interface built on top of our implementation of the STAC Collection Specification.

Because STAC is an open standard, you are not restricted to using this S5P-PAL-hosted GUI. If you have access to another STAC browser or viewer, you can point it directly towards the S5P-PAL s5p-l2 catalogue at:

https://data-portal.s5p-pal.com/api/s5p-l2

(or any of the "root collections" for individual product types you will see displayed on the screen when browsing) and it should work out of the box.

Programmatic access

Similarly, you can use a STAC library, such as the PySTAC Client for Python, to programmatically access the S5P-PAL product catalogue and, for example, obtain download links for product files that way. (There are similar packages and tools for other languages.)

S5P-PAL implements the STAC Item Search API, exposing a search endpoint for querying items in the catalogue based on geospatial or temporal criteria. This search endpoint is not available through the current version of the browser interface, so in order to make use of it doing so programmatically is, for now, the only way.

Item Search Examples

In this section we give some basic examples to help you on your way, using the aforementioned PySTAC Client library for Python. PySTAC Client requires that you also have the lower-level PySTAC library installed.

The following is a simple script that will print some information about the most recently generated product in the catalogue, and then download it. (This script uses the Python Requests library to handle the actual download.)

from pystac import Catalog
from pystac_client import ItemSearch
import requests
import os

L2_CATALOG = "https://data-portal.s5p-pal.com/api/s5p-l2"

def get_most_recent_product(catalog_url):
    catalog = Catalog.from_file(catalog_url)
    endpoint = catalog.get_single_link("search").target

    items = ItemSearch(
        endpoint,
        sortby=[{"field": "properties.archive_date", "direction": "desc"}],
        max_items=1,
    ).items()
    item = list(items)[0]

    product = item.assets["product"]
    extra_fields = product.extra_fields

    download_url = product.href
    product_local_path = extra_fields["file:local_path"]
    product_size = extra_fields["file:size"]

    print(f"Downloading {product_local_path}...")
    r = requests.get(download_url)
    with open(f"./{product_local_path}", "wb") as product_file:
        product_file.write(r.content)
    print("Comparing file sizes...")
    file_size = os.path.getsize(f"./{product_local_path}")
    print(f"{file_size=}")
    assert file_size == product_size
    print("Sizes match; product was downloaded correctly")

if __name__ == "__main__":
    get_most_recent_product(L2_CATALOG)

Downloading S5P_PAL__L2__SO2CBR_20241006T223202_20241007T001331_36186_03_020001_20241015T143245.nc...
Comparing file sizes...
Sizes match; product was downloaded correctly

As you can see in the code, we configure an instance of the ItemSearch class to query the endpoint. The various filtering parameters you can supply are listed in the PySTAC API reference, but of particular interest are datetime and intersects. The former allows searching for product within a time interval, the second allows searching for products that intersect with a polygon. For example:

timefilter = "2024-01-27"
items = ItemSearch(endpoint, datetime=timefilter).items()
print(f"found {len(list(items))} products for time period {timefilter}")

geofilter = {
    'type': 'Polygon',
    'coordinates': [[[6.42425537109375, 53.174765470134616], [7.344360351562499, 53.174765470134616], [7.344360351562499, 53.67393435835391], [6.42425537109375, 53.67393435835391], [6.42425537109375, 53.174765470134616]]]
}
items = ItemSearch(endpoint, datetime=timefilter, intersects=geofilter).items()
print(f"found {len(list(items))} products for time period {timefilter} when geofiltered")

found 99 products for time period 2024-01-27
found 14 products for time period 2024-01-27 when geofiltered

The first ItemSearch gives us the 59 products generated for January 27, 2024. The second ItemSearch narrows the search down to those products whose footprint intersects with the specified region.

The call to items() returns objects of type pystac.item.Item, which can then be further examined to retrieve, for example, the product download links:

print("Download links:")
for item in list(items):
    url = item.assets['download'].href
    print(f"  {url}")

Download links:
  https://data-portal.s5p-pal.com/download/s5p-l2/db9c58c3-8a69-4ded-a9af-269dde07fbb4
  https://data-portal.s5p-pal.com/download/s5p-l2/17c581e1-9d69-4e28-828c-cb315c009f75
  [...]

Item Search extensions

The S5P-PAL STAC search implementation also supports the STAC Sort and the STAC Filter fragments. These are OpenAPI extensions to the basic search query parameters accepted by the Search API.

The Sort fragment introduces the sortby parameter we already used in the 'get_most_recent_product' example above, allowing you to define fields by which to sort the query results (in that case: descending by archive_date).

The Filter fragment provides a mechanism for searching based on item attributes. For example:

items = list(ItemSearch(
    endpoint,
    filter="s5p:file_type='L2__TCWV__' and (s5p:orbit<2830 or s5p:orbit=32584)",
).items())

will select only those products matching that particular filter expression.

Finally, it it also possible for the filter to be specified as a JSON dictionary (as detailed in the Filter Fragment documentation) rather than as a text string. Our previous example would then become:

items = list(ItemSearch(
    endpoint,
    filter=
    {
        "op": "and", "args": [
            { "op": "=", "args": [ {"property": "s5p:file_type"}, "L2__TCWV__" ] },
            {
                "op": "or", "args": [
                    { "op": "<" , "args": [ {"property": "s5p:orbit"}, 2830 ] },
                    { "op": "=" , "args": [ {"property": "s5p:orbit"}, 32584 ] }
                ]
            }
        ]
    }
).items())

Note that in the text example the single quotes around string values such as the file type specification L2__TCWV__ are mandatory, whereas in the JSON version they should not be used. Similarly, in the text version, operators such as and and or are case-insensitive, whereas in the JSON version they must be lowercase.

Limitations

The PySTAC Client package also contains a command-line utility (as well as some other programmatic classes and functions) that are not currently compatible with S5P-PAL's catalogues. This may change in the future, but for now the method described above, using ItemSearch and an explicitly created endpoint are the way to approach searching. This caveat is of course specific to PySTAC and may not apply to other libraries and tools.