Merge branch 'develop'

CHANGES.rst

@@ -1,6 +1,26 @@
 Release history
 ---------------
 
+3.0.0b2 (2024-06-29)
+++++++++++++++++++++
+
+Core features and fixes
+^^^^^^^^^^^^^^^^^^^^^^^
+
+* New :class:`~eodag.api.search_result.SearchResult` HTML representation for notebooks (:pull:`1243`)
+
+Plugins new features and fixes
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* Fixed missing ``products`` configuration in ``Api`` plugin download (:pull:`1241`)
+* Fixed ``pagination`` configuration to be not allways mandatory (:pull:`1240`)
+
+Miscellaneous
+^^^^^^^^^^^^^
+
+* **[docs]** Custom mock search plugin example (:pull:`1242`)
+* External product types reference updates (:pull:`1234`)
+
 3.0.0b1 (2024-06-24)
 ++++++++++++++++++++
 
@@ -63,7 +83,7 @@ Plugins new features and fixes
 * Refresh token for :class:`~eodag.plugins.authentication.openid_connect.OIDCAuthorizationCodeFlowAuth` plugin
   (:pull:`1138`), tests (:pull:`1135`), and fix (:pull:`1232`)
 * :class:`~eodag.plugins.authentication.header.HTTPHeaderAuth` accepts headers definition in credentials (:pull:`1215`)
-* ``flatten_top_dirs`` download plugins option set to true by default (#1220)
+* ``flatten_top_dirs`` download plugins option set to true by default (:pull:`1220`)
 * ``base_uri`` download plugins setting is not systematically mandatory any more (:pull:`1230`)
 * Re-login in :class:`~eodag.plugins.apis.usgs.UsgsApi` plugin on api file error (:pull:`1046`)
 * Allow no auth for :class:`~eodag.plugins.download.http.HTTPDownload` download requests (:pull:`1196`)

README.rst

@@ -183,7 +183,7 @@ An eodag instance can be exposed through a STAC compliant REST api from the comm
 
 .. code-block:: bash
 
-    docker run -p 5000:5000 --rm csspace/eodag-server:3.0.0b1
+    docker run -p 5000:5000 --rm csspace/eodag-server:3.0.0b2
 
 You can also browse over your STAC API server using `STAC Browser <https://github.com/radiantearth/stac-browser>`_.
 Simply run:

charts/eodag-server/Chart.yaml

@@ -1,5 +1,5 @@
 apiVersion: v2
-appVersion: 3.0.0b1
+appVersion: 3.0.0b2
 dependencies:
 - name: common
   repository:  oci://registry-1.docker.io/bitnamicharts
@@ -15,4 +15,4 @@ name: eodag-server
 sources:
 - https://github.com/CS-SI/eodag
 type: application
-version: 3.0.0b1
+version: 3.0.0b2

docs/conf.py

@@ -40,6 +40,7 @@
     "sphinx.ext.viewcode",
     "nbsphinx",
     "sphinx_copybutton",
+    "sphinx_tabs.tabs",
 ]
 
 # Notebook integration parameters

docs/plugins.rst

@@ -108,41 +108,121 @@ named ``your_package.your_module``:
       def download(self):
          pass
 
-Then, in the `setup.py <https://setuptools.readthedocs.io/en/latest/userguide/quickstart.html#id2>`_ of your Python project,
+Then, in the `configuration file <https://setuptools.readthedocs.io/en/latest/userguide/quickstart.html#id2>`_ of your Python project,
 add this:
 
-.. code-block:: python
 
-   setup(
-      ...
-      entry_points={
-         ...
-         'eodag.plugins.api': [
-            'SampleApiPlugin = your_package.your_module:SampleApiPlugin'
-         ],
-         ...
-      },
-      ...
-   )
+.. tabs::
+   .. tab:: pyproject.toml
+      .. code-block:: toml
+
+         [project.entry-points."eodag.plugins.api"]
+         SampleApiPlugin = "your_package.your_module:SampleApiPlugin"
+
+   .. tab:: setup.py
+      .. code-block:: python
+
+         setup(
+            # ...
+            entry_points={
+               'eodag.plugins.api': [
+                  'SampleApiPlugin = your_package.your_module:SampleApiPlugin'
+               ]
+            }
+         )
 
 See `what the PyPa explains <https://packaging.python.org/guides/creating-and-discovering-plugins/#using-package-metadata>`_ to better
 understand this concept. In EODAG, the name you give to your plugin in the
-`setup.py` script's entry point doesn't matter, but we prefer it to be the
-same as the class name of the plugin. What matters is that the entry point
-must be a class deriving from one of the 5 plugin topics supported. Be
-particularly careful with consistency between the entry point name and the
-super class of you plugin class. Here is a list of entry point names and the
-plugin topic to which they map:
+entry point doesn't matter, but we prefer it to be the same as the class
+name of the plugin. What matters is that the entry point must be a class
+deriving from one of the 5 plugin topics supported. Be particularly careful
+with consistency between the entry point name and the super class of you
+plugin class. Here is a list of entry point names and the plugin topic to
+which they map:
 
 * 'eodag.plugins.api'      : :class:`~eodag.plugins.apis.base.Api`
 * 'eodag.plugins.auth'     : :class:`~eodag.plugins.auth.base.Authentication`
 * 'eodag.plugins.crunch'   : :class:`~eodag.plugins.crunch.base.Crunch`
 * 'eodag.plugins.download' : :class:`~eodag.plugins.download.base.Download`
 * 'eodag.plugins.search'   : :class:`~eodag.plugins.search.base.Search`
 
+
+Read the :ref:`mock search plugin` section to get an example of how to build a search
+plugin.
+
 As an example of external plugin, you can have a look to
 `eodag-sentinelsat <https://github.com/CS-SI/eodag-sentinelsat>`_.
 
+.. _mock search plugin:
+
+A sample mock search plugin
+"""""""""""""""""""""""""""
+
+In this section, we demonstrate the creation of custom search plugin. This plugin will
+return a list of mocked EOProducts on search requests.
+
+1. Create a folder called `eodag_search_mock`. It will be the name of our package.
+
+2. Make an empty `__init.py__` file in the package folder.
+
+3. Make a `mock_search.py` file and paste in it the following class
+   definition.
+
+   It is our search plugin. The `query()` function is the one called by EODAG when
+   searching for products.
+
+   .. code-block:: python
+
+      # mock_search.py
+      from typing import Any, List, Optional, Tuple
+      from eodag.plugins.search import PreparedSearch
+      from eodag.plugins.search.base import Search
+      from eodag.api.product import EOProduct
+
+      class MockSearch(Search):
+      """Implement a Mock search plugin"""
+
+      def query(
+         self,
+         prep: PreparedSearch = PreparedSearch(),
+         **kwargs: Any,
+      ) -> Tuple[List[EOProduct], Optional[int]]:
+         """Generate EOProduct from the request"""
+         return ([EOProduct(
+            provider=self.provider,
+            properties={
+               **{
+                  "id": f"mock_{kwargs.get('productType')}_{i}"
+                  },
+               **kwargs
+            }
+            ) for i in range(0, prep.items_per_page)],
+            prep.items_per_page if prep.count else None
+         )
+
+4. Create a `pyproject.toml` file in the package folder and paste in it the following
+   content.
+
+   The `projects.entry-points` is crucial for EODAG to detect this new plugin.
+
+   .. code-block:: toml
+
+      [project]
+      name = "eodag-search-mock"
+      version = "0.0.1"
+      description = "Mock Search plugin for EODAG"
+      requires-python = ">=3.8"
+      dependencies = [ "eodag" ]
+
+      [project.entry-points."eodag.plugins.search"]
+      MockSearch = "mock_search:MockSearch"
+
+5. Your plugin is now ready. You need to install it for EODAG to be able to use it.
+
+   .. code-block:: shell
+
+      pip install eodag_search_mock
+
 Provider configuration
 ^^^^^^^^^^^^^^^^^^^^^^
 

docs/stac_rest.rst

@@ -120,7 +120,7 @@ available on `https://hub.docker.com/r/csspace/eodag-server <https://hub.docker.
 
 .. code-block:: bash
 
-    $ docker run -p 5000:5000 --rm csspace/eodag-server:3.0.0b1
+    $ docker run -p 5000:5000 --rm csspace/eodag-server:3.0.0b2
 
 Example
 -------

eodag/api/core.py

@@ -1445,8 +1445,12 @@ def search_all(
             start=start, end=end, geom=geom, locations=locations, **kwargs
         )
         for i, search_plugin in enumerate(search_plugins):
-            itp = items_per_page or search_plugin.config.pagination.get(
-                "max_items_per_page", DEFAULT_MAX_ITEMS_PER_PAGE
+            itp = (
+                items_per_page
+                or getattr(search_plugin.config, "pagination", {}).get(
+                    "max_items_per_page"
+                )
+                or DEFAULT_MAX_ITEMS_PER_PAGE
             )
             logger.debug(
                 "Searching for all the products with provider %s and a maximum of %s "

eodag/api/product/_assets.py

@@ -22,6 +22,7 @@
 from typing import TYPE_CHECKING, Any, Dict, List, Optional
 
 from eodag.utils.exceptions import NotAvailableError
+from eodag.utils.repr import dict_to_html_table
 
 if TYPE_CHECKING:
     from eodag.api.product import EOProduct
@@ -84,6 +85,43 @@ def get_values(self, asset_filter: str = "") -> List[Asset]:
         else:
             return [a for a in self.values() if "href" in a]
 
+    def _repr_html_(self, embeded=False):
+        thead = (
+            f"""<thead><tr><td style='text-align: left; color: grey;'>
+                {type(self).__name__}&ensp;({len(self)})
+                </td></tr></thead>
+            """
+            if not embeded
+            else ""
+        )
+        tr_style = "style='background-color: transparent;'" if embeded else ""
+        return (
+            f"<table>{thead}"
+            + "".join(
+                [
+                    f"""<tr {tr_style}><td style='text-align: left;'>
+                <details><summary style='color: grey;'>
+                    <span style='color: black'>'{k}'</span>:&ensp;
+                    {{
+                        {"'roles': '<span style='color: black'>"+str(v['roles'])+"</span>',&ensp;"
+                            if v.get("roles") else ""}
+                        {"'type': '"+str(v['type'])+"',&ensp;"
+                            if v.get("type") else ""}
+                        {"'title': '<span style='color: black'>"+str(v['title'])+"</span>',&ensp;"
+                            if v.get("title") else ""}
+                        ...
+                    }}
+                </summary>
+                    {dict_to_html_table(v, depth=1)}
+                </details>
+                </td></tr>
+                """
+                    for k, v in self.items()
+                ]
+            )
+            + "</table>"
+        )
+
 
 class Asset(UserDict):
     """A UserDict object containg one of the assets of a
@@ -127,3 +165,15 @@ def download(self, **kwargs: Unpack[DownloadConf]) -> str:
         :rtype: str
         """
         return self.product.download(asset=self.key, **kwargs)
+
+    def _repr_html_(self):
+        thead = f"""<thead><tr><td style='text-align: left; color: grey;'>
+            {type(self).__name__}&ensp;-&ensp;{self.key}
+            </td></tr></thead>
+        """
+        return f"""<table>{thead}
+                <tr><td style='text-align: left;'>
+                    {dict_to_html_table(self)}
+                </details>
+                </td></tr>
+            </table>"""

eodag/api/product/_product.py

@@ -51,6 +51,7 @@
     get_geometry_from_various,
 )
 from eodag.utils.exceptions import DownloadError, MisconfiguredError
+from eodag.utils.repr import dict_to_html_table
 
 if TYPE_CHECKING:
     from shapely.geometry.base import BaseGeometry
@@ -517,3 +518,45 @@ def get_driver(self) -> DatasetDriver:
             )
             pass
         return NoDriver()
+
+    def _repr_html_(self):
+        thumbnail = self.properties.get("thumbnail", None)
+        thumbnail_html = (
+            f"<img src='{thumbnail}' width=100 alt='thumbnail'/>"
+            if thumbnail and not thumbnail.startswith("s3")
+            else ""
+        )
+        geom_style = "style='color: grey; text-align: center; min-width:100px; vertical-align: top;'"
+        thumbnail_style = (
+            "style='padding-top: 1.5em; min-width:100px; vertical-align: top;'"
+        )
+
+        return f"""<table>
+                <thead><tr style='background-color: transparent;'><td style='text-align: left; color: grey;'>
+                {type(self).__name__}
+                </td></tr></thead>
+
+                <tr style='background-color: transparent;'>
+                    <td style='text-align: left; vertical-align: top;'>
+                        {dict_to_html_table({
+                            "provider": self.provider,
+                            "product_type": self.product_type,
+                            "properties[&quot;id&quot;]": self.properties.get('id', None),
+                            "properties[&quot;startTimeFromAscendingNode&quot;]": self.properties.get(
+                                'startTimeFromAscendingNode', None
+                            ),
+                            "properties[&quot;completionTimeFromAscendingNode&quot;]": self.properties.get(
+                                'completionTimeFromAscendingNode', None
+                            ),
+                        }, brackets=False)}
+                        <details><summary style='color: grey; margin-top: 10px;'>properties:&ensp;({
+                            len(self.properties)
+                        })</summary>{dict_to_html_table(self.properties, depth=1)}</details>
+                        <details><summary style='color: grey; margin-top: 10px;'>assets:&ensp;({
+                            len(self.assets)
+                        })</summary>{self.assets._repr_html_(embeded=True)}</details>
+                    </td>
+                    <td {geom_style} title='geometry'>geometry<br />{self.geometry._repr_svg_()}</td>
+                    <td {thumbnail_style} title='properties[&quot;thumbnail&quot;]'>{thumbnail_html}</td>
+                </tr>
+            </table>"""

eodag/api/search_result.py

@@ -174,6 +174,33 @@ def __geo_interface__(self) -> Dict[str, Any]:
         """
         return self.as_geojson_object()
 
+    def _repr_html_(self):
+        total_count = f"/{self.number_matched}" if self.number_matched else ""
+        return (
+            f"""<table>
+                <thead><tr><td style='text-align: left; color: grey;'>
+                 {type(self).__name__}&ensp;({len(self)}{total_count})
+                </td></tr></thead>
+            """
+            + "".join(
+                [
+                    f"""<tr><td style='text-align: left;'>
+                <details><summary style='color: grey; font-family: monospace;'>
+                    {i}&ensp;
+                    {type(p).__name__}(id=<span style='color: black;'>{
+                        p.properties['id']
+                    }</span>, provider={p.provider})
+                </summary>
+                {p._repr_html_()}
+                </details>
+                </td></tr>
+                """
+                    for i, p in enumerate(self)
+                ]
+            )
+            + "</table>"
+        )
+
 
 class RawSearchResult(UserList):
     """An object representing a collection of raw/unparsed search results obtained from a provider.

eodag/plugins/manager.py

@@ -242,6 +242,7 @@ def get_download_plugin(self, product: EOProduct) -> Union[Download, Api]:
                 self._build_plugin(product.provider, download, Download),
             )
         elif api := getattr(plugin_conf, "api", None):
+            plugin_conf.api.products = plugin_conf.products
             plugin_conf.api.priority = plugin_conf.priority
             plugin = cast(Api, self._build_plugin(product.provider, api, Api))
         else: