Misc housekeeping

README.md

@@ -6,10 +6,10 @@
 # NxBench
 
 <p align="center">
-  <img src="doc/_static/nxbench_logo.png" alt="NxBench Logo" width="150"/>
+  <img src="doc/_static/assets/nxbench_logo.png" alt="NxBench Logo" width="150"/>
 </p>
 
-**nxbench** is a comprehensive benchmarking suite designed to facilitate comparative profiling of graph analytic algorithms across NetworkX and compatible backends. Built with an emphasis on extensibility and detailed performance analysis, nxbench aims to enable developers and researchers to optimize their graph analysis workflows efficiently and reproducibly.
+**nxbench** is a comprehensive benchmarking suite designed to facilitate comparative profiling of graph analytic algorithms across NetworkX and compatible backends. Built on top of [Airspeed Velocity (ASV)](https://github.com/airspeed-velocity/asv), nxbench places an emphasis on extensible and granular performance analysis, enabling developers and researchers to optimize their graph analysis workflows efficiently and reproducibly.
 
 ## Key Features
 
@@ -31,7 +31,13 @@ PyPi:
 pip install nxbench
 ```
 
-From a local clone, try Docker:
+From a local clone:
+
+```bash
+make install
+```
+
+Docker:
 
 ```bash
 # CPU-only
@@ -73,9 +79,14 @@ nxbench benchmark export 'results/results.csv' --output-format csv  # convert be
 4. View results:
 
 ```bash
-nxbench viz serve  # launch interactive dashboard
+nxbench viz serve  # visualize results using parallel categories dashboard
 ```
 
+<p align="center">
+  <img src="doc/_static/assets/animation.gif" alt="Parallel Categories Animation" width="1000"/>
+</p>
+
+
 ## Advanced Command Line Interface
 
 The CLI provides comprehensive management of benchmarks, datasets, and visualization:
@@ -90,7 +101,7 @@ nxbench --config 'nxbench/configs/example.yaml' -vvv benchmark run  # debug benc
 nxbench benchmark export 'results/benchmarks.sqlite' --output-format sql # export the results into a sql database
 
 # Visualization
-nxbench viz serve  # launch parallel categories dashboard
+nxbench viz serve  # visualize results using parallel categories dashboard
 nxbench viz publish  # generate static asv report
 ```
 

doc/conf.py

@@ -61,8 +61,8 @@
 html_title = "NxBench"
 html_baseurl = "https://networkx.org"
 html_copy_source = False
-html_favicon = "_static/favicon.ico"
-html_logo = "_static/nxbench_logo.png"
+html_favicon = "_static/assets/favicon.ico"
+html_logo = "_static/assets/nxbench_logo.png"
 html_theme_options = {
     # "gtag": "G-XXXXXXXXXX",
     "source_repository": "https://github.com/dpys/nxbench/",

doc/examples.md

@@ -166,27 +166,26 @@ matrix:
 
 ---
 
-## **5. ASV Configuration**
+## **5. Environment Configuration**
 
 ### **Purpose**
 
-Configures ASV-specific settings, including the repository URL, branches to benchmark, and required dependencies.
+Configures environment settings, such as the python and dependency versions.
 
 ### **Fields**
 
-- **`repo`** *(string, required)*: URL of the Git repository to benchmark.
-- **`branches`** *(list of strings, required)*: Specifies the branches in the repository to benchmark.
+- **`python`** *(list of strings, required)*: List of valid python versions (e.g. "3.10", "3.11")
 - **`req`** *(list of strings, required)*: Lists the Python dependencies required for benchmarking.
 
 ### **Example Entry**
 
 ```yaml
-asv_config:
-  repo: "https://github.com/dpys/nxbench.git"
-  branches:
-    - "main"
+env_config:
   req:
     - "networkx==3.4.2"
     - "nx_parallel==0.3"
     - "graphblas_algorithms==2023.10.0"
+  pythons:
+    - "3.10"
+    - "3.11"
 ```

doc/index.rst

@@ -3,7 +3,7 @@ Welcome to NxBench's Documentation
 
 Overview
 ========
-**nxbench** is a comprehensive benchmarking suite designed to facilitate comparative profiling of graph analytic algorithms across NetworkX and compatible backends. Built with an emphasis on extensibility and detailed performance analysis, nxbench aims to enable developers and researchers to optimize their graph analysis workflows efficiently and reproducibly.
+is a comprehensive benchmarking suite designed to facilitate comparative profiling of graph analytic algorithms across NetworkX and compatible backends. Built on top of [Airspeed Velocity (ASV)](https://github.com/airspeed-velocity/asv), nxbench places an emphasis on extensible and granular performance analysis, enabling developers and researchers to optimize their graph analysis workflows efficiently and reproducibly.
 
 Key Features
 ============

doc/installation.md

@@ -6,7 +6,13 @@ PyPi:
 pip install nxbench
 ```
 
-From a local clone, try Docker:
+From a local clone:
+
+```bash
+make install
+```
+
+Docker:
 
 ```bash
 # CPU-only

nxbench/benchmarks/benchmark.py

@@ -6,6 +6,7 @@
 import traceback
 import warnings
 from functools import partial
+from importlib import import_module
 from typing import Any
 
 import networkx as nx
@@ -14,6 +15,9 @@
 from nxbench.benchmarks.utils import (
     get_available_backends,
     get_benchmark_config,
+    is_graphblas_available,
+    is_nx_cugraph_available,
+    is_nx_parallel_available,
     memory_tracker,
 )
 from nxbench.data.loader import BenchmarkDataManager
@@ -36,7 +40,7 @@ def generate_benchmark_methods(cls):
     """Generate benchmark methods dynamically for each combination of algorithm,
     backend, and number of threads without redundant executions.
     """
-    config = get_benchmark_config()
+    config = cls.config
     algorithms = config.algorithms
     datasets = [ds.name for ds in config.datasets]
     available_backends = get_available_backends()
@@ -102,6 +106,8 @@ def track_method(self):
 class GraphBenchmark:
     """Base class for all graph algorithm benchmarks."""
 
+    config = get_benchmark_config()
+
     def __init__(self):
         self.data_manager = BenchmarkDataManager()
         self.graphs = {}
@@ -110,12 +116,11 @@ def setup_cache(self):
         """Cache graph data for benchmarks."""
         self.graphs = {}
 
-        config = get_benchmark_config()
-        datasets = [ds.name for ds in config.datasets]
+        datasets = [ds.name for ds in self.config.datasets]
 
         for dataset_name in datasets:
             dataset_config = next(
-                (ds for ds in config.datasets if ds.name == dataset_name),
+                (ds for ds in self.config.datasets if ds.name == dataset_name),
                 None,
             )
             if dataset_config is None:
@@ -160,37 +165,52 @@ def prepare_benchmark(
             f"{original_graph.number_of_edges()} edges"
         )
 
-        try:
-            if backend == "networkx":
-                converted_graph = original_graph
-            elif "parallel" in backend:
-                os.environ["NUM_THREAD"] = str(num_thread)
-                os.environ["OMP_NUM_THREADS"] = str(num_thread)
-                os.environ["MKL_NUM_THREADS"] = str(num_thread)
-                os.environ["OPENBLAS_NUM_THREADS"] = str(num_thread)
-
-                nx.config.backends.parallel.active = True
-                nx.config.backends.parallel.n_jobs = num_thread
-                converted_graph = original_graph
-            elif "cugraph" in backend:
-                import cugraph
+        for var_name in [
+            "NUM_THREAD",
+            "OMP_NUM_THREADS",
+            "MKL_NUM_THREADS",
+            "OPENBLAS_NUM_THREADS",
+        ]:
+            os.environ[var_name] = str(num_thread)
+
+        if backend == "networkx":
+            return original_graph
+
+        if "parallel" in backend and is_nx_parallel_available():
+            try:
+                nxp = import_module("nx_parallel")
+            except ImportError:
+                logger.exception("nx-parallel backend not available")
+                return None
+            return nxp.ParallelGraph(original_graph)
 
+        if "cugraph" in backend and is_nx_cugraph_available():
+            try:
+                cugraph = import_module("nx_cugraph")
+            except ImportError:
+                logger.exception("cugraph backend not available")
+                return None
+            try:
                 edge_attr = "weight" if nx.is_weighted(original_graph) else None
-                converted_graph = cugraph.from_networkx(
-                    original_graph, edge_attrs=edge_attr
-                )
-            elif "graphblas" in backend:
-                import graphblas_algorithms as ga
+                return cugraph.from_networkx(original_graph, edge_attrs=edge_attr)
+            except Exception:
+                logger.exception("Error converting graph to cugraph format")
+                return None
 
-                converted_graph = ga.Graph.from_networkx(original_graph)
-            else:
-                logger.error(f"Unsupported backend: {backend}")
+        if "graphblas" in backend and is_graphblas_available():
+            try:
+                ga = import_module("graphblas_algorithms")
+            except ImportError:
+                logger.exception("graphblas_algorithms backend not available")
+                return None
+            try:
+                return ga.Graph.from_networkx(original_graph)
+            except Exception:
+                logger.exception("Error converting graph to graphblas format")
                 return None
-        except Exception:
-            logger.exception("Error in prepare_benchmark")
-            return None
         else:
-            return converted_graph
+            logger.error(f"Unsupported backend: {backend}")
+            return None
 
     def do_benchmark(
         self,
@@ -210,11 +230,10 @@ def do_benchmark(
 
         try:
             algo_func = get_algorithm_function(algo_config, backend)
-            alg_func_name = (
-                algo_func.func.__name__
-                if hasattr(algo_func, "func")
-                else algo_func.__name__
-            )
+            if isinstance(algo_func, partial):
+                alg_func_name = algo_func.func.__name__
+            else:
+                alg_func_name = algo_func.__name__
             logger.debug(f"Got algorithm function: {alg_func_name}")
         except (ImportError, AttributeError):
             logger.exception(f"Function not available for backend {backend}")

nxbench/benchmarks/config.py

@@ -94,7 +94,8 @@ class BenchmarkConfig:
     datasets: list[DatasetConfig]
     matrix: dict[str, Any]
     machine_info: dict[str, Any] = field(default_factory=dict)
-    output_dir: Path = field(default_factory=lambda: Path("../results"))
+    output_dir: Path = field(default_factory=lambda: Path("~/results"))
+    env_data: dict[str, Any] = field(default_factory=dict)
 
     @classmethod
     def from_yaml(cls, path: str | Path) -> "BenchmarkConfig":
@@ -134,6 +135,8 @@ def from_yaml(cls, path: str | Path) -> "BenchmarkConfig":
             logger.error(f"'matrix' should be a dict in the config file: {path}")
             matrix_data = {}
 
+        env_data = data.get("env_config") or {}
+
         algorithms = [AlgorithmConfig(**algo_data) for algo_data in algorithms_data]
 
         datasets = [DatasetConfig(**ds_data) for ds_data in datasets_data]
@@ -143,7 +146,8 @@ def from_yaml(cls, path: str | Path) -> "BenchmarkConfig":
             datasets=datasets,
             matrix=matrix_data,
             machine_info=data.get("machine_info", {}),
-            output_dir=Path(data.get("output_dir", "../results")),
+            output_dir=Path(data.get("output_dir", "~/results")),
+            env_data=env_data,
         )
 
     def to_yaml(self, path: str | Path) -> None: