Skip to content

Commit

Permalink
ScuDF doc updates
Browse files Browse the repository at this point in the history
  • Loading branch information
Martin Durant committed Oct 10, 2024
1 parent f6f2ad7 commit c244f3d
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 7 deletions.
12 changes: 12 additions & 0 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,16 @@ Backends
akimbo.pandas.PandasAwkwardAccessor
akimbo.dask.DaskAwkwardAccessor
akimbo.polars.PolarsAwkwardAccessor
akimbo.cudf.CudfAwkwardAccessor

.. autoclass:: akimbo.pandas.PandasAwkwardAccessor

.. autoclass:: akimbo.dask.DaskAwkwardAccessor

.. autoclass:: akimbo.polars.PolarsAwkwardAccessor

.. autoclass:: akimbo.polars.CudfAwkwardAccessor


Extensions
~~~~~~~~~~
Expand All @@ -63,3 +66,12 @@ to see the operations available.

<script data-goatcounter="https://akimbo.goatcounter.com/count"
async src="//gc.zgo.at/count.js"></script>

The cuDF backend also has these implemented with GPU-specific variants,
``akimbo.cudf.CudfStringAccessor`` and ``akimbo.cudf.CudfDatetimeAccessor``.

Adding Extensions (advanced)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The patterns used in the two builtin extensions above can be used to add
type-specific functionality to ``akimbo``. One
25 changes: 18 additions & 7 deletions src/akimbo/apply_tree.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
from __future__ import annotations

import functools
import inspect
from typing import Sequence
from typing import Callable, Literal, Sequence

import awkward as ak
import pyarrow as pa


class NoDtype:
kind = ""


def leaf(*layout, **_):
"""True for the lowest elements of any akwward layout tree"""
return layout[0].is_leaf


Expand Down Expand Up @@ -42,8 +41,20 @@ def func(layout, **kwargs):
return ak.transform(func, arr, *others)


def dec(func, match=leaf, outtype=None, inmode="arrow"):
"""Make a nested/ragged version of an operation to apply throughout a tree"""
def dec(func: callable, match: Callable[[ak.contents.Content], bool] = leaf,
outtype: Callable[[ak.contents.Content], ak.contents.Content] | None = None,
inmode: Literal["arrow", "numpy", "ak"] = "arrow"):

"""Make a nested/ragged version of an operation to apply throughout a tree
Parameters
----------
func: which we want to apply to (parts of) inputted data
match: function to determine if a part of the data structure matches the type we want to
operate on
outtype: postprocessing function after transform
inmode: how ``func`` expects its inputs: as awkward arrays (ak), numpy or arrow
"""

@functools.wraps(func)
def f(self, *args, where=None, match_kwargs=None, **kwargs):
Expand Down

0 comments on commit c244f3d

Please sign in to comment.