Draft JOSS article about SciPy elementwise iterative methods

paper.bib

@@ -0,0 +1,82 @@
+@misc{numdifftools,
+  author = {Per A. Brodtkorb},
+  title = {numdifftools 0.9.41},
+  year = {2022},
+  publisher = {GitHub},
+  url = {https://github.com/pbrod/numdifftools}
+}
+
+@misc{quadpy,
+  author = {Nico Schl{\"o}mer},
+  title = {quadpy},
+  year = {2023},
+  publisher = {GitHub},
+  url = {https://github.com/sigma-py/quadpy}
+}
+
+@misc{arrayapi,
+  author = {{Consortium for Python Data API Standards},
+  title = {Python Array {API} Standard,
+  year = {2024},
+  url = {https://data-apis.org/array-api/latest/}
+}
+
+@inproceedings{pytorch,
+  title={Py{T}orch 2: Faster machine learning through dynamic {P}ython bytecode transformation and graph compilation},
+  author={Ansel, Jason and Yang, Edward and He, Horace and Gimelshein, Natalia and Jain, Animesh and Voznesensky, Michael and Bao, Bin and Bell, Peter and Berard, David and Burovski, Evgeni and others},
+  booktitle={Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
+  pages={929--947},
+  year={2024},
+  doi={10.1145/3620665.3640366},
+}
+
+@inproceedings{cupy,
+  title={Cu{Y}y: A {N}um{P}y-compatible library for {NVIDIA GPU} calculations},
+  author={Okuta, Ryosuke and Unno, Yuya and Nishino, Daisuke and Hido, Shohei and Loomis, Crissman},
+  journal={Proceedings of Workshop on Machine Learning Systems ({L}earning{S}ys) in The Thirty-first Annual Conference on Neural Information Processing Systems {NIPS}},
+  year={2017}
+}
+
+@misc{mpmath,
+  author  = {{The mpmath development team}},
+  title   = {{mpmath}: a {P}ython library for arbitrary-precision floating-point arithmetic},
+  year    = {2023},
+  url = {http://mpmath.org/}
+}
+
+@article{numpy,
+  title={Array programming with NumPy},
+  author={Harris, Charles R and Millman, K Jarrod and Van Der Walt, St{\'e}fan J and Gommers, Ralf and Virtanen, Pauli and Cournapeau, David and Wieser, Eric and Taylor, Julian and Berg, Sebastian and Smith, Nathaniel J and others},
+  doi = {10.1038/s41586-020-2649-2},
+  journal={Nature},
+  volume={585},
+  number={7825},
+  pages={357--362},
+  year={2020},
+  publisher={Nature Publishing Group}
+}
+
+@article{scipy,
+  title={{S}ci{P}y 1.0: fundamental algorithms for scientific computing in {P}ython},
+  author={Virtanen, Pauli and Gommers, Ralf and Oliphant, Travis E and Haberland, Matt and Reddy, Tyler and others},
+  doi = {10.1038/s41592-019-0686-2},
+  journal={Nature methods},
+  volume={17},
+  number={3},
+  pages={261--272},
+  year={2020},
+  publisher={Nature Publishing Group}
+}
+
+@article{PyNumDiff, 
+  doi = {10.21105/joss.04078},
+  url = {https://doi.org/10.21105/joss.04078},
+  year = {2022},
+  publisher = {The Open Journal},
+  volume = {7},
+  number = {71},
+  pages = {4078},
+  author = {Floris Van Breugel and Yuying Liu and Bingni W. Brunton and J. Nathan Kutz},
+  title = {PyNumDiff: A Python package for numerical differentiation of noisy time-series data}, 
+  journal = {Journal of Open Source Software}
+}

paper.md

@@ -0,0 +1,60 @@
+---
+title: 'Vectorized, Python Array API Standard Compatible Functions for Quadrature, Series Summmation, Differentiation, Optimization, and Root Finding in SciPy'
+tags:
+  - Python
+  - SciPy
+  - integration
+  - differentiation
+  - optimization
+  - root finding
+authors:
+  - name: Matt Haberland^[Corresponding author]
+    affiliation: 1
+    orcid: 0000-0003-4806-3601
+  - name: Albert Steppi
+    affiliation: 2
+    orcid: 0000-0001-5871-6245
+  - name: Pamphile Roy
+    affiliation: 3
+    orcid: 0000-0001-9816-1416
+affiliations:
+ - name: California Polytechnic State University, San Luis Obispo, USA
+   index: 1
+ - name: Quansight Labs, Austin, USA
+   index: 2
+ - name: Consulting Manao GMBH, Vienna, Austria
+   index: 3
+date: 1 March 2025
+bibliography: paper.bib
+
+---
+
+# Summary
+
+Numerical integration, series summation, differentiation, optimization, and root finding are fundamental problems with applications in essentially all domains of science and engineering. Frequently, such problems do not arise individually, but rather in batches; e.g., differentiation of a single curve at many points or minimization of a function for many values of a parameter. When the related problems are all independent, embarrassingly parallel calculation is possible However, SciPy [@scipy] – the de facto standard Python library for such algorithms – did not offer features for vectorized solution of such problems with NumPy [@numpy] arrays, let alone array library agnostic implementations.
+
+This paper discusses several new features that fill this gap, included in SciPy 1.15.0:
+
+- `scipy.differentiate`, a sub-package for numerical differentiation of scalar or vector-valued functions of one or more variables,
+- `scipy.integrate.tanhsinh` for quadrature of scalar or vector-valued integrands of one variable,
+- `scipy.integrate.nsum` for summation of real-valued finite or infinite series,
+- `scipy.optimize.elementwise` for finding roots and minimization of a single-input, single output function.
+
+Although the details of the algorithms are inherently distinct, these features rely on a common framework for elementwise iterative methods and share similar interfaces. For example, `scipy.differentiate.derivative` accepts the scalar function to be differentiated as a Python callable, the points at which to evaluate the derivative as an N-dimensional array, and any other parameters of the callable as N-dimensional arrays. `derivative` passes abscissae and parameters to the callable as N-dimensional arrays, adaptively approximating the derivative(s) to achieve specified tolerances. Furthermore, arrays need not be NumPy arrays: the same implementation works with several Python Array API Standard [@arrayapi] compatible arrays, such as PyTorch [@pytorch] tensors and CuPy [@cupy] arrays. These features will dramatically improve performance of end-user applications, and together, they form the backbone of SciPy's new random variable infrastructure.
+
+# Statement of need
+
+Before the release of SciPy 1.15.0, the need for these capabilities was partially met in the scientific Python ecosystem. As popular examples, Numdifftools [@numdifftools] and PyNumDiff [@PyNumDiff] provide tools for numerical differentiation, `quadpy` [@quadpy] and `scipy.integrate` offer several functions for numerical integration, `mpmath`'s [@mpmath] `nsum` function is for series summation, and `scipy.optimize` offers functions for scalar minimization and root finding. However, to the authors' knowledge, the new implementations are unique in that they offer the following advantages.
+
+- Backend-independence: all of these features are "Array API compatible" in the sense that they rely almost entirely on the Python standard libary and calls to Python Array API Standard functions/methods. Consequently, the functions accept objects other than NumPy arrays, such as PyTorch tensors and CuPy arrays, and the calculations are performed by the underlying array library.
+- Speed: the features take full advantage of vectorized user callables, avoiding slow Python loops and the excessive overhead of repeatedly calling compiled code. 
+- Prevalence: SciPy is one of the most popular scientific Python packages. If a scientific Python user needs these features, chances are that they already have SciPy installed, eliminating the need to find and learn a new package.
+- Ease-of-use: the function API reference is thorough, and the interfaces share common features and operate smoothly with other SciPy functions.
+- Dependability: as with all SciPy code, these features were designed and implemented following good software development practices. They have been carefully peer-reviewed, and extensive unit tests protect against backward incompatible changes and regressions.
+
+# Acknowledgements
+
+We gratefully acknowledge the support of Chan Zuckerberg Initiative Essential
+Open Source Software for Science Grant CZI EOSS5-0000000176, "SciPy: Fundamental Tools for Biomedical Research". Thanks to all those who have submitted feature requests, bug reports, and review comments related to these features, and especially to SciPy maintainers who reviewed and merged related work.
+
+# References