Skip to content

Commit

Permalink
Update docs and remove cleanup other files
Browse files Browse the repository at this point in the history
  • Loading branch information
bkarsin committed Jan 7, 2025
1 parent e8f9c25 commit e05882f
Show file tree
Hide file tree
Showing 4 changed files with 350 additions and 2 deletions.
230 changes: 229 additions & 1 deletion cpp/include/cuvs/neighbors/vamana.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -215,34 +215,198 @@ struct index : cuvs::neighbors::index {
* @{
*/
/**
* @brief Build the index from the dataset for efficient search.
* @brief Build the index from the dataset for efficient DiskANN search.
*
* The build utilies the Vamana insertion-based algorithm to create the graph. The algorithm
* starts with an empty graph and iteratively iserts batches of nodes. Each batch involves
* performing a greedy search for each vector to be inserted, and inserting it with edges to
* all nodes traversed during the search. Reverse edges are also inserted and robustPrune is applied
* to improve graph quality. The index_params struct controls the degree of the final graph.
*
* The following distance metrics are supported:
* - L2
*
* Usage example:
* @code{.cpp}
* using namespace cuvs::neighbors;
* // use default index parameters;
* vamana::index_params index_params;
* // create and fill index from a [N, D] dataset;
* auto index = vamana::build(res, index_params, dataset);
* // write index to file to be used by CPU-based DiskANN search (cuVS does not yet support search)
* vamana::serialize(res, filename, index);
*
* @param[in] res
* @param[in] params parameters for building the index
* @param[in] dataset a matrix view (device) to a row-major matrix [n_rows, dim]
*
* @return the constructed vamana index
*/
auto build(raft::resources const& handle,
const cuvs::neighbors::vamana::index_params& params,
raft::device_matrix_view<const float, int64_t, raft::row_major> dataset)
-> cuvs::neighbors::vamana::index<float, uint32_t>;

/**
* @brief Build the index from the dataset for efficient DiskANN search.
*
* The build utilies the Vamana insertion-based algorithm to create the graph. The algorithm
* starts with an empty graph and iteratively iserts batches of nodes. Each batch involves
* performing a greedy search for each vector to be inserted, and inserting it with edges to
* all nodes traversed during the search. Reverse edges are also inserted and robustPrune is applied
* to improve graph quality. The index_params struct controls the degree of the final graph.
*
* The following distance metrics are supported:
* - L2
*
* Usage example:
* @code{.cpp}
* using namespace cuvs::neighbors;
* // use default index parameters;
* vamana::index_params index_params;
* // create and fill index from a [N, D] dataset;
* auto index = vamana::build(res, index_params, dataset);
* // write index to file to be used by CPU-based DiskANN search (cuVS does not yet support search)
* vamana::serialize(res, filename, index);
*
* @param[in] res
* @param[in] params parameters for building the index
* @param[in] dataset a matrix view (host) to a row-major matrix [n_rows, dim]
*
* @return the constructed vamana index
*/
auto build(raft::resources const& handle,
const cuvs::neighbors::vamana::index_params& params,
raft::host_matrix_view<const float, int64_t, raft::row_major> dataset)
-> cuvs::neighbors::vamana::index<float, uint32_t>;

/**
* @brief Build the index from the dataset for efficient DiskANN search.
*
* The build utilies the Vamana insertion-based algorithm to create the graph. The algorithm
* starts with an empty graph and iteratively iserts batches of nodes. Each batch involves
* performing a greedy search for each vector to be inserted, and inserting it with edges to
* all nodes traversed during the search. Reverse edges are also inserted and robustPrune is applied
* to improve graph quality. The index_params struct controls the degree of the final graph.
*
* The following distance metrics are supported:
* - L2
*
* Usage example:
* @code{.cpp}
* using namespace cuvs::neighbors;
* // use default index parameters;
* vamana::index_params index_params;
* // create and fill index from a [N, D] dataset;
* auto index = vamana::build(res, index_params, dataset);
* // write index to file to be used by CPU-based DiskANN search (cuVS does not yet support search)
* vamana::serialize(res, filename, index);
*
* @param[in] res
* @param[in] params parameters for building the index
* @param[in] dataset a matrix view (device) to a row-major matrix [n_rows, dim]
*
* @return the constructed vamana index
*/
auto build(raft::resources const& handle,
const cuvs::neighbors::vamana::index_params& params,
raft::device_matrix_view<const int8_t, int64_t, raft::row_major> dataset)
-> cuvs::neighbors::vamana::index<int8_t, uint32_t>;

/**
* @brief Build the index from the dataset for efficient DiskANN search.
*
* The build utilies the Vamana insertion-based algorithm to create the graph. The algorithm
* starts with an empty graph and iteratively iserts batches of nodes. Each batch involves
* performing a greedy search for each vector to be inserted, and inserting it with edges to
* all nodes traversed during the search. Reverse edges are also inserted and robustPrune is applied
* to improve graph quality. The index_params struct controls the degree of the final graph.
*
* The following distance metrics are supported:
* - L2
*
* Usage example:
* @code{.cpp}
* using namespace cuvs::neighbors;
* // use default index parameters;
* vamana::index_params index_params;
* // create and fill index from a [N, D] dataset;
* auto index = vamana::build(res, index_params, dataset);
* // write index to file to be used by CPU-based DiskANN search (cuVS does not yet support search)
* vamana::serialize(res, filename, index);
*
* @param[in] res
* @param[in] params parameters for building the index
* @param[in] dataset a matrix view (host) to a row-major matrix [n_rows, dim]
*
* @return the constructed vamana index
*/
auto build(raft::resources const& handle,
const cuvs::neighbors::vamana::index_params& params,
raft::host_matrix_view<const int8_t, int64_t, raft::row_major> dataset)
-> cuvs::neighbors::vamana::index<int8_t, uint32_t>;

/**
* @brief Build the index from the dataset for efficient DiskANN search.
*
* The build utilies the Vamana insertion-based algorithm to create the graph. The algorithm
* starts with an empty graph and iteratively iserts batches of nodes. Each batch involves
* performing a greedy search for each vector to be inserted, and inserting it with edges to
* all nodes traversed during the search. Reverse edges are also inserted and robustPrune is applied
* to improve graph quality. The index_params struct controls the degree of the final graph.
*
* The following distance metrics are supported:
* - L2
*
* Usage example:
* @code{.cpp}
* using namespace cuvs::neighbors;
* // use default index parameters;
* vamana::index_params index_params;
* // create and fill index from a [N, D] dataset;
* auto index = vamana::build(res, index_params, dataset);
* // write index to file to be used by CPU-based DiskANN search (cuVS does not yet support search)
* vamana::serialize(res, filename, index);
*
* @param[in] res
* @param[in] params parameters for building the index
* @param[in] dataset a matrix view (device) to a row-major matrix [n_rows, dim]
*
* @return the constructed vamana index
*/
auto build(raft::resources const& handle,
const cuvs::neighbors::vamana::index_params& params,
raft::device_matrix_view<const uint8_t, int64_t, raft::row_major> dataset)
-> cuvs::neighbors::vamana::index<uint8_t, uint32_t>;

/**
* @brief Build the index from the dataset for efficient DiskANN search.
*
* The build utilies the Vamana insertion-based algorithm to create the graph. The algorithm
* starts with an empty graph and iteratively iserts batches of nodes. Each batch involves
* performing a greedy search for each vector to be inserted, and inserting it with edges to
* all nodes traversed during the search. Reverse edges are also inserted and robustPrune is applied
* to improve graph quality. The index_params struct controls the degree of the final graph.
*
* The following distance metrics are supported:
* - L2
*
* Usage example:
* @code{.cpp}
* using namespace cuvs::neighbors;
* // use default index parameters;
* vamana::index_params index_params;
* // create and fill index from a [N, D] dataset;
* auto index = vamana::build(res, index_params, dataset);
* // write index to file to be used by CPU-based DiskANN search (cuVS does not yet support search)
* vamana::serialize(res, filename, index);
*
* @param[in] res
* @param[in] params parameters for building the index
* @param[in] dataset a matrix view (host) to a row-major matrix [n_rows, dim]
*
* @return the constructed vamana index
*/
auto build(raft::resources const& handle,
const cuvs::neighbors::vamana::index_params& params,
raft::host_matrix_view<const uint8_t, int64_t, raft::row_major> dataset)
Expand All @@ -252,18 +416,82 @@ auto build(raft::resources const& handle,
* @defgroup vamana_cpp_serialize Vamana serialize functions
* @{
*/

/**
* Save the index to file.
*
* Matches the file format used by the DiskANN open-source repository, allowing cross-compatabilty.
*
* @code{.cpp}
* #include <raft/core/resources.hpp>
* #include <cuvs/neighbors/vamana.hpp>
*
* raft::resources handle;
*
* // create a string with a filepath
* std::string file_prefix("/path/to/index/prefix");
* // create an index with `auto index = cuvs::neighbors::vamana::build(...);`
* cuvs::neighbors::vamana::serialize(handle, file_prefix, index);
* @endcode
*
* @param[in] handle the raft handle
* @param[in] file_prefix prefix of path and name of index files
* @param[in] index Vamana index
*
*/

void serialize(raft::resources const& handle,
const std::string& file_prefix,
const cuvs::neighbors::vamana::index<float, uint32_t>& index);

/**
* Save the index to file.
*
* Matches the file format used by the DiskANN open-source repository, allowing cross-compatabilty.
*
* @code{.cpp}
* #include <raft/core/resources.hpp>
* #include <cuvs/neighbors/vamana.hpp>
*
* raft::resources handle;
*
* // create a string with a filepath
* std::string file_prefix("/path/to/index/prefix");
* // create an index with `auto index = cuvs::neighbors::vamana::build(...);`
* cuvs::neighbors::vamana::serialize(handle, file_prefix, index);
* @endcode
*
* @param[in] handle the raft handle
* @param[in] file_prefix prefix of path and name of index files
* @param[in] index Vamana index
*
*/
void serialize(raft::resources const& handle,
const std::string& file_prefix,
const cuvs::neighbors::vamana::index<int8_t, uint32_t>& index);

/**
* Save the index to file.
*
* Matches the file format used by the DiskANN open-source repository, allowing cross-compatabilty.
*
* @code{.cpp}
* #include <raft/core/resources.hpp>
* #include <cuvs/neighbors/vamana.hpp>
*
* raft::resources handle;
*
* // create a string with a filepath
* std::string file_prefix("/path/to/index/prefix");
* // create an index with `auto index = cuvs::neighbors::vamana::build(...);`
* cuvs::neighbors::vamana::serialize(handle, file_prefix, index);
* @endcode
*
* @param[in] handle the raft handle
* @param[in] file_prefix prefix of path and name of index files
* @param[in] index Vamana index
*
*/
void serialize(raft::resources const& handle,
const std::string& file_prefix,
const cuvs::neighbors::vamana::index<uint8_t, uint32_t>& index);
Expand Down
44 changes: 44 additions & 0 deletions docs/source/cpp_api/neighbors_vamana.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
Vamana
=====

Vamana is the graph construction algorithm behind the well-known DiskANN vector search solution. The cuVS implementation of Vamana/DiskANN is a custom GPU-acceleration version of the algorithm that aims to reduce index construction time using NVIDIA GPUs.

.. role:: py(code)
:language: c++
:class: highlight

``#include <cuvs/neighbors/vamana.hpp>``

namespace *cuvs::neighbors::vamana*

Index build parameters
----------------------

.. doxygengroup:: vamana_cpp_index_params
:project: cuvs
:members:
:content-only:

Index
-----

.. doxygengroup:: vamana_cpp_index
:project: cuvs
:members:
:content-only:

Index build
-----------

.. doxygengroup:: vamana_cpp_index_build
:project: cuvs
:members:
:content-only:

Index serialize
---------------

.. doxygengroup:: vamana_cpp_serialize
:project: cuvs
:members:
:content-only:
Loading

0 comments on commit e05882f

Please sign in to comment.