-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.Rmd
105 lines (77 loc) · 3.66 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# sakura
<!-- badges: start -->
[](https://shikokuchuo.r-universe.dev/sakura)
[](https://github.com/shikokuchuo/sakura/actions)
<!-- badges: end -->
```
________
/\ sa \
/ \ ku \
\ / ra /
\/_______/
```
### Extension to R Serialization
An extension of R native serialization using the 'refhook' system for custom serialization and unserialization of non-system reference objects.
This package was a request from a meeting of the [R Consortium](https://r-consortium.org/) [Marshalling and Serialization Working Group](https://github.com/RConsortium/marshalling-wg/) held at useR!2024 in Salzburg, Austria. It is designed to further discussion around a common framework for marshalling in R.
It extracts the functionality embedded within the [nanonext](https://github.com/shikokuchuo/nanonext) and [mirai](https://github.com/shikokuchuo/mirai) async frameworks for use in other contexts.
### Overview
Some R objects by their nature cannot be serialized, such as those accessed via an external pointer.
Using the [`arrow`](https://arrow.apache.org/docs/r/) package as an example:
```{r arrowfail,error=TRUE}
library(arrow, warn.conflicts = FALSE)
x <- list(as_arrow_table(iris), as_arrow_table(mtcars))
unserialize(serialize(x, NULL))
```
In such cases, `sakura::serial_config()` can be used to create custom serialization configurations, specifying functions that hook into R's native serialization mechanism for reference objects ('refhooks').
```{r arrowcfg}
cfg <- sakura::serial_config(
class = "ArrowTabular",
sfunc = arrow::write_to_raw,
ufunc = function(x) arrow::read_ipc_stream(x, as_data_frame = FALSE)
)
```
This configuration can then be supplied as the 'hook' argument for `sakura::serialize()` and `sakura::unserialize()`.
```{r arrowpass}
sakura::unserialize(sakura::serialize(x, cfg), cfg)
```
This time, the arrow tables are handled seamlessly.
Other types of serialization function are vectorized and in this case, the configuration should be created specifying `vec = TRUE`. Using `torch` as an example:
```{r torchfail, error=TRUE}
library(torch)
x <- list(torch_rand(5L), runif(5L))
unserialize(serialize(x, NULL))
```
Base R serialization above fails, but `sakura` serialization succeeds:
```{r torchpass}
cfg <- sakura::serial_config(
class = "torch_tensor",
sfunc = torch::torch_serialize,
ufunc = torch::torch_load,
vec = TRUE
)
sakura::unserialize(sakura::serialize(x, cfg), cfg)
```
### Acknowledgements
We would like to thank in particular:
- [R Core](https://www.r-project.org/contributors.html) for providing the interface to the R serialization mechanism.
- [Luke Tierney](https://github.com/ltierney/) and [Mike Cheng](https://github.com/coolbutuseless) for their meticulous efforts in documenting the serialization interface.
- [Daniel Falbel](https://github.com/dfalbel) for discussion around an efficient solution to serialization and transmission of torch tensors.
### Installation
The current development version is available from R-universe:
```{r universe, eval=FALSE}
install.packages("sakura", repos = "https://shikokuchuo.r-universe.dev")
```
--
Please note that this project is released with a [Contributor Code of Conduct](https://shikokuchuo.net/sakura/CODE_OF_CONDUCT.html). By participating in this project you agree to abide by its terms.