This is a quick introductory guide to Gerbil for seasoned schemers; it assumes familiarity with scheme and exposure to a couple of different implementations.
In the following $ is the shell prompt and > the gxi
interpreter prompt.
Add $GERBIL_HOME/bin to your path and invoke the interpreter
for the obligatory "hello world":
$ export PATH=$PATH:$GERBIL_HOME/bin
$ gxi
> (displayln "hello world")
hello world
The "hello world" script:
$ cat > hello.ss <<EOF
#!/usr/bin/env gxi
(def (main . args)
(displayln "hello world"))
EOF
$ chmod +x hello.ss
$./hello.ss
hello world
The "hello world" executable:
$ cat > hello.ss <<EOF
package: example
(export main)
(def (main . args)
(displayln "hello world"))
EOF
$ gxc -exe -o hello hello.ss
$ ./hello
hello world
And if you want a statically linked hello with no runtime dependencies to the Gerbil environment:
$ gxc -static -exe -o hello hello.ss
$ ./hello
hello world
The standard Scheme primitive forms and macros are all supported.
Runtime bindings are defined with the short forms def and defvalues:
(def (say-hello who)
(displayln "hello " who))
(defvalues (a b)
(values 1 2))
For those who prefer the classic long forms, define and define-values
are also available as in standard Scheme.
Procedures are defined with lambda and can have optional and keyword
formal arguments:
(def (a-simple-function a b)
(+ a b))
> (a-simple-function 1 2)
=> 3
(def (an-opt-lambda a (b 1))
(+ a b))
> (an-opt-lambda 1)
=> 2
(def (a-keyword-lambda a b: (b 1))
(+ a b))
> (a-keyword-lambda 1)
=> 2
> (a-keyword-lambda 1 b: 2)
=> 3
Multiple arity lambdas can be declared with case-lambda:
(def my-case-lambda
(case-lambda
((a b) (+ a b))
((a) (+ a 1))))
; or the short definition form
(def* my-case-lambda
((a b) (+ a b))
((a) (+ a 1)))
Let bindings can have a short form for single arguments, as well as multiple value bindings mixed in:
> ((let (x 1) (lambda (y) (+ x 1))) 1)
=> 2
> (let ((values a b) (values 1 2)) (+ a b))
=> 3
> (let ((x 1)
((values y z) (values 2 3)))
(+ x y z))
=> 6
Note that the _ identifier is reserved for bindings to
mean the null binding; that is the expression value
is ignored and no lexical binding is generated:
(def (a-function x . _) ; accepts 1 or more ignored args
(+ x 1))
> (a-function 1 2)
=> 2
Apart from cons and list, pairs and lists can also be can be constructed with short-hand syntax using square brackets:
; cons a pair
> [1 . 2]
=> (1 . 2)
; cons a list
> [1 2 3]
=> (1 2 3)
The short-hand syntax also supports list splicing using
using the ellipsis ...:
; splice nested list
> [1 2 [3 4 5] ... 6]
=> (1 2 3 4 5 6)
Bindings can be mutated with set! as usual.
> (def a #f)
> (set! a 'a)
> a
=> 'a
set! also expands with s-expressions as the target
of mutation.
When the head of the s-expression is a setf-macro it
is invoked to expand the syntax.
If the head is a plain identifier, as is the case
in the example below, to expands an identifier-set!
invocation.
> (def a-pair (cons 'a 'b))
> (set! (car a-pair) 'c)
> (car a-pair)
=> 'c
Finally, macros that mixin the setq-macro class,
like the ones created by identifier-rules, can also
be the target of mutation which leads to an expander
application.
All the usual Scheme macros are available, with common syntactic forms described later in the guide.
Gerbil supports Object-oriented programming with structs and classes. Structs are index-based types with single inheritance, while classes are slot-based types with multiple inheritance.
Structs are defined with defstruct:
(defstruct point (x y))
(defstruct (point-3d point) (z))
> (make-point-3d 1 2 3)
=> #<point-3d>
For each struct defstruct defines a constructor, a type predicate,
a runtime type descriptor, accessors and mutators, expansion time
struct info and a match expander.
So:
> (def my-point (make-point-3d 1 2 3))
> (point-x my-point)
=> 1
> (point-y my-point)
=> 2
> (point-3d-z my-point)
=> 3
> (set! (point-3d-z my-point) 0)
> (point-3d-z my-point)
=> 0
Classes are defined with defclass with slot accessed fields and support multiple inheritance. For example:
(defclass A (a))
(defclass B (b))
(defclass (C A B) (c))
...
> (def my (make-C a: 1 b: 2 c: 3))
> (A? my)
=> #t
> (B? my)
=> #t
> (@ my a)
=> 1
> (@ my b)
=> 2
> (@ my c)
=> 3
> (set! (@ my c) 0)
> (@ my c)
=> 0
Gerbil supports single dispatch for methods associated with a struct and class
type. Methods are defined with defmethod and invoked with curly brace {}
s-expressions.
For instance:
(import :std/format)
(defmethod {print point}
(lambda (self)
(with ((point x y) self)
(printf "{point x:~a y:~a}~n" x y))))
> {print my-point}
{point x:1 y:2}
(defmethod {print point-3d}
(lambda (self)
(with ((point-3d x y z) self)
(printf "{point-3d x:~a y:~a z:~a}~n" x y z))))
> {print my-point}
{point-3d x:1 y:2 z:0}
If you want to dispatch to the next method in the hierarchy, then you can use
the @next-method macro that is locally bound inside your method definition:
(defmethod {identify point}
(lambda (self)
(displayln 'point)))
(defmethod {identify point-3d}
(lambda (self)
(displayln 'point-3d)
(@next-method self)))
> {identify my-point}
point-3d
point
By default, the constructors generated for structs expect all the fields in indexed order, while the class constructor expects optional keywords for slots in the class. A custom constructor can be defined by specifying a constructor property designating a method at struct or class definition. For example:
(defstruct (point-3d point) (z)
constructor: :init!)
(defmethod {:init! point-3d}
(lambda (self x y (z 0))
(set! (point-x self) x)
(set! (point-y self) y)
(set! (point-z self) z)))
> (def my-point (make-point-3d 1 2))
> (point-3d-z my-point)
=> 0
Structs can only extend other structs with single inheritance. In contrast, classes can freely mixin structs, as long as the mixins contain a compatible base struct.
For example, the following constructs a diamond hierarchy with a base struct:
(defstruct A (x))
(defclass (B A) ())
(defclass (C A) ())
(defclass (D B C) () constructor: :init!)
(defmethod {:init! D}
(lambda (self x)
(set! (A-x self) x)))
(def d (make-D 1))
> (A? d)
=> #t
> (A-x d)
=> 1
> (B? d)
=> #t
> (C? d)
=> #t
> (D? d)
=> #t
> (type-descriptor-mixin D::t)
=> (#<type #5 B> #<type #6 C> #<type #7 A>)
Gerbil uses pattern matching extensively, so a suitable match macro is provided by the language. The pattern language is similar to plt's match macro, with structs destuctured by the structure name. In addition, the square brackets destructure lists symmetrically to their construction.
For example:
(def (my-destructurer obj)
(match obj
([a . b]
(printf "a pair (~a . ~a)~n" a b)
'pair)
((point-3d x y z)
(printf "a 3d-point (~a ~a ~a)~n" x y z)
'point-3d)
((point x y)
(printf "a 2d point (~a ~a)~n" x y)
'point-2d)
(else 'something-else)))
> (my-destructurer [1 2 3])
a pair (1 . (2 3))
=> 'pair
> (my-destructurer (make-point 1 2))
a 2d point (1 2)
=> 'point-2d
> (my-destructurer (make-point-3d 1 2))
a 3d-point (2 0 0)
=> 'point-3d
Gerbil's match provides a shorthand syntax for match lambdas:
(def car+cdr (match <> ([a . b] (values a b))))
> (car+cdr [1 2 3])
=> values 1 '(2 3)
It is also common to destructure-bind an object, thus a common
destructuring-bind form with is provided. The form can
bind a single object with short-hand notation or multiple
objects with a let-style head:
(def (car+cdr obj)
(with ([a . b] obj)
(values a b)))
(def (car+cdrx2 lsta lstb)
(with (([a-car . a-cdr] lsta)
([b-car . b-cdr] lstb))
(values a-car a-cdr b-car b-cdr)))
Gerbil has pervasive macro facilities and is a macro-rich language.
The full meta-syntactic tower is provided, with macro hygiene support
with syntax-case and quote-syntax.
Most macros are simple and medium syntax-rules macros, and thus Gerbil provides a short form for defining syntax-rules macros:
(defrules macro-id (id ...)
(head [guard] body) ...)
; equivalent:
(defsyntax macro-id
(syntax-rules (id ...)
(head [guard] body) ...))
More complicated macros are defined defsyntax and syntax-case
directly. Here is an example that introduces an identifier
hygienically:
(defsyntax (with-magic stx)
(syntax-case stx ()
((macro expr body ...)
(with-syntax ((magic-id (datum->syntax #'macro 'magic)))
#'(let (magic-id expr) body ...)))))
> (with-magic 3 (+ magic 1))
=> 4
The match expander is also macro capable; you can define a match
macro with defsyntax-for-match, which has the following form:
(defsyntax-for-match id match-macro [macro])
Both macros are procedures at phi+1, with the match-macro invoked when
expanding a match pattern and the optional normal macro invoked at normal
procedure application.
For example, the following defines a match macro for constructing and
destructuring pairs tagged with 'foo:
(defsyntax-for-match foo
(syntax-rules ()
((_ pat) (cons 'foo pat)))
(syntax-rules ()
((_ val) (cons 'foo val))))
> (def my-foo (foo 1))
> my-foo
=> (foo . 1)
> (match my-foo ((foo x) x))
=> 1
> (def my-bar (cons 'bar 2))
> (match my-bar ((foo x) x) (else 'not-a-foo))
=> not-a-foo
If you need to shift the phase of the expander to evaluate support code
for macros, you can do so with begin-syntax:
(begin-syntax form ...)
For example, the following macro uses a utility function in the fender, which is defined at phi=+1:
(begin-syntax
(def (identifier-or-keyword? stx)
(or (identifier? stx)
(stx-keyword? stx)))
(def (identifiers-or-keywords? lst)
(andmap identifier-or-keyword? lst)))
(defrules qlist ()
((_ (key val) ...)
(identifiers-or-keywords? #'(key ...))
[['key . val] ...]))
The full meta-syntactic tower is supported, so you can use the full
language at phi=+1 and shift higher with a nested begin-syntax. You
will have to import :gerbil/core at higher phases however, as the
prelude only provides bindings for phi=+1.
Modules are self-contained pieces of code. All identifiers
used in the runtime of the module must be bound. They
are either available from the prelude, imported from
another module, or declared as extern to indicate
runtime-provided identifiers.
Modules can be declared at the top level with the module
special form, can be defined in a file, or can be part of a library.
They can also be nested in another module.
Here is an example of a simple top module, which provides
a function that usesdisplay-exception from the runtime as extern:
(module A
(export with-display-exception)
(extern (display-exception display-exception))
(def (with-display-exception thunk)
(with-catch (lambda (e) (display-exception e (current-error-port)) e)
thunk)))
> (import A)
> (with-display-exception (lambda () (error 'it-is-an-error)))
it-is-an-error
=> #<error-exception #5>
Identifiers are imported from a module with the import special
form, which must appear at a top context (either top-level
or module scope).
It has the following syntax:
(import <import-spec> ...)
import-spec:
<module-path>
(import-expander <import-spec> expander-args ...)
module-path:
identifier ; top or module scope module
:identifier ; identifier with ':' prefix, library module
"path-to-module-file" ; file module, .ss extension optional
As we can see, import allows macros to manipulate the import set
of some import source (a module or another expansion).
They can be defined with defsyntax-for-import
An example macro is only-in, provided by the prelude:
(import (only-in :std/text/json read-json))
Here we import form :std/text/json only the read-json procedure.
Modules define the set of exported identifiers with the export
special form, which must appear at module scope.
It has the following syntax:
(export <export-sec> ...)
export-spec:
#t ; export all defined identifiers
identifier ; export a specific identifiers
(rename: id name) ; export an identifier with a different name
(import: <module-path>) ; re-export all imports from <module-path>
(export-expander <export-spec> args ...) ; export macro
Similarly to import, export also supports macros, which can
be defined with defsyntax-for-export.
An usual export macro is except-out, provided by the prelude:
(export (except-out #t display-exception))
This form exports all defined symbols, except display-exception.
It could be used by the example module A above to the same
effect.
Modules can be written directly in files, without a surrounding
module form.
For example, we can place our module A into a file A.ss
$ cat > A.ss <<EOF
(export with-display-exception)
(extern (display-exception display-exception))
(def (with-display-exception thunk)
(with-catch (lambda (e) (display-exception e (current-error-port)) e)
thunk))
EOF
> (import "A")
File modules take their name from the including file, so this
module is named A and uses A# as the namespace prefix.
You can be explicit about the namespace the module uses by
having a namespace: id declaration at the top of the module.
You can compile file modules with gxc:
$ gxc -d . A.ss
$ gxi
> (import "A") ; compiled form takes precedence
Library modules are imported with the :library-module-path form.
For example, to use the json module from the Gerbil std library
you need the following import statement:
(import :std/text/json)
The library module is defined in a file named json.ss in the Gerbil
std library source tree. The module declares that it is part of the
std/text package, which places compiler artefacts in the
$GERBIL_HOME/lib/std/text directory.
The namespace prefix for identifiers defined in the module is
std/text/json#.
When writing a library module, you should choose an appropriate package
for your code.
The package is specified with a package: package-path declaration
at the top of a module. It effects the namespace of the module and
placement of compiled code.
By default library modules are looked up in the $GERBIL_HOME/lib and
~/.gerbil/lib directories. You can specify additional directories to
be searched with the GERBIL_LOADPATH environment variable. You can
also modify the load-path at runtime with add-load-path.
This is best illustrated with an example, so let's package the A module
above into a library.
For this, we designate example as the library package,
and then give the module a more appropriate name.
Here, we call it util with the expectation that the library
and module will grow further:
$ mkdir example
$ cat > example/util.ss <<EOF
package: example
(export with-display-exception)
(extern (display-exception display-exception))
(def (with-display-exception thunk)
(with-catch (lambda (e) (display-exception e (current-error-port)) e)
thunk))
EOF
You can now compile the library module by invoking gxc and import it as
:example/util:
$ gxc example/util.ss
$ gxi
> (import :example/util)
By default, the compiler will place compiled modules in ~/.gerbil/lib.
If you want a separate directory structure for your library, you can
specify a different directory with the -d option:
$ gxc -d your-library-path example/util.ss
The gerbil compiler can also create executables that invoke the main function of a module. The generated executables only load runtime dependencies without linking and initializing the expander, resulting in significantly faster load times compared to wrapper scripts.
Note that by default, the executable is linked dynamically: the module is still compiled as a dynamic loadable module and must be available in the gerbil library path for the executable to work.
For example, suppose we have a module example/hello.ss that we want to compile as an executable module:
$ cat > example/hello.ss <<EOF
package: example
(export main)
(def (main . args)
(displayln "hello world"))
EOF
The module must define and export a main function that gets
invoked with the command line arguments after loading the gerbil
runtime and module dependencies.
You can compile it to an executable with gxc with the
following command:
$ gxc -exe -o hello example/hello.ss
$ ./hello
hello world
You can also compile the executable with static linkage, which links parts of the gerbil runtime and all dependent modules statically and allows the executable to work without a local gerbil environment:
$ gxc -static -exe -o hello example/hello.ss
$ ./hello
hello world
The disadvantage of static linkage is that the executables are bigger, and can only use the baseline parts of the gerbil runtime (gx-gambc0). That means that static executables can't use the expander or the compiler, as the meta levels of the gerbil runtime are not linked and initialized. They also take quite longer to compile.
The advantage is that static executables don't require a local Gerbil
installation to work, which makes them suitable for binary distribution.
They also load faster, as they don't have to do any dynamic module loading.
Furthermore, because dependencies are compiled in together, you can apply
declarations like (not safe) to the whole program, resulting in potentially
significant performance gains. And as of Gerbil-v0.13-DEV-50-gaf81bba the
compiler performs full program optimization with tree shaking, which provides
further performance benefits.
Every identifier accessible to a Gerbil module has to be defined somewhere, either as a concrete definition or an extern reference. The initial bindings in a module come from the prelude and the root context which is the parent context of every module.
The root context is a special context that contains the core macro expanders that provide the core language. The prelude context on the other hand, is an ordinary module that exports any number of bindings that form the language of the module.
When a prelude is not specified, the default prelude is the Gerbil
core prelude.
Any module however can designate a different prelude with the prelude:
module directive, which allows us to design custom languages.
Apart from standard bindings, custom preludes can also override some special expander indirection hooks by exporting macros with these names:
%%refcan intercept and redefine ordinary identifier references.%%appcan intercept and redefine ordinary procedure application.%%begin-modulecan intercept the expansion of a module body and provide custom full or partial expansion.
Language extensibility does not stop there however: prelude modules can
also specify a custom surface syntax, by providing a module reader.
The custom reader is invoked by using a #lang declaration at the beginning
of the module file:
#lang prelude [package: pkg-id] [namespace: namespace-id]
When the expander sources a module that begins with a #lang declaration
it imports the prelude and looks for a read-module-body export.
The function, which must be defined for syntax, takes as input a the port
containing the module body and returns a list of syntax objects which then
become the body of the module. The module is then expanded with the usual
expansion mechanism, including custom module body expansion as defined
by the prelude.
Custom languages are a big topic and this only touches the surface; they are further explored in the Custom Language tutorial.
As of Gerbil-v0.12-DEV-845-g39f54e4, you can elide the package: and
prelude: declations in your module and have them automatically deduced
from the file system layout of your library/package.
You can do so by creating a gerbil.pkg file in the root of your library,
which contains a property list.
The package: property specifies the prefix package at the root of your
hierarchy. The package of individual modules will extend this prefix to
mirror the directory structure.
The prelude: property specifies an implicit custom prelude for s-expression
based languages.
If the gerbil.pkg file is empty, then it is treated as an empty property list. This allows you to simply touch a gerbil.pkg at the root of your source hierarchy when you don't need a custom prelude and use a directory structure that mimics your logical package structure.
The gerbil standard library is located at src/std; it includes
several common libraries (SRFIs, and usual scheme libraries like
:std/pregexp, :std/sort, and :std/format), along with
Gerbil-specific libraries.
Here we provide examples and brief documentation for the more
interesting of the Gerbil-specific libraries.
Some library modules are not built by default, because they have external
library dependencies that may not be present in your system.
The build configuration for the std library is specified in
$GERBIL_HOME/src/std/build-features.ss.
If you have the required libraries (documented in build-features) in your
system, you can enable building by setting the (enable feature #f)
statement in build-features.ss to #t. You can then build the optional
library modules by running $GERBIL_HOME/src/build.sh stdlib.
The :std/sugar library provides, among other macros, a try syntactic
form for handling exceptions in imperative style.
For example:
> (try (error "my error")
(catch (e) (display-exception e (current-error-port)))
(finally (displayln "finally!")))
my error
finally!
The general syntax is
(try body ...
[catch-clause] ...
[finally-clause])
catch-clause:
(catch pred => K)
(catch (pred var) body ...)
(catch (var) body ...)
(catch _ body ...)
finally-clause:
(finally body ...)
Gerbil supports generic multi-method dispatch, with the requisite
runtime support and macros provided by :std/generic.
For example, the following defines a generic method my-add that
dispatches on numbers and strings:
(import :std/generic)
(defgeneric my-add
(lambda args #f))
(defmethod (my-add (a <number>) (b <number>))
(+ a b))
(defmethod (my-add (a <string>) (b <string>))
(string-append a b))
The code defined a generic method with the defgeneric macro,
providing a default method which is dispatched when there are no
matching methods. Next, we defined the two methods, operating
on numbers and strings. We can use the generic method as a procedure:
> (my-add 1 2)
=> 3
> (my-add "a" "b")
=> "ab"
We can similarly define methods for user-defined types as well.
Here we define an implementation for instances of a struct A:
> (my-add (make-A 1) (make-A 2))
=> #f
(defstruct A (x))
(defmethod (my-add (a A) (b A))
(make-A (+ (A-x a) (A-x b))))
> (my-add (make-A 1) (make-A 2))
=> #<A a: 3>
Inside the body of every method implementation, @next-method is bound
to a procedure which dispatches to the next matching method.
For example:
(defmethod (my-add (a <fixnum>) (b <fixnum>))
(displayln "add fixnums")
(@next-method a b))
Normally in the procedure body we would add with fx+, but for
the shake of the example we display a message and let the generic
number method to add.
> (my-add 1 2)
add fixnums
=> 3
The :std/iter library provides support for iteration using
the iterator protocol. The library also provides macros of
the for family for iterating over sequences or other objects
that implement the iteration protocol.
The basic iteration macro is the imperative for comprehension.
The syntax matches patterns to iterators in parallel, and invokes
the body as long as none of the iterators have signalled end
of iteration.
For example:
(import :std/iter)
> (for (x '(1 2 3))
(displayln x))
1
2
3
> (for ((x '(1 2 3))
(y '#(a b c d)))
(displayln x " " y))
1 a
2 b
3 c
All patterns supported by the match macro can be matched in lieu
of plain variable bindings.
For instance:
> (for ([key . val] '((a . 1) (b . 2) (c . 3)))
(displayln key " " val))
a 1
b 2
c 3
The iteration macro supports the usual suspects for generic iteration: lists, vectors, strings, hash-tables, input-ports, and ranges.
Simple filters can be specified with the when and unless keyword in
the binding for:
> (for ([x . y] '((a . 0) (b . 1) (c . 2)) when (> y 0)) (displayln x))
b
c
> (for ([x . y] '((a . 0) (b . 1) (c . 2)) unless (> y 0)) (displayln x))
a
The variant for* performs multi-dimensional iteration, equivalent
to nested fors:
> (for* ((x (in-range 2)) (y (in-range 2)))
(displayln x y))
00
01
10
11
The values of an iteration can be collected in a list with for/collect:
> (for/collect ((x (in-naturals))
(y '#(a b c d)))
(cons x y))
=> ((1 . a) (2 . b) (3 . c) (4 . d))
Finally, the values of an iteration can be folded to produce a value;
in this example we construct a reversed list out of an iterator
with a folding cons:
> (for/fold (r []) (x (in-range 1 5))
(cons x r))
=> (5 4 3 2 1)
Iteration dispatch applies the generic method :iter in order
to produce an iterator object. The default implementation calls
the method :iter on the object. There are methods for
iterating lists, hashes, input-ports, ranges etc.
The easiest way to implement an iterator is through a coroutine
procedure. The procedure is coexecuted with the iteration loop,
and produces values for the loop with yield:
(def (my-generator n)
(lambda ()
(let lp ((k 0))
(when (< k n)
(yield k)
(lp (1+ k))))))
> (for (x (my-generator 3))
(displayln x))
0
1
2
We can now use this generator to produce an iterator for a user-defined struct:
(defstruct A (x))
(defmethod {:iter A}
(lambda (self)
(:iter (my-generator (A-x self)))))
> (for (x (make-A 3))
(displayln x))
0
1
2
The :std/coroutine library provides support for coroutines, running
in a separate thread and yielding results with yield. The user creates
the coroutine with coroutine, and receives results with continue which
blocks the current thread and passes control to the coroutine until
it yields a value or exits. After the coroutine procedure finishes,
all further calls to continue will return the final result or
deliver an exception.
For example:
(import :std/coroutine)
(def (my-coroutine)
(yield 1)
(yield 2)
(yield 3))
(def cort (coroutine my-coroutine))
> (continue cort)
=> 1
> (continue cort)
=> 2
> (continue cort)
=> 3
> (continue cort)
=> #!void ; coroutine ended
> (continue cort)
=> #!void ; all
The :std/event library provides procedures and macros for event-driven
programming.
These are the low level primitives, which wait and multiplex on primitive selectors:
- Threads, which signal when the thread terminates.
- Pairs of a locked mutex with a condition variable, which signal when the condition signals after the mutex has been unlocked.
- Naked i/o condvars, which are signaled by the runtime scheduler.
wait blocks the current thread until a selector is signalled, while select blocks on
multiple selectors concurrently, using a thread for each selector. Both procedures accept
an optional timeout and return the selector that was signalled or #f in the case of timeout.
(def (wait selector (timeout #f)) ...)
(def (select list-of-selectors (timeout #f)) ...)
For example:
(import :std/event)
(def my-thread (spawn (lambda () (thread-sleep! 10))))
> (wait my-thread 1) ; or (select [my-thread] 1)
=> #f ; after a second elapses
> (wait my-thread) ; or (select [my-thread])
=> my-thread ; after the thread completes its sleep
sync is the high-level synchronization primitive from PLT-Scheme, which works with
high level events.
A valid argument for sync is any synchronizable object, automatically wrapped with wrap-evt:
- events
- primitive selectors
- input ports
- timeouts
- any object that implements the
:eventmethod to return a synchronizable object
An event is
- the primitive events
never-evt(bottom) andalways-evt(top) - an event object, constructed with
make-eventorwrap-evt - an event-set object, constructed with
choice-evt - an event-handler object, constructed with
handle-evt; it is an event tied with a continuation function which is tail invoked with the value of the event. Multiple continuations can be chained withhandle-evteach receiving the values of the previous, starting with the value of the vent.
sync accepts an arbitrary of events as arguments, and returns when exactly one of them is
ready. The value of sync is the value of the event: by default, timeouts have a value of #f
and other events have usually the synchronizer as value.
For example:
(def my-thread (spawn (lambda () (thread-sleep! 10))))
> (sync 1 my-thread)
=> #f ; after a second elapses
> (sync my-thread)
=> my-thread ; after the thread completes its sleep
A more complicated example which utilizes handle-evt for loops:
(def my-thread (spawn (lambda () (thread-sleep! 10) 'done)))
> (let lp ((n 0))
(sync (handle-evt 1
(lambda (_) (displayln "timeout " n) (lp (fx1+ n))))
(handle-evt my-thread
(lambda (thr) (thread-join! thr)))))
timeout 0
timeout 1
timeout 2
timeout 3
timeout 4
timeout 5
=> 'done
The library also offers a couple of macros, ! and !*, which simplify
event driven programming. ! syncs a single event while !* syncs
multiple events.
For example:
;; sync on my-thread
(! my-thread (displayln "my thread exited"))
;; rewrite the previous example loop:
(let lp ((n 0))
(!* (1 (displayln "timeout " n) (lp (fx1+ n)))
(my-thread
(thread-join! my-thread))))
At the low-level Gerbil builds on Gambit's thread mailboxes to provide actor-oriented programming capabilities and remote interactor communication.
Gerbil's actors are threads, either in the current or a remote processes and communicate exchanging messages. Messages can be arbitrary objects, but usually actors communicate with structured messages:
(defstruct message (e source dest options))
(def (send dest value) ...) ; send raw message
(def (send-message dest value (options #f)) ...) ; send structured message
(def (send-message/timeout dest value timeo) ...)
Actors process messages and events with two main macros, << and <-.
They both synchronize on the thread's mailbox and pattern match incoming messages;
the difference is that << matches on raw messages and <- matches on
structured message values.
Within a <- pattern body, the variables @message, @value, @source,
@dest and @options are bound from the structured message.
Within the pattern body, the -> can be used as shorthand syntax to send messages
to @source.
For example, a simple echo actor:
(import :std/actor)
(def (my-echo)
(let lp ()
(<- (value
(displayln @source " says " value)
(-> value)
(lp)))))
(def echo (spawn my-echo))
> (send-message echo 'hello)
#<thread #1 primordial> says hello
> (<- (value value))
=> 'hello
Beyond structured messages, Gerbil provides protocols for structured interaction.
Protocol messages can be one way messages (instances of !event), a remote
call (instances of !call) or a value for a previous call (!value or !error).
Protocols are defined with defproto, which defines structures and macros
for using the protocol interfaces, together with marshalling support.
For example, let's define an echo protocol:
(defproto echo
id: echo
call: (hello what))
(def (my-echo)
(let lp ()
(<- ((!echo.hello what k)
(displayln @source " says " what)
(!!value what k)
(lp)))))
(def echo (spawn my-echo))
> (!!echo.hello echo 'hello)
#<thread #1 primordial> says hello
=> 'hello
In the example, we define a protocol echo with a single call hello.
The macro defines the structures and macros for using the interface:
(defstruct echo.hello (what))
(defsyntax-for-match !echo.hello ...)
(defrules !!echo.hello ...
The invocation (!!echo.hello echo 'hello) constructs a !call with
value an instance of echo.hello and a gensymed continuation id.
It then sends a message with the !call to the echo actor and
waits for a !value or !error message matching the continuation.
If it receive a !value it returns it, and if it receives an !error
it signals an error.
In the actor, the (!echo.hello what k) matches a !call with
the value matching (echo.hello what) and the continuation token
bound to k. The actor displays its message, and then responds by
sending a value with the !!value macro. An error could be signalled
with the !!error macro.
The syntax for !!value and !!error is similar: the take
an optional destination (which defaults to @source), a value
or error message and the continuation token:
(!!value [@source] val token)
(!!error [@source] msg token)
In addition to calls and events, actors can also open streams.
A stream is like a call, but it returns multiple values using
!!yield until the stream's end or an error occurs.
For example, the following server generates a stream of numbers as specified by the argument:
(defproto simple-stream
stream: (count N))
(def (my-simple-stream)
(def (stream dest N k)
(let lp ((n 0))
(if (< n N)
(begin
(!!yield dest n k)
(!!sync dest k) ; request sync
(<- ((!continue k) ; flow control
(lp (1+ n)))
((!close k) ; stream closed
(!!end dest k))
((!abort k) ; stream aborted
(void))))
(!!end dest k))))
(let lp ()
(<- ((!simple-stream.count N k)
(spawn stream @source N k)
(lp)))))
(def my-stream (spawn my-simple-stream))
Yielded values can be processed as messages, or with the !!pipe
macro which constructs a vector pipe and pipes the yielded values
through it in a background thread.
Here is an example that uses a pipe:
> (let ((values inp close) (!!pipe (!!simple-stream.count my-stream 5)))
(for (x inp)
(displayln x)))
0
1
2
3
4
The pipe may be convenient, but it forgoes end-to-end back pressure
and synchronization with !!sync. Here is the same example again,
but with explicit processing of messages through the actor mailbox:
(let (k (!!simple-stream.count my-stream 5))
(let lp ()
(<- ((!yield x (eq? k))
(displayln x)
(lp))
((!sync (eq? k))
(!!continue k)
(lp))
((!end (eq? k))
(void)))))
The interaction so far has been local. In order to interact with remote actors, Gerbil provides an rpc protocol and server for handling the necessary network interaction.
Using rpc is very simple: An rpc server can be constructed
with start-rpc-server! which accepts an optional server address
to bind and a wire protocol implementation with a keyword.
In one shell:
(def (my-echo rpcd)
(rpc-register rpcd 'echo echo::proto)
(let lp ()
(<- ((!echo.hello what k)
(displayln @source " says " what)
(!!value what k)
(lp)))))
(def serverd (start-rpc-server! "127.0.0.1:9999"))
(def echod (spawn my-echo serverd))
This starts an rpc server at port 9999 in the localhost.
The echo actor binds itself under the id echo using the
echo protocol echo::proto for marshalling and unmarshalling.
In a different shell, we can connect to our echo with a remote handle:
(def clientd (start-rpc-server!))
(def echod (rpc-connect clientd 'echo "127.0.0.1:9999" echo::proto))
> (!!echo.hello echod 'hello)
=> 'hello
If your actors are well-known (application scoped), then you can globally bind
a protocol to the name with bind-protocol! and you don't need to specify
the protocol in rpc-register and rpc-connect:
(bind-protocol! 'echo echo::proto)
(def clientd ...)
(def echod (rpc-connect clientd 'echo "127.0.0.1:9999"))
By default, a null rpc protocol is used which does no authentication or encryption. If you intend to expose your actors to the Internet you should use authentication and optionally encryption.
For authentication, you can generate a shared cookie with rpc-generate-cookie!
and start your rpc-server using the rpc-cookie-proto:
$ mkdir ~/.gerbil
> (rpc-generate-cookie!)
; generates a cookie in ~/.gerbil/cookie
...
(def remoted
(start-rpc-server! "127.0.0.1:9999"
proto: (rpc-cookie-proto)))
...
(def locald
(start-rpc-server! proto: (rpc-cookie-proto)))
...
If you also want to encrypt your communications, then use
the rpc-cookie-cipher-proto as proto: argument for your rpc
servers. On top of cookie authentication, this protocol performs
a Diffie-Hellman key exchange and then encrypts all messages with
AES/HMAC (it encrypts and then MACs).
Gerbil provides a simple interface for making http(s) requests, inspired by python's requests library. Here is an example for how to use the library:
> (import :std/net/request)
> (def req (http-get "freegeoip.net/json/8.8.8.8"))
> (request-status req)
=> 200
> (request-text req)
=> "{\"ip\":\"8.8.8.8\",\"country_code\":\"US\",\"country_name\":\"United States\",\"region_code\":\"CA\",\"region_name\":\"California\",\"city\":\"Mountain View\",\"zip_code\":\"94040\",\"time_zone\":\"America/Los_Angeles\",\"latitude\":37.3845,\"longitude\":-122.0881,\"metro_code\":807}\n"
> (hash->list (request-json req))
=> ((country_name . "United States")
(metro_code . 807)
(longitude . -122.0881)
(country_code . "US")
(latitude . 37.3845)
(time_zone . "America/Los_Angeles")
(region_name . "California")
(ip . "8.8.8.8")
(zip_code . "94040")
(city . "Mountain View")
(region_code . "CA"))
Gerbil has library support for JSON with the :std/text/json library.
The library provides the following procedures:
(def (read-json (port (current-input-port)) ...)
(def (string->json-object str) ...)
(def (write-json obj (port (current-output-port))) ...)
(def (json-object->string obj) ...)
The mapping of Scheme Objects to JSON objects is similar to other Scheme JSON libraries.
The read-json procedure constructs primitive objects (strings, numbers, lists, symbol hashes).
The write-json writes JSON objects with the JSON external data representation.
The following is a convertible JSON object:
- booleans, corresponding to
trueandfalse #!void, corresponding tonull- real numbers
- strings
- proper lists of JSON objects
- vectors of JSON objects
- hashes mapping symbols to JSON objects
- any object that defines a
:jsonmethod mapping the object to a JSON object.
Gerbil supports XML and HTML with the :std/xml library.
The library supports parsing and querying with Oleg's SXML/SSAX/SXPath and
provides additional facilities for processing SXML.
Optionally, when configured so, the library can also use libxml2 to parse
real world HTML (and plain old XML).
The libxml2 dependent components are not built by default.
You can build them by editing std/build-features.ss to set (enable libxml #t)
and rerunning the std library build script as described earlier in the guide.
For example, here is a parse of the bing front page without scripts, style, and CDATA:
> (import :std/net/request :std/xml/libxml)
> (def req (http-get "https://www.bing.com"))
> (parse-html (request-text req) filter: '("script" "style" "CDATA"))
=> (*TOP* (html (@ (lang "el")
(xml:lang "el")
(xmlns "http://www.w3.org/1999/xhtml"))
(head (meta (@ (content "text/html; charset=utf-8")
(http-equiv "content-type")))
(title "Bing")
(link (@ (rel "icon")
(sizes "any")
(mask "")
(href "/fd/s/a/hp/bing.svg")))
(meta (@ (name "theme-color") (content "#4F4F4F")))
(link (@ (href "/s/a/bing_p.ico") (rel "icon")))
(meta (@ (content "Το Bing σ"))))))
; so much for modern html!
; everything script, style, and CDATA in the page.
Gerbil offers two options to support web applications:
- fastcgi with a rack-style interface in
:std/web/rack. - an embedded http server in
:std/net/httpd.
The rack/fastcgi server has been in the standard library since early releases of Gerbil and has a very simple interface familiar from other languages. It works with standard ports so it supports non-development versions of Gambit which don't have raw devices.
The embedded http server is a new development in Gerbil-v0.12-DEV, and utilizes raw devices. It is significantly faster and offers a low level interface oriented towards API programming.
This is the obligatory hello web example:
(import :std/web/rack)
(def (respond env data)
(values 200 '((Content-Type . "text/plain")) "hello world\n"))
(start-rack-fastcgi-server! "127.0.0.1:9000" respond)
The fastcgi web handler is started with start-rack-fastcgi-server! from
the std/web/rack library module. The procedure accepts an address where
it will listen for fastcgi requests and a handler procedure.
The handler accepts two arguments, a hashtable which contains the CGI
request environment and the data attached to the request as a u8vector.
The handler returns 3 values: the status code for the response, the
HTTP headers as an associative list, and the content which can be a string,
u8vector or an iterable yielding a stream of string or u8vector data.
Here is a more complex example that prints all request variables to the response:
(def (respond env data)
(values 200 '((Content . "text/html")) (print-headers env)))
(def (print-headers env)
(lambda ()
(yield "<pre>\n")
(for ((values key val) env)
(yield (format "~a: ~a\n" key val)))
(yield "</pre>\n")))
Here is the hello world example using the embedded httpd:
(import :std/net/httpd)
;; start the httpd
(def httpd
(start-http-server! "127.0.0.1:8080"))
;; define a handler
(def (hello-handler req res)
(http-response-write res
200 ; status
'(("Content-Type" . "text/plain")) ; headers
"hello world\n"))
;; register the handler
> (http-register-handler httpd "/hello" hello-handler)
Here, we start an httpd server, which is a background thread serving
HTTP requests. We then register a handler for the /hello path, which
will serve all requests for /hello and subpaths.
The handler is a function that accepts two arguments: a request and a
response. This handler does not read the response body, and simply
responds with hello world with a single http-response-write call.
We can see the handler at work with curl:
$ curl http://localhost:8080/hello
hello world
For more examples of httpd handlers, see the httpd tutorial.
Gerbil include support for SQL databases (SQLite, PostgreSQL, MySQL)
and key-value stores (LevelDB, LMDB) with the :std/db package.
The :std/db/dbi library provides the implementation of the
database interface, while individual modules (:std/db/sqlite,
:std/db/postgresql and :std/db/mysql) provide the drivers for
particular databases.
Note that not all drivers are built by default, as some are FFI
drivers (SQLite, MySQL), so you will need to enable them for your
installation in $GERBIL_HOME/src/std/build-features.ss.
Here is an example of using the dbi interface with SQLite. First, the necessary imports and a connection to an in-memory database:
> (import :std/db/dbi :std/db/sqlite)
> (def db (sql-connect sqlite-open ":memory:"))
Then we create a simple table with sql-eval, which evaluates an SQL statement:
> (sql-eval db "CREATE TABLE Users (FirstName VARCHAR, LastName VARCHAR, Secret VARCHAR)")
Let's insert some data in our table, using a prepared statements:
> (def insert (sql-prepare db "INSERT INTO Users (FirstName, LastName, Secret) VALUES (?, ?, ?)"))
> (sql-txn-begin db)
> (sql-bind insert "John" "Smith" "very secret")
> (sql-exec insert)
> (sql-bind insert "Marc" "Thompson" "oh so secret")
> (sql-exec insert)
> (sql-txn-commit db)
And finally a query:
> (def users (sql-prepare db "SELECT FirstName, LastName FROM Users"))
> (sql-query users)
=> (#("John" "Smith") #("Marc" "Thompson"))
We can also iterate on the results of a query:
> (import :std/iter)
> (for (#(FirstName LastName) (in-sql-query users))
(displayln "First name: " FirstName " Last name: " LastName))
First name: John Last name: Smith
First name: Marc Last name: Thompson
And we are done, we can close our database connection:
> (sql-close db)
The :std/db/leveldb library provides support for LevelDB,
while the :std/db/lmdb library provides support for LMDB.
The libraries are not built by default, as they have foreign dependencies, so you need to
enable them for your installation in $GERBIL_HOME/src/std/build-features.ss.
For example, here we use the LevelDB library for some simple operations:
> (import :std/db/leveldb
:std/sugar)
> (def db (leveldb-open "/tmp/leveldb-test.db"))
;; let's put some values
> (leveldb-put db "abc" "this is the value of abc")
> (leveldb-put db "def" "this is the value of def")
;; we can retrieve them -- objects are always stored as u8vectors
> (displayln (bytes->string (leveldb-get db "abc")))
this is the value of abc
;; let's iterate and print the contents of the store
> (let (itor (leveldb-iterator db))
(leveldb-iterator-seek-first itor)
(while (leveldb-iterator-valid? itor)
(displayln (bytes->string (leveldb-iterator-key itor))
" => "
(bytes->string (leveldb-iterator-value itor)))
(leveldb-iterator-next itor))
(leveldb-iterator-close itor))
abc => this is the value of abc
def => this is the value of def
;; we can do the same with a for loop
> (for ((values key val) (in-leveldb db))
(displayln (bytes->string key)
" => "
(bytes->string val)))
abc => this is the value of abc
def => this is the value of def
;; Let's delete a value
> (leveldb-delete db "abc")
> (leveldb-get db "abc")
=> #f
;; we are done, let's close the db
(leveldb-close db)
The LMDB library is covered in in the Key-Value Store Server tutorial.