Modules

[tertsdiepraam]

While Roto does have some namespacing at the moment, it does not have modules and no multi-file support. We'd like to fix that. However, modules aren't easy because every language seems to kind of have it's own flavour of handling modules. Some treat the filesystem hierarchy as the module (e.g. Python), some allow the user to build up the tree (e.g. Rust), others just use a no namespacing at all (*cough* C *cough*). It's generally expected though that the name of the file and its location means something (even in Rust's case this is true).

Warning

I hope to evolve this issue into a full design for Roto's module system, but it's not done yet.

Here's some open questions surrounding module systems:

1. What files are included in a Roto project (and hence accessible in the code).
2. How are these files structured?
3. How do we transfer those files to a remote Rotonda instance?
4. What syntax do we use for import/use etc?
5. How do we control visibility like if at all (like Rust's pub)?
6. How do we do namespacing folding (like use module::*)?
7. Do we allow circular imports?
8. Do we have something similar to crates/packages. Say if a script references the root of the module tree, do they get the root of their package or the root of the program.

Some things to keep in mind from how Roto currently works:

• Roto uses . for namespacing currently, at least for enum variants.
• Roto is meant to be a scripting language, so we need this to be somewhat simple to use, so a system as full-featured as Rust's doesn't really work for Roto.

Things I dislike about other module systems (at least for the purpose of copying it to Roto)

1. Having to write mod in Rust.
2. Having a flat hierarchy by default in C.
3. Being able to add items anywhere in the module tree in Haskell.
4. Locations of files depending on PWD in Python.
5. Pythons from ... import * being swapped from import ....
6. Visibility based on capitalization in Go.
7. Cyclic imports happen easily in Python, but aren't allowed.
8. An import should not execute any code like in Python and many other scripting languages.

A first design

Here's the first somewhat sensible design I can come up with:

• A Roto project is given by a folder.
• The compiler scans this folder for sub-folders containing .roto files.¹
• A mod.roto file is the file that specifies the module for the folder it lives in, the other files are sub-modules.
• foo.roto and the folder foo are not allowed to co-exist in the same folder.
• Imports are done with import module.module.item and import module.module.* where * is only allowed as the last part.
• Items can be referenced with . as well.
• All items are always visible, because visibility rules are hard. Although we could hide items starting with _ or something like that and do the same for modules.
• Cyclic imports are allowed because we don't execute any code and we need to check for cyclicness anyway to make sure name resolution is not infinite.
• An item or an import starting with . is taken from the root.
• There are no crates or packages.
• super refers to the parent module in any path.

An example:

.
├── foo
│   ├── bar
│   │   └── mod.roto
│   ├── baz.roto
│   └── mod.roto
├── baz.roto
└── lib.roto

Would create the following modules:

lib
lib.foo
lib.foo.bar
lib.foo.baz
lib.baz

Any item in foo/bar.roto can then be referenced with lib.foo.bar.name either imported or directly. No imports are technically necessary to reference anything.

In foo/baz.roto:

import bar.square          # imports the square function from ./foo/bar/mod.roto
import lib.foo.bar.square  # same as above
import super.baz.double    # imports the double function from ./baz.roto
import lib.baz.double      # same as above
import lib.foo.*           # imports all items in ./foo/mod.roto

function square_three_times(a: u32) {
    let x = square(a);             # use imported symbol
    let y = bar.square(x);         # use relative path for symbol
    let z = lib.foo.bar.square(y); # use absolute path for symbol
    z
}

We send the files to a remote Rotonda instance by providing something like a JS importmap: a simple file mapping each file to it's path, so Rotonda gets all the information it needs to compile. We can use this for tests as well!

Alternatives

Instead of making every submodule public, they could be private by default but exposed with a pub import:

# in foo.roto
pub import bar # makes foo.bar accessible from the outside

References

• An overview of module systems in different languages: [WIP] A summary on module systems Draco-lang/Language-suggestions#73
• Some posts by Johnathan Goodwin:
  • https://pling.jondgoodwin.com/post/modularity_in_programming/
  • https://pling.jondgoodwin.com/post/cone-modules/
• An old paper comparing Ada, Modula-2, Eiffel and Oberon: https://www.researchgate.net/publication/220178692_A_comparison_of_module_constructs_in_programming_languages

Footnotes

1. This scan might need to be limited to folders containing .roto files or with some maximum depth or number of files to prevent scanning an entire filesystem or something like that. ↩

[ximon18]

Is . as a separator a good idea, e.g. in bar.square(x) it looks like a fn call on a type instance, maybe use :: like Rust instead?

[tertsdiepraam]

Good point! I chose . because I was confused by :: when I started Rust. Many scripting languages use . and I think it's unambiguous.

Python, Java & Kotlin use it for example, as do all dynamic languages that basically treat modules as dictionaries (which we obviously don't want to do). I've never really gotten tripped up by it in Python.

You're right though that it's overloading . which might not be a good idea.