CLAP Grammar Proposal

CLAP grammar (Command Line Argument Parse)

EARLY DRAFT STATE

Conceptually, we want to specify a grammar in much the same way as something like Backus-Naur form. But, it needs to be augmented with additional information like descriptions and tags (which specify what is to be completed). We want it to be concise and focussed on completion candidates but still have something that can be built into a NFA or DFA. It should also be open to use for things other than completion.

Parsing goes left-right up to the cursor. And the set of possible states at this stage indicates things to be completed. Parsing should also proceed to the right of the cursor (perhaps going backwards from the end) which can further limit the sets, mainly by manipulating sentinel values.

Conceptually, this is similar to Zsh's _regex_arguments but the aim is to be able to define things as succinctly as _arguments. _arguments handles common command-line parser variants, including:

• -- terminates options (or not)
• non-option argument terminates options (or not)
• option packing: -abc for -a -b -c and -abc arg forms
• -a arg -a=arg --aa arg --aa=arg

It therefore will need some sort of macro layer on top so that you can specify the specific form of the parser.

{ … } blocks of alternatives (no alternation operator unless you count newline)
<…>   delimits tags these correspond to things that might be completed
[…]   descriptions
/…/   regex for consumed characters (may also need lookahead and lookbehind, perhaps \…\, |…| and /…/)
\x…x  regex with alternate delimiter
(…)   sentinel conditions
=>    concatenation (continuation in next argument)
->    concatenation (splitting the current argument)
$     named sequence (like <…> in BNF), can be scoped
@     macro or function invocation can also be scoped
=     assign a sequence to a name, newline terminates but blocks and trailing operators allow continuation

Alternation is the default within {…} blocks, each alternative can only be passed once unless followed by (*).

The following is a sort of example to demonstrate usage

CLAP 0.0                              # declare grammar specification version
command cmd = $args                    # define grammar for a command's arguments
stdin cmd = $in                       # define grammar for redirection in
stdout cmd = $out                     # define grammar for redirection out
stderr cmd = $err                     # define grammar for stderr redirection
envar VAR = $var                      # define grammar for environment variable value

args = $opts => $arg1 => $arg2

opts = {                              # define a named sequence
  <options>:[option]
  /-./                                # match and ignore unknown options
  /--.+/
  -f [option description]             # short option with description
  { -l --long } [synonyms]
  -r [repeatable option] (*)          # repeatable option
  -v [verbose]          (*3)          # repeatable with maximum
  -x [specify argument] => {          # option taking argument
    <users>:[user] /.*/               # defer to shell's user completion
  }
  -y [desc] (!a)                      # option excluded by sentinel
  { -o --opt } $sep =>
  <options/actions>:[action option] (+a) # subgroup incrementing a sentinel
  --do
  --undo
} (*)


arg1 = {
  @lowercase()                        # predefined function for mapping lower to upper
  <arg>:[argument]:!"$arg --list-args"  # shell-command to list arguments:descriptions
}

arg2 = {
  @partial(_)                         # predefined function so sn_ca matches snake_case
  snake_case
  joined_words
}

macros: TBD

Sentinels:

• This is the basis for specifying mutual exclusion between options.
• * – allow many
• ^ - exclusive (allow many but not duplicates)
• ^list - exclusive, sharing a dedup list between places.
• +o – increment o counter
• -debug – decrement debug counter
• ?opt – require opt counter >0
• !opt – require opt counter <=0
• 0opt – zero counter
• 1opt – set counter

Common list of standard tags (plural names):

• options
• processes
• files
• directories
• users
• hosts For common tags, you can elide the description. We'll need ways to provide options to certain tags like file extensions to be completed. Tags also come with predefined regexes for characters to be consumed.

Predefined macros (mostly affecting matching rules):

• @quote(start,protected[,end])
• @shell_quote() - nested level of shell quoting
• @packed_options() - allow -abc to match -a -b -c
• @uppercase() - lowercase matching uppercase
• @lowercase() - uppercase matching lowercase
• @map() - character equivalence classes
• @partial() - e.g. /u/l/b completing to /usr/local/bin
• @initial() - initial characters that match nothing, e.g. initial zeroes on a number @camel() - partial matching of camel case (or some generalisation there-of)