This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Documentation

Status: Experimental. Preferably, only use it in throwaway projects.

If you are a lisper, see Introduction to Moonli for Lispers.

Moonli is a beautiful programming language with a simple syntax and a very interactive development experience. This enables rapidly prototyping new programs. At the same time, once a prototype is ready, the programs can also be optimized for runtime performance as well as long term maintenance.

1 - Installation

Method 1: Binaries

Grab a binary from the latest release. If you want an interactive REPL as in the above gif, grab the binaries with "repl" in their name.

If you want to run a few moonli files, grab the regular binaries.

However, the repl binaries depend on readline. This can be installed as follows:

  • Ubuntu: sudo apt install libreadline-dev
  • Mac OS: brew link readline --force
  • Windows: pacman -S mingw-w64-x86_64-readline

In case of any installation issues, please create an issue on github or gitlab.

Method 2: From fresh compilers

Step 0. Install a package manager

Mac OS: brew

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Windows:

  • MSYS2: this will install the pacman package manager
  • Or choco

Step 1. Install a compiler and some tools

  • Ubuntu: sudo apt install git sbcl
  • Mac OS: brew install git sbcl
  • Windows:
    • pacman -S git mingw-w64-x86_64-sbcl
    • choco install git sbcl

Once this step is successful, you should be able to type sbcl --help and see something similar to the following:

Usage: sbcl [runtime-options] [toplevel-options] [user-options]
Common runtime options:
  --help                     Print this message and exit.
  --version                  Print version information and exit.
  --core <filename>          Use the specified core file instead of the default.
  --dynamic-space-size <MiB> Size of reserved dynamic space in megabytes.
  --control-stack-size <MiB> Size of reserved control stack in megabytes.
  --tls-limit                Maximum number of thread-local symbols.

Common toplevel options:
  --sysinit <filename>       System-wide init-file to use instead of default.
  --userinit <filename>      Per-user init-file to use instead of default.
  --no-sysinit               Inhibit processing of any system-wide init-file.
  --no-userinit              Inhibit processing of any per-user init-file.
  --disable-debugger         Invoke sb-ext:disable-debugger.
  --noprint                  Run a Read-Eval Loop without printing results.
  --script [<filename>]      Skip #! line, disable debugger, avoid verbosity.
  --quit                     Exit with code 0 after option processing.
  --non-interactive          Sets both --quit and --disable-debugger.
Common toplevel options that are processed in order:
  --eval <form>              Form to eval when processing this option.
  --load <filename>          File to load when processing this option.

User options are not processed by SBCL. All runtime options must
appear before toplevel options, and all toplevel options must
appear before user options.

For more information please refer to the SBCL User Manual, which
should be installed along with SBCL, and is also available from the
website <http://www.sbcl.org/>.

Step 2. Install ocicl

Follow the instructions at https://github.com/ocicl/ocicl.

Briefly:

  • MacOS: brew install ocicl && ocicl setup
  • Others:
git clone https://github.com/ocicl/ocicl
cd ocicl
sbcl --load setup.lisp
ocicl setup

Edit the compiler init file (eg. ~/.sbclrc) to include the current directory, or

#-ocicl
(when (probe-file #P"/Users/user/.local/share/ocicl/ocicl-runtime.lisp")
  (load #P"/Users/user/.local/share/ocicl/ocicl-runtime.lisp"))
(asdf:initialize-source-registry
 ;; (list :source-registry
 ;;       (list :tree (uiop:strcat (uiop:getenv "HOME") "/Common Lisp/"))
 ;;       :inherit-configuration)
 (list :source-registry
       (list :directory (uiop:getcwd))
       :inherit-configuration))

Step 3. Moonli

  1. 3.1. Obtain the source

    git clone https://github.com/digikar99/moonli
    
  2. 3.2. Start the REPL

    sbcl --eval '(asdf:load-system "moonli-repl")' --eval '(cl-repl:main)'
    
  3. 3.3a. (Optional) Building basic binary

    The following should create a moonli binary in the root directory of moonli.

    (asdf:make :moonli)
    
    ./moonli --help
    A basic moonli transpiler over SBCL
    Available options:
      -h, --help                 Print this help text
      -l, --load-lisp ARG        Load lisp file
      -m, --load-moonli ARG      Load moonli file
      -t, --transpile-moonli ARG
                                 Transpile moonli file to lisp file
    
  4. 3.3b. (Optional) Build REPL

    The following should create a moonli.repl binary in the root directory of moonli.

    (asdf:make :moonli/repl)
    

2 - Introduction to Moonli for Common Lispers

What

This is a syntax layer that transpiles to Common Lisp.

defun sum(args):
  if null(args):
    0
  else:
    first(args) + sum(rest(args))
  end
end

transpiles to:

(defun sum (args)
   (cond ((null args)
          0)
         (t
          (+ (first args)
             (sum (rest args))))))

See ./moonli-sample.asd and ./sample/sample.moonli to include in your project.

Table of Contents

Why

Due to tools like paredit and lispy (beyond macros and metaprogramming), s-expression (Lists) based syntax of lisps is very powerful. However, not every one has the time or patience to become comfortable with them, especially when it comes to reading code, or sharing it with your colleagues.

In the 21st century, very many more people are familiar with python, matlab and julia than they are with lisps. Given the power and flexibility of common lisp, moonli is an attempt to provide a thin syntax layer over common lisp. It is thin in the sense it can be easily transpiled to common lisp. The semantics remain the same and clean as common lisp. (Common Lisp is also good for reasons beyond macros.)

Features

For common lispers

  • Case sensitive, but invert-case reader to maintain common lisp compatibility
  • Transpile to common lisp, so lispers need not "learn a new language"
  • Extensible using moonli:define-moonli-macro and moonli:define-short-moonli-macro. See ./src/macros/ directory for examples.
  • Inability to access internal symbols of another package through "A::B" syntax; this syntax rather translates to (the B A)

For programmers in general

  • Sane variable scoping rules as given by common lisp
  • Sane namespace scoping thanks to common lisp package system
  • Sane restarts and condition system thanks to common lisp
  • Optional typing, optional dynamic scoping
  • Availability of optimizing compilers such as SBCL
  • Sensitive to newlines and semicolons but not to spaces and tabs (indentation insensitive)
  • Returning multiple values without an intermediate data structure
  • Support for rapid prototyping through CLOS and image-based development

Here's a brief comparison of features across different languages.

    FEATURES                             MOONLI       COMMON LISP       JULIA       HASKELL         RUST         PYTHON       JAVASCRIPT         C  
  ----------------------------------- ------------ ----------------- ----------- ------------- -------------- ------------ ---------------- -----------
  Syntax                                    +              +              +            +            ---             +              -             -
  Interactivity (Rapid Prototyping)       High         Very High      Moderate        Low           None        Moderate       Moderate        None
  Typing (Strong/Weak)                   Strong         Strong         Strong       Strong         Strong        Strong          Weak          Weak
  Typing (Static/Dynamic)               Flexible       Flexible       Flexible      Static         Static       Dynamic        Dynamic        Dynamic
  Typing (Expressivity)                 Flexible       Flexible       Moderate     Very High     Very High        Low            Low            Low
  Compiler Speed                        Flexible       Flexible         Slow       Moderate         Slow        Moderate       Moderate      Moderate
  Runtime Speed                         Flexible       Flexible         Fast       Moderate         Fast          Slow         Moderate        Fast
  Runtime Error Recovery                Advanced       Advanced        Limited     Moderate         None        Moderate       Moderate        None
  Binary Size                           Flexible       Flexible         Large          ?           Small          None           None          Small
  User Extensibility                      High           High         Moderate        Low           Low           None           None          None
  Compiler built-in optimizations         Low             Low         Very High        ?         Very High        Low          Moderate      Very High
  Long Term Support                       Low          Very High      Moderate         ?          Moderate      Moderate         Low         Very High
  Ecosystem (without interop)            Small           Small        Moderate       Small        Moderate       Large          Large          Large
  Memory Management                       Heap           Heap         Reference      Heap       Compile Time   Reference          ?           Manual

Plan

  • Real numbers, strings, characters, lists, infix arithmetic operators, literal hash-tables, literal hash-sets
  • Typing using "expr::type" operator
  • Support for declare and declaim
  • Literal syntax for vectors, array access
  • BODMAS rule for parsing expressions
  • Binaries
  • VS Code integration
  • Emacs mode and integration with slime
  • Infix Logical operators
  • Add more forms: progn, mvb, dsb, let+, more…
  • Add more tests
  • Reverse transpile from common lisp

Syntax

As with lisp, everything is an expression.

Simple syntax table:

  Lisp                       Moonli
  -------------------------- -------------------------
  #\a                        'a'
  "hello world"              "hello world"
  2, 2.0, 2d-3, 2.0d-3       2, 2.0, 2d-3, 2.0d-3
  'quoted-symbol             $quoted-symbol
  package:exported-symbol    package:exported-symbol
  package::internal-symbol   <WONTDO>
  (the type expr)            expr :: type
  (list form-1 form-2)       (form-1, form-2)
  (fn arg1 arg2)             fn(arg1, arg2)
  #c(re, im)                 <TODO>

Global variables

defparameter *global* = 23

Local variables

let answer-to-everything = 42 :
  answer-to-everything
end

Symbols

Most valid symbols can be written in moonli. For example, above *global* and answer-to-everything are each single symbols. This is unlike mainstream languages where * - ? ! and several other characters are not allowed in symbols.

However, this means that symbols must be separated from each other by space. This is necessary to make a distinction between whether a character stands for an infix operation or is part of a symbol. a+b is a single symbol, but a + b is translated to the lisp expression (+ a b).

Function-like calls

identity("hello world")
function(identity)

Because lisp macros and functions follow similar syntax, moonli syntax for function calls can also be used for macro calls when the macro syntax is simple. (Indeed, this can be inconvenient; see [defining your own]{.spurious-link target=“defining your own”}.)

destructuring-bind(a(b),(1,2),+(1,2))

transpiles to

(destructuring-bind (a b) (list 1 2)
  (+ 1 2))

Functions

Like lisp, return is implicit.

defun fib(n):
  if n < 0:
     error("Don't know how to compute fib for n=~d < 0", n)
  elif n == 0 or n == 1:
     1
  else:
    fib(n - 1) + fib(n - 2)
  end
end

Dictionaries or Hash-tables

{
  :a : 2,
  "b": $cl:progn
}

transpiles to

(fill-hash-table (:a 2) ("b" 'progn))

which expands to

(let ((#:hash-table413 (make-hash-table :test #'equal :size 2)))
  (setf (gethash :a #:hash-table413) 2
        (gethash "b" #:hash-table413) 'progn)
  #:hash-table413)

Sets or Hash-sets

{:a, "b" , $cl:progn}

transpiles to

(fill-hash-set :a "b" 'progn)

which expands to

(let ((#:hash-set417 (make-hash-table :test #'equal :size 3)))
  (setf (gethash :a #:hash-set417) t
        (gethash "b" #:hash-set417) t
        (gethash 'progn #:hash-set417) t)
  #:hash-set417)

Infix operators

The following infix operators are recognized:

  • + - * / ^
  • or and not
  • < <= == != >= >

lm

lm (): nil

transpiles to

(lambda () nil)
lm (x): x

transpiles to

(lambda (x) x)
lm (x, y): x + y

transpiles to

(lambda (x y) (+ x y))

declaim

declaim inline(foo)

transpiles to

(declaim (inline foo))
declaim type(hash-table, *map*)

transpiles to

(declaim (type hash-table *map*))

declare

declare type(single-float, x, y)

transpiles to

(declare (type single-float x y))
declare type(single-float, x, y), optimize(debug(3))

transpiles to

(declare (type single-float x y)
         (optimize (debug 3)))

ifelse

ifelse a 5

transpiles to

(if a
    5
    nil)
ifelse a :hello :bye

transpiles to

(if a
    hello
    bye)

lambda

lambda (): nil end

transpiles to

(lambda () nil)
lambda (x):
  x
end

transpiles to

(lambda (x) x)
lambda (x, y):
  let sum = x + y:
    sum ^ 2
  end
end

transpiles to

(lambda (x y)
  (let ((sum (+ x y)))
    (expt sum 2)))

let-plus:let+

let-plus:let+ x = 42: x
end

transpiles to

(let+ ((x 42))
  x)
let-plus:let+ (a,b) = list(1,2):
  a + b
end

transpiles to

(let+ (((a b) (list 1 2)))
  (+ a b))
let-plus:let+ let-plus:&values(a,b) = list(1,2):
  a + b
end

transpiles to

(let+ (((&values a b) (list 1 2)))
  (+ a b))
let-plus:let+
  let-plus:&values(a,b) = list(1,2),
  (c,d,e) = list(1,2,3):
  {a,b,c,d,e}
end

transpiles to

(let+ (((&values a b) (list 1 2)) ((c d e) (list 1 2 3)))
  (fill-hash-set a b c d e))

loop

loop end loop

transpiles to

(loop)
loop :repeat n :do
  print("hello")
end

transpiles to

(loop repeat n
      do (print hello))
loop :for i :below n :do
  print(i + 1)
end

transpiles to

(loop for i below n
      do (print (+ i 1)))

defun

defun our-identity(x): x end

transpiles to

(defun our-identity (x) x)
defun add (&rest, args):
 args
end defun

transpiles to

(defun add (&rest args) args)
defun add(args):
  if null(args):
    0
  else:
    first(args) + add(rest(args))
  end if
end

transpiles to

(defun add (args) (cond ((null args) 0) (t (+ (first args) (add (rest args))))))
defun foo(&optional, a = 5): a end

transpiles to

(defun foo (&optional (a 5)) a)

if

if a: b end if

transpiles to

(cond (a b) (t))
if a:
  b; c
end

transpiles to

(cond (a b c) (t))
if a: b
else: c
end if

transpiles to

(cond (a b) (t c))
if a:
   b; d
else:
   c; e
end if

transpiles to

(cond (a b d) (t c e))
if a: b
elif c: d; e
else: f
end if

transpiles to

(cond (a b) (c d e) (t f))
(if a: b else: c; end)::boolean

transpiles to

(the boolean (cond (a b) (t c)))
if null(args): 0; else: 1 end

transpiles to

(cond ((null args) 0) (t 1))
if null(args):
    0
else:
    first(args)
end if

transpiles to

(cond ((null args) 0) (t (first args)))
if null(args):
  0
else:
  2 + 3
end if

transpiles to

(cond ((null args) 0) (t (+ 2 3)))
if null(args):
  0
else:
  first(args) + add(rest(args))
end if

transpiles to

(cond ((null args) 0) (t (+ (first args) (add (rest args)))))

let

let a = 2, b = 3:
   a + b
end

transpiles to

(let ((a 2) (b 3))
  (+ a b))
let a = 2, b = 3:
   a + b
end let

transpiles to

(let ((a 2) (b 3))
  (+ a b))

for:for

for:for (i,j) in ((1,2),(3,4)):
  print(i + j)
end

transpiles to

(for (((i j) in (list (list 1 2) (list 3 4))))
  (print (+ i j)))
for:for i in (1,2,3), j in (2,3,4):
  print(i + j)
end

transpiles to

(for ((i in (list 1 2 3)) (j in (list 2 3 4)))
  (print (+ i j)))

3 - Tutorial

3.1 - 0. Evaluation

Starting Moonli either using the included binary or using VS Code will start a Read–Eval–Print Loop (REPL).

In VS Code, you will need to switch to a separate tab that contains the REPL. TODO: Elaborate more on evaluation in VS Code.

MOONLI-USER> 

A REPL is how many interactive programming languages work. It’s a simple cycle: the computer reads what you type, evaluates it (figures out what it means and runs it), prints the result, and then loops back to wait for your next command. For example, if you type 2 + 3 in the REPL, it reads the expression, evaluates it to get 5, prints that result, and waits for more input.

MOONLI-USER> 2 + 3
[OUT]: 5

MOONLI-USER> 

This loop makes programming feel conversational – you can test ideas instantly, explore code step by step, and see exactly how the language thinks and responds.

For any input expression, you can prevent evaluation by prefixing it with $. This is known as quote-ing.

MOONLI-USER> $x
[OUT]: x
MOONLI-USER> $(1 + 2)
[OUT]: (+ 1 2)

Note that for $(1 + 2), even though evaluation has been avoided, the expression is printed as it appears to Moonli internally, just before the step of evaluation. What you type at the REPL is converted to an internal representation that Moonli can then work with.

3.2 - 1. Literal Objects

In a programming language, a literal object is something the evaluator doesn’t need to compute—it already is its own value. When the REPL reads a literal like 5, “hello”, or #t, the eval step simply returns it unchanged, because it directly represents itself.

For example, since 5 is already a number:

MOONLI-USER> 5
[OUT]: 5

By contrast, when you type an expression like 2 + 3, the result is a new value obtained through the process of evaluation. Literals skip that process entirely.

MOONLI-USER> 2 + 3
[OUT]: 5

Moonli has several kinds of literal objects.

Literal Objects in Moonli

Symbols

A symbol is a name that represents something else, like a label attached to a value or concept. In Moonli, symbols are the basic building blocks of code.

For example, recall the x that you typed at the REPL. It was a symbol. When you had quoted the symbol x, Moonli returned the symbol itself.

MOONLI-USER> $x
[OUT]: x

A special kind of symbols that do not require to be quoted are keywords.

MOONLI-USER> :i-am-a-keyword
[OUT]: :i-am-a-keyword

However, keywords require to be prefixed with a colon :. Note also that symbols in Moonli can contain hyphens.

We will look at symbols in more detail in the next chapter.

Numbers

Recall from high school mathematics, that there can be different kinds of numbers, like integers, and real numbers. Programming languages can also represent different kinds of numbers. Two important ones include integers and floats.

Integers are whole numbers without fractions – like -3, 0, or 42. They’re exact and good for counting or discrete steps.

MOONLI-USER> 42
[OUT]: 42

Floating-point numbers (or floats) represent real numbers that may include decimals – like 3.14 or -0.001. They’re useful for measurements or continuous values but can lose precision because they’re stored in binary form.

When evaluated, both kinds of numbers are literal objects – they evaluate to themselves. So typing 3.14 in a REPL simply returns 3.14, already fully evaluated.

MOONLI-USER> 3.14
[OUT]: 3.14
MOONLI-USER> -0.001
[OUT]: -0.001

Note that it is important that there is no space between - and 0.001 for Moonli to understand it as -0.001.

Strings

Strings are sequences of characters used to represent text – like “hello”, “42”, or “Moonli rocks!”. They’re written between quotation marks so the evaluator knows they’re text, not symbols or code.

MOONLI-USER> "Moonli rocks!"
[OUT]: "Moonli rocks!"

When the REPL reads a string, it treats it as a literal object, meaning it already represents its own value and needs no further evaluation. For example, typing "cat" simply returns "cat".

MOONLI-USER> "cat"
[OUT]: "cat"

Strings can contain letters, digits, spaces, or even special symbols, and most languages let you combine (concatenate) or inspect them with built-in functions. They’re essential for displaying messages, storing words, or communicating with users.

While symbols are useful for working with code, strings are useful for working with text.

Characters

Characters can be understood as blocks of strings. Each string is a sequence of characters. An individual character can be input to the REPL using single quotation marks.

MOONLI-USER> 'a'
[OUT]: #\a

3.3 - 2. Symbols, Variables, and Values

A critical aspect of programming is building abstractions. The first step to such abstractions involves using variables to stand in for literal values that we studied in the last section.

Programmatically, variables are symbols that can be eval-uated to obtain the value they are bound to. You have already seen a few symbols. Below, x and :i-am-a-keyword are two symbols.

MOONLI-USER> $x
[OUT]: x
MOONLI-USER> :i-am-a-keyword
[OUT]: :i-am-a-keyword

Recall that the $-prefix was used to quote the symbols to prevent their eval-uation. (Recall the initial section on evaluation.) Recall also that keywords are symbols that begin with a colon :, and they do not need to be quoted. What happens when you omit the $-prefix for non-keywords?

MOONLI-USER> x
unbound-variable: The variable x is unbound.

This time, instead of receiving the output, we received an error message. It tells us that the REPL does not know how to evaluate x. We can provide REPL this information by defparameter.

MOONLI-USER> defparameter x = 42
[OUT]: x

This tells the REPL to bind the symbol x to the literal value 400. After this, if you type the unquoted x, even without the $-prefix, you do not receive an error. Instead, the REPL tells you the value that the symbol is bound to.

MOONLI-USER> x
[OUT]: 42

You can check whether a symbol is bound to using boundp. The output t means true. Indeed the symbol x is bound to some value!

MOONLI-USER> boundp($x)
[OUT]: t

On the other hand, unless you had defined the value of y using defparameter beforehand, the symbol y would be unbound. This is indicated by the output nil which means false. The symbol y is not bound to any value.

MOONLI-USER> boundp($y)
[OUT]: nil

The idea of a variable is conceptual. It is something we use to talk about or describe programming. On the other hand, for Moonli (and for Lisps in general), symbols are programmatic objects that we can manipulate. We will revisit this idea later.

The bindings of the variables can be updated. In other words, variables can be assigned new values. This is a frequent part of programming. For example:

MOONLI-USER> x = 84
[OUT]: 84
MOONLI-USER> x
[OUT]: 84

Global and Local

Variables defined using defparameter are global variables.

Global variables are accessible from anywhere within the program. This means anyone can assign them new values or change the values they are assigned to. So, if used frequently, programs can be hard to understand because it will be difficult to figure out where a particular variable is being reassigned.

That is why, the more common approach to using a variable is by using local variables. In Moonli, These can be defined using let and let+. We show an example below.

MOONLI-USER> let a = 1, b = 2:
  a + b
end
[OUT]: 3
MOONLI-USER> let+ a = 1, b = 2:
  a + b
end
[OUT]: 3

One difference between let and let+ is that let can be described as performing parallel binding, while let+ can be described as performing sequential binding.

The following works with let+ but not let.

MOONLI-USER> let+ a = 1, b = a:
  a + b
end
[OUT]: 2

MOONLI-USER> let a = 1, b = a:
  a + b
end
; in: progn (let ((a 1) (b a))
;          (+ a b))
;     (MOONLI-USER::B MOONLI-USER::A)
;
; caught warning:
;   undefined variable: moonli-user::a
;
; compilation unit finished
;   Undefined variable:
;     a
;   caught 1 WARNING condition
unbound-variable: The variable a is unbound.

With let, we get the error that the variable a is unbound. This is because let binds all its variables as if they are made at once. In let a = 1, b = a, we are asking the REPL to bind b to its value at the same time as a. But the value of b is told to be a. At that point in the program, the value of a is unavailable, resulting in the error.

The algorithm for let can be written as:

  1. Compute the values of all the expressions assigned to the variables, without assigning them.
  2. Bind the variables to the values of the respective expressions
  3. Execute the body. (In this case the body is simply a + b. But it could be any valid Moonli code.)
  4. Unbind the variables.

In contrast the algorithm for let+ can be written as:

  1. Bind the first variable to the value of the corresponding expression.
  2. Bind the second variable to the value of the corresponding expression. … Repeat the same for all variables …
  3. Execute the body. (In this case the body is simply a + b. But it could be any valid Moonli code.)
  4. Unbind the variables.

It is generally recommended to use let since it often allows you to think about the value of each variable independently of the other variables. But when you can’t use let, feel free to use let+.

Global variables should have earmuffs

Suppose you see an arbitrary variable in some code. How can you tell whether it is global or local? Different programming languages or projects have different conventions. For Moonli (and Lisps), it is recommended that global variables should have earmuffs *...* around them.

Thus one should write

MOONLI-USER> defparameter *x* = 42
[OUT]: *x*
MOONLI-USER> *x*
[OUT]: 42

Instead of

MOONLI-USER> defparameter x = 42
[OUT]: x

Unbinding using makunbound

One can remove the binding of a variable using makunbound:

MOONLI-USER> makunbound($x)
[OUT]: x
MOONLI-USER> x
unbound-variable: The variable x is unbound.
MOONLI-USER> boundp($x)
[OUT]: nil

While it is okay to use makunbound in the REPL, it is recommended to avoid using it in the code you write and save in files and share with others. Creation, deletion, re-creation is harder to understand that a single creation. Use local variables wherever possible.

Variables and Abstraction

Suppose you had a program to multiply 23 with itself thrice. You program would then be a single file with the following single line of code:

print(23 * 23 * 23)

Now, suppose you wanted to change this program to multiply 47 by itself thrice. You’d need to make changes in three places. The new code would look like this.

print(47 * 47 * 47)

But, with the use of variables, you can achieve the same with just a single change!

The following program multiples 23 with itself thrice:

let x = 23:
  print(x * x * x)
end

It can be changed to multiply 47 with itself thrice by making a single change!

let x = 47:
  print(x * x * x)
end

Instead of saying “multiply 23 by itself”, now you are saying “multiply x by itself (whatever x may be)”. This replacement of something specific (23) by something general (a variable x) is essentially abstraction.

Abstractions can help to keep code simple – it allows code to be reused.

3.4 - 3. Functions and Abstractions

Besides variables that we studied in the last chapter, the second key element to abstraction are functions.

Even if you write abstract code such as the one in the last chapter, you will need to write it again and again for different values of x:

let x = 23:
  print(x * x * x)
end

let x = 47:
  print(x * x * x)
end

Functions help us get rid of this code repetition and further enable reuse. Functions are characterized by parameters, which are essentially lists of variables. For example, the below code defines a function multiply-thrice that has a single parameter x:

defun multiply-thrice(x):
  print(x * x * x)
end

It takes this parameter, multiplies it with itself thrice, and returns the result.

MOONLI-USER> multiply-thrice(23)

12167
[OUT]: 12167
MOONLI-USER> multiply-thrice(47)

103823
[OUT]: 103823

Thus, the lines of code multiply-thrice(23) or multiply-thrice(47) achieves the same effect as the two blocks of let we used earlier.

The full code looks something like the following:

defun multiply-thrice(x):
  print(x * x * x)
end

multiply-thrice(23)
multiply-thrice(47)

There are many built-in functions that Moonli provides. print is one such function. You may have noticed that the result is printed twice in the REPL. This is because of print. If you omit the print, the REPL would look something like this:

MOONLI-USER> defun new-multiply-thrice(x):
  x * x * x
end
[OUT]: new-multiply-thrice
MOONLI-USER> new-multiply-thrice(23)
[OUT]: 12167
MOONLI-USER> new-multiply-thrice(47)
[OUT]: 103823

A function can span multiple lines of code. By default, it returns the value resulting from the evaluation of the last line of code (excluding end). For our new-multiply-thrice, this refers to the value of x * x * x. For our old multiply-thrice, it would have been print(x * x * x).

print(x * x * x) first multiplies x with itself thrice. Prints the result. And returns it. You can see this in action if you instantiate x with a concrete value such as 23 or 47:

MOONLI-USER> print(23 * 23 * 23)
12167
[OUT]: 12167
MOONLI-USER> print(47 * 47 * 47)
103823
[OUT]: 103823

Since print(x * x * x) was the last line of muliply-thrice, the return value of multiply-thrice is the same as the return value of print(x * x * x).

MOONLI-USER> multiply-thrice(23)

12167
[OUT]: 12167
MOONLI-USER> multiply-thrice(47)

103823
[OUT]: 103823

Similarly, the following code defines an add function that takes in two parameters x and y. It multiplies the values of these parameters and returns the result.

MOONLI-USER> defun add(x,y):
  x + y
end
[OUT]: add
MOONLI-USER> add(2,3)
[OUT]: 5

Error: Invalid number of arguments

What happens if you call add with just a single argument?

MOONLI-USER> add(2)
simple-program-error: invalid number of arguments: 1
Backtrace for: #<SB-THREAD:THREAD tid=259 "main thread" RUNNING {7005490613}>
0: (MOONLI-USER::ADD 2) [external]
1: (SB-INT:SIMPLE-EVAL-IN-LEXENV (MOONLI-USER::ADD 2) #<NULL-LEXENV>)
2: (SB-INT:SIMPLE-EVAL-IN-LEXENV (PROGN (MOONLI-USER::ADD 2)) #<NULL-LEXENV>)
...

The first line of this says that there was an error in the code you asked the REPL to evaluate. In particular, the code has invalid number of arguments. Similarly, what happens when if you call multiply-thrice with three arguments? This results in a similar error:

MOONLI-USER> multiply-thrice(2,3,4)
simple-program-error: invalid number of arguments: 3
Backtrace for: #<SB-THREAD:THREAD tid=259 "main thread" RUNNING {7005490613}>
0: (MOONLI-USER::MULTIPLY-THRICE 2 3 4) [external]
1: (SB-INT:SIMPLE-EVAL-IN-LEXENV (MOONLI-USER::MULTIPLY-THRICE 2 3 4) #<NULL-LEXENV>)
...

Calling other functions

Notice how you called print function from inside the multiply-thrice function? Calling other functions from one function is very common. You can also write another function that uses both new-multiply-thrice and add:

defun multiply-thrice-and-add(x,y):
  let x-cubed = new-multiply-thrice(x),
      y-cubed = new-multiply-thrice(y):
    add(x-cubed, y-cubed)
  end
end

The return value of multiply-thrice-and-add is determined by add(x-cubed, y-cubed) since that is the last line of code (excluding end).

You can see what this function is doing in more detail by printing the values of the variables in intermediate steps. This can be done using the format function. We will explain format in more detail later:

defun multiply-thrice-and-add(x,y):
  let x-cubed = new-multiply-thrice(x),
      y-cubed = new-multiply-thrice(y):
    format(t, "x-cubed has value: ~a~%", x-cubed)
    format(t, "y-cubed has value: ~a~%", y-cubed)
    add(x-cubed, y-cubed)
  end
end

You can see the intermediate steps:

MOONLI-USER> multiply-thrice-and-add(2,3)
x-cubed has value: 8
y-cubed has value: 27
[OUT]: 35

Usually, writing something as a function instead of repeating it needlessly everywhere is helpful because it enables reuse.

3.5 - 4. Packages and Namespaces

Suppose you write some code and share it with other people. Or you want to use code written by other people. It may happen that both of you are using the same function names but are doing different things. For example, your process-data function might be doing something different that somebody else’ process-data function.

To deal with this, most modern languages implement the concept of namespaces. The implementation of namespaces in Moonli (and Common Lisp) is made by a data structure called package. One way to find or identify packages is using strings.

MOONLI-USER> find-package("CL")
[OUT]: #<package "COMMON-LISP">
MOONLI-USER> find-package("COMMON-LISP")
[OUT]: #<package "COMMON-LISP">
MOONLI-USER> find-package("MOONLI")
[OUT]: #<package "MOONLI">

The above says that "CL" and "COMMON-LISP" are both two different names of the same package. On the other hand, the string "MOONLI" identifies a different package. Note that package names are case sensitive. This means "moonli" and "MOONLI" identify different packages. In fact, if you are following the tutorial, so far there is no package named "moonli", but only the package named "MOONLI":

MOONLI-USER> find-package("moonli")
[OUT]: nil

A package maps symbol names to symbols. Symbol names are again strings. You can use find-symbol to find a symbol in a particular package. For example, the following finds the symbol named "LIST" in package named "CL".

MOONLI-USER> find-symbol("LIST", "CL")
[OUT]: list
:external

Note that the second argument to find-symbol can be either a package name (a string) or the package itself, for example:

MOONLI-USER> let pkg = find-package("CL"):
  print(pkg)
  find-symbol("LIST", pkg)
end

#<package "COMMON-LISP">
[OUT]: list
:external

In fact, the second argument is optional. When you do not supply it, find-symbol finds the symbol in the package bound to the special variable *package*.

MOONLI-USER> find-symbol("LIST", "MOONLI-USER")
[OUT]: list
:external
MOONLI-USER> *package*
[OUT]: #<package "MOONLI-USER">
MOONLI-USER> find-symbol("LIST")
[OUT]: list
:external

You may wonder whether the symbol list above belongs to the package named "MOONLI-USER" or to the package named "COMMON-LISP" given that they look identical. To identify the package which a symbol belongs to, you can use the function symbol-package. Note that the symbol list is quoted by prefixing it with $, since we want to refer to the symbol itself and not the value it is bound to:

MOONLI-USER> symbol-package($list)
[OUT]: #<package "COMMON-LISP">

This says that the symbol list belongs to the package "COMMON-LISP". To confirm:

MOONLI-USER> let sym = find-symbol("LIST", "MOONLI-USER"):
  symbol-package(sym)
end
[OUT]: #<package "COMMON-LISP">

Indeed, the symbol named "LIST" in package "MOONLI-USER" has as its package "COMMON-LISP"!

In fact, packages provide much more than simple mapping. They provide a way to organize the symbols themselves. Packages also have the notion of internal and external symbols. Above, the second return value :external shows that the list symbol is external to the respective packages.

While writing your own code, you should define your own package as the first step. You can define new packages using defpackage. This takes in a number of options, a few of them include:

  • :use: which specifies preexisting packages the new package should use. This means that all the external symbols of these packages will be used in the new package.
  • :export: which specifies which should be the external symbols of this new package. These are the symbols that you want to make available to other users of your code.
MOONLI-USER> defpackage "TUTORIAL"
  :use "CL", "MOONLI-USER";
  :export "ADD";
end
[OUT]: #<package "TUTORIAL">

This specifies that the new package named "TUTORIAL" uses the external symbols of the existing packages "CL" and "MOONLI-USER". It also specifies that the package "TUTORIAL" has a symbol named "ADD" as an external symbol. We can switch to the new package using in-package.

MOONLI-USER> in-package tutorial
[OUT]: #<package "TUTORIAL">
TUTORIAL>

You will notice that the prompt MOONLI-USER> has changed to TUTORIAL>. The prompt also indicates the which package you are currently in. Once you have defined a new package, you can use in-package to switch to that package (or any other) no matter in which file you are in. This does not require quoting, because in-package is a special form (also called a macro).

You may note that we had said package names are case sensitive. Yet, we used the symbol tutorial to refer to the package named "TUTORIAL". In Moonli, the symbols are related to their names by case inversion. This is for backward compatibility with Common Lisp and to enable the use of Common Lisp libraries with minimal disruption. The Common Lisp convention is to distinguish symbols not by their case, but by making the names themselves distinct regardless of case. However, Moonli wants to provide better names while interfacing with C and Python libraries, thus, the Moonli parser is case sensitive.

The external symbols from another package can be used by prefixing the symbol name with the package name and using a colon : as a separator (without spaces). For example, the function ensure-list in package alexandria takes any object, and if it is not a list, it wraps it into a list. You can access this symbol using alexandria:ensure-list.

TUTORIAL> listp((1,2,3))
[OUT]: t
TUTORIAL> alexandria:ensure-list((1,2,3))
[OUT]: (1 2 3)
TUTORIAL> alexandria:ensure-list("a")
[OUT]: ("a")
TUTORIAL> alexandria:ensure-list(42.0)
[OUT]: (42.0)