Part 1: Core Language Specification

1.1 Lexical Structure

Character Set

udewy source code is ASCII. Bytes outside the ASCII range may only appear inside string literals, path literals, and comments; the compiler never interprets them. Inside string literals they pass through to the program verbatim (see String Literals).

Valid ASCII characters:

• Letters: A-Z, a-z
• Digits: 0-9
• Symbols: ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ { | } ~
• Whitespace: space (0x20), tab (0x09), carriage return (0x0D), newline (0x0A)

Whitespace

Whitespace characters (space, tab, carriage return, newline) are ignored except as token separators. There are no significant indentation or newline rules.

Comments

Only line comments are supported. A # character begins a comment that extends to the end of the line:

# This is a comment
let x:int = 42  # inline comment

Block comments are not supported.

Two comment forms carry metadata for debuggers. A line of the form # @loc path:line:column names the source position of the statements that follow it (until the next such line); a compiler that emits udewy from another language uses it to point a debugger at the original source. Without markers, a statement's position is its own line in the udewy file. A line of the form # @var name shown formatter type right before a declaration (or before a function, for its parameters) describes that variable to the debugger: the name it is shown under (- keeps the declared name), a function in the program a debugger may call to render the value as text (- for none; it takes the variable's word and returns the address of a [length][bytes] block), and its type as spelled, to the end of the line; # @var name - hides the declaration (a compiler's temporary). Without a marker a variable is shown under its own name with its udewy annotation. Both are debug information only — a backend that ignores them produces the same program — and reach the backend through the mark_location, note_local, begin_scope, and end_scope hooks of the Backend protocol; the x86_64 backend emits DWARF line and variable information from them.

Identifiers

Identifiers consist of letters, digits, and underscores, and must begin with a letter or underscore:

identifier ::= [a-zA-Z_][a-zA-Z0-9_]*

Identifiers are case-sensitive. The following are reserved keywords and cannot be used as identifiers:

let  const  if  else  loop  break  continue  return
import  extern  transmute  and  or  xor  not  true  false  void

NOTE: import is a preprocessing-only directive. It is recognized before any actual code; any import that reaches tokenization is an error.

Number Literals

udewy supports three number formats. All produce 64-bit integer values. Underscore separators (_) are allowed anywhere within the digit sequence for readability.

Decimal integers:

42
1_000_000
0

Hexadecimal integers (prefix 0x, digits case-insensitive):

0xff
0x1a2b_3c4d
0xDEAD_BEEF

Binary integers (prefix 0b):

0b1010
0b1111_0000

All number literals are unsigned and must fit within 64 bits.

Boolean Literals

Boolean values are represented as specific 64-bit integer patterns:

This representation allows bitwise operators (and, or, xor, not) to function correctly as both bitwise and logical operators.

String Literals

Strings are enclosed in double quotes. Strings may span multiple lines:

let msg:int = "Hello, World!"
let multi:int = "This string
spans multiple lines"

Escape sequences: A backslash followed by certain characters produces special values:

The last rule means \" produces a double-quote character and \\ produces a backslash. Hex byte escapes insert a single raw byte; for example, "\x41" is the byte 0x41 ('A'), and "\xce\xbc" is the UTF-8 encoding of μ.

Line continuation: A backslash immediately before a newline causes that newline to be skipped:

let long:int = "\
This is a very \
long string that appears \
on one line"

Non-ASCII content: Aside from escape sequences, the bytes between the quotes are copied into the literal verbatim. The compiler performs no encoding validation, conversion, or normalization, so a literal contains exactly the bytes of the source file. Since source files are conventionally UTF-8, "μZero" produces the UTF-8 encoding of (0xCE 0xBC 0x5A 0x65 0x72 0x6F) -- 2 initial bytes for μ followed by ascii Zero. A source file saved in another encoding would pass its raw bytes through just the same in that encoding rather than UTF-8. The length prefix counts bytes, not characters or codepoints; all text-encoding interpretation is left to the program. A backslash followed by a non-ASCII character passes that character's bytes through unchanged.

NOTE: for an explicit byte-stable version with a fixed encoding regardless of the underlying file encoding, use byte escapes for any non-ASCII content, e.g. "\xce\xbcZero".

Memory layout: String literals are stored in static memory with an 8-byte length prefix. The variable holds a pointer to the first character (after the length). See Memory Layout.

Based String Literals

Based strings use 0b"..." or 0x"..." to write exact bytes directly:

let bits:int = 0b"1010_0001 11"       # a1 c0
let header:int = 0x"de ad
    # comments may separate digits
    be ef"

Based strings are ordinary expressions, not directives or a special declaration form. Only base 2 and base 16 are supported, and the prefixes are exactly lowercase 0b and 0x. Binary bodies accept 0 and 1; hexadecimal bodies accept 0-9, a-f, and A-F.

Whitespace, underscores, and # line comments are separators and contribute no bits. After removing those separators, digits are packed in source order using MSB-first wire order: the first digit supplies the highest bit or nibble of the first byte, then packing continues toward the low bits before moving to the next byte. If the final byte is incomplete, its remaining low bits are automatically zero-filled on the right. Thus 0b"1" is the byte 0x80, 0b"1111_0000 1" is 0xF0 0x80, and 0x"a b c" is 0xAB 0xC0.

Because a # comment continues through the newline, a closing quote on the same line is part of the comment; place the closing quote on a later line.

No target-endian conversion is performed. The bytes in static storage are exactly this wire-order sequence on every backend. A based string carries no element type, dimensions, or shape metadata; consumers are responsible for knowing the data's format.

Ordinary strings and based strings share the same static-storage layout: an 8-byte byte-length immediately before the data, with the expression evaluating to a pointer to the first data byte. __load__(ptr - 8) is therefore the byte length for both forms.

Path Literals

Path Literals use the p"..." syntax. They are only recognized by the import preprocessor and are not part of the regular token stream or parser grammar:

import p"utils.udewy"
import p"../lib/helpers.udewy"

Path literals support the same escape sequences as regular strings.

The void Keyword

void is a special keyword representing the absence of a meaningful value. It is primarily used in:

return void              # return from a void function
let f = ():>void => {}   # declare a void-returning function