Lexical Structure

Language design. Generated coverage names measured programs and target restrictions; examples here illustrate the design.

Source Text

Dewy source is Unicode text. Source-file suffixes are conventional and do not alter tokenization or semantic rules.

Case

Dewy is case-sensitive. Keywords (let, loop, if), word operators (and, or, not, in?, is?), the booleans true and false, and symbols are matched exactly as written: True, AND, and Loop are ordinary identifiers, distinct from each other and from true, and, and loop. Case does not matter in two places, both where the strict reading would silently mean something else: inside a numeral, the alphabetic digits of a based integer (0xff and 0xFF) and the exponent marker (1e5 and 1E5) may be written either way, while the base prefix itself is lowercase (0x, 0b; 0X is not a prefix); and the two string escapes that take digits, \u/\U (a scalar, \u{1F600}) and \x/\X (a hex byte escape, which Dewy rejects), are recognized in either case. Every other escape is lowercase (\n), and a backslash before any other character is that character itself (\N is N).

Identifiers

An identifier contains at least one base character. Decorations may appear before or after that base character, and decimal digits may follow it.

The current base repertoire contains ASCII Latin letters, the ordinary Greek alphabet, _, ‾, !, °, and selected mathematical letter symbols such as ℂ, ℕ, ℤ, ℚ, and ℝ. Decorations include the supported Unicode superscript and subscript letters and digits, prime marks, and ℠, ™, ©, and ®.

Identifiers are case-sensitive. The micro sign µ and Greek mu μ name the same binding. ASCII and Greek letters remain distinct: A is not Α.

An underscore followed by decimal digits spells subscript digits; an overline ‾ (U+203E) followed by decimal digits spells superscript digits. Adjacent runs combine, and the ordinary Unicode spelling names the same binding:

let x_12 = 42       # also x_1_2 or x₁₂
let x‾12 = 7       # also x¹²; distinct from x₁₂ and x12
let foo_2_bar = 3  # also foo₂_bar; foo_bar stays unchanged

This rule applies only to digits. Write letter decorations directly (xᵢ, xᵀ); x_i remains a different name. Superscripts in identifiers are labels, not exponentiation. Normalization does not change string contents or the source spans used in diagnostics.

The full Unicode repertoire and additional equivalences remain open. A doubled marker as an escape is still a design question.

Reserved operator words such as and, or, not, in, as, and transmute tokenize as operators in their grammatical contexts. A word operator cannot simultaneously be used as an ordinary identifier in that context.

The names extern, intrinsic, none, void, end, and new are reserved. They cannot name variables, parameters, fields, methods, loop bindings, type aliases, or import aliases. Their existing expression and type roles remain: none is the absence value/type, void is a no-value result, and xs[end] uses the last valid index. <none> in a match arm tests the type and does not introduce a binding. untyped is not reserved; the compiler's internal inference marker does not reserve a source name. Reserving new does not yet implement the planned singleton-axis indexing operation.

Whitespace and Juxtaposition

Whitespace separates tokens. Dewy does not generally use commas to separate arguments, parameters, or array elements.

Spacing can also determine whether expressions are juxtaposed with a punctuation operator. Range endpoints are the clearest example:

first..last    # both endpoints
first ..last   # no left endpoint
first.. last   # no right endpoint
first .. last  # no endpoints

Newlines normally behave as whitespace. A construct may assign additional structural meaning to line boundaries only where its grammar explicitly says so. One does: return and yield take a value only from their own line, so at a line end they return nothing — if done return followed by a statement on the next line returns, and the statement is not the returned value. A value that needs several lines starts on the keyword's line (return ( … )). A ; ends a bare return on the same line.

Comments

# begins a line comment. #{ and }# delimit a nestable block comment.

# one line
#{ outer
   #{ nested }#
   outer again
}#

Comment markers inside strings are string contents.

A documentation string is an ordinary call of the prelude's doc with a string — usually a """ block — at the top of a module, a function body, or a type. The compiler keeps nothing from it yet, so the call is a no-op:

doc"""
Tokenizer framework.
"""

Tokens and Ambiguity

Tokenization chooses the longest valid token subject to explicit lexical rules. Whitespace always separates expressions. At head(...), the head's type decides once between a call and a juxtaposed multiplication (see Operators and Precedence); no alternative parses are kept.

See Literals for literal tokens and Operators and Precedence for expression grouping.