diff --git a/content/docs/guide/architecture.mdx b/content/docs/guide/architecture.mdx new file mode 100644 index 0000000..127c0f7 --- /dev/null +++ b/content/docs/guide/architecture.mdx @@ -0,0 +1,48 @@ +--- +title: Compiler Architecture +description: High-level pipeline overview of the Mist compiler and crate organization. +icon: Building2 +--- + +### Pipeline Overview + +The Mist compiler operates in distinct phases: + +``` +Source (.mist) + │ + ▼ +┌─────────────┐ +│ Lexer │ (pest PEG grammar) +│ & Parser │ +└─────────────┘ + │ AST + ▼ +┌─────────────┐ +│ Semantics │ (field init checking) +└─────────────┘ + │ + ▼ +┌─────────────┐ +│ Codegen │ (AST → Rust source) +└─────────────┘ + │ .rs + .map.json + ▼ +┌─────────────┐ +│ cargo │ (Rust compilation) +└─────────────┘ + │ + ▼ + Binary / Library +``` + +### Crate Organization + +The compiler is written in Rust and Mist itself, organized into four main crates: + +| Crate | Language | Role | +|-------|----------|------| +| **mist-parser** | Rust | PEG parsing via pest, AST construction, semantic checks, position mapping | +| **mist-codegen** | Rust | AST → Rust source code generation | +| **mist-analyzer** | Rust | Language server bridging mist-editor ↔ rust-analyzer | +| **mist-api** | Mist | Orchestrates transpilation, module tree building, cargo invocation, error remapping | diff --git a/content/docs/guide/attributes.mdx b/content/docs/guide/attributes.mdx new file mode 100644 index 0000000..5365448 --- /dev/null +++ b/content/docs/guide/attributes.mdx @@ -0,0 +1,32 @@ +--- +title: Attributes +description: Inner and outer attributes for modules, items, and metadata. +icon: Hash +--- + +Inner attributes apply to the containing module: + +```mist +#![allow(unused_variables)] +``` + +Outer attributes apply to the next item: + +```mist +#[derive(Debug, Clone)] +struct Point { + i32 x, + i32 y, +} + +#[test] +void my_test() +{ + assert_eq!(1, 1); +} +``` + +Attribute syntax: +- `#[path]` +- `#[path = literal]` +- `#[path(item1, item2, ...)]` diff --git a/content/docs/guide/classes.mdx b/content/docs/guide/classes.mdx new file mode 100644 index 0000000..378fd4b --- /dev/null +++ b/content/docs/guide/classes.mdx @@ -0,0 +1,69 @@ +--- +title: Classes +description: Class declarations, constructors, inheritance, virtual dispatch, and the override keyword. +icon: Shapes +--- + +Mist introduces `class` as syntactic sugar for a Rust struct with a virtual method table (vtable). + +```mist +pub class Animal { + str& name, + + pub constructor(str& name) + { + self.name = name; + } + + pub void speak(&self) + { + println!("..."); + } +} +``` + +Class fields can have default initializers: + +```mist +class Player { + i32 health = 100, + str& name, +} +``` + +### Inheritance + +```mist +class Dog : Animal { + pub constructor(str& name) + { + super(name); + } + + override void speak(&self) + { + println!("Woof!"); + } +} +``` + +The `override` keyword supports explicit base class targeting: + +```mist +override(Animal) void speak(&self) +{ + println!("Woof!"); +} +``` + +### Under the Hood + +A class `Dog : Animal` generates: + +1. A Rust struct with a `_super: Animal` field (or `_vptr: &'static [*const c_void]` for root classes) +2. A vtable constant with function pointers for each public method +3. An `impl` block with `Deref` and `DerefMut` +4. Method trampolines (`__m_`) that are dispatched through the vtable +5. A `new()` constructor that initializes via `MaybeUninit` and calls the user's `constructor(&mut self)` + +The vtable is unified: parent entries are copied, overridden entries replace parent slots, and new methods are appended. diff --git a/content/docs/guide/control-flow.mdx b/content/docs/guide/control-flow.mdx new file mode 100644 index 0000000..effa561 --- /dev/null +++ b/content/docs/guide/control-flow.mdx @@ -0,0 +1,87 @@ +--- +title: Control Flow +description: If/else, while, for, c-style for, loop, match, break, continue, and return. +icon: ArrowLeftRight +--- + +### If/Else + +```mist +if condition { + // body +} else if other_condition { + // body +} else { + // body +} +``` + +Conditions must be parenthesized: `if (expr) { }` or `if expr_no_struct { }` (struct literals require parentheses). + +Used as an expression: + +```mist +let x = if true { 1 } else { 2 }; +``` + +### While + +```mist +while condition { + // body +} +``` + +### For + +```mist +for pattern in iterator { + // body +} + +for i in 0 .. 10 { + // body +} +``` + +### C-Style For + +```mist +for (let mut i = 0; i < 10; i++) { + // body +} +``` + +### Loop + +```mist +loop { + // infinite loop +} +``` + +### Match + +```mist +match value { + pattern1 => expr, + pattern2 => { + // block body + } + pattern3 | pattern4 => expr, +} +``` + +### Break / Continue + +```mist +break; +continue; +``` + +### Return + +```mist +return; +return value; +``` diff --git a/content/docs/guide/functions.mdx b/content/docs/guide/functions.mdx new file mode 100644 index 0000000..c01e6b7 --- /dev/null +++ b/content/docs/guide/functions.mdx @@ -0,0 +1,58 @@ +--- +title: Functions +description: Function declarations, self parameters, return types, and unsafe functions. +icon: Function +--- + +```mist +// Return type is `void` (unit) +void greet(str& name) +{ + println!("Hello, {name}"); +} + +// With return type +i32 add(i32 a, i32 b) +{ + return a + b; +} + +// Expression body (last expression is the return value) +i32 square(i32 x) +{ + x * x +} + +// Public function +pub i32 multiply(i32 a, i32 b) +{ + a * b +} + +// Generic function +T identity(T value) +{ + value +} + +// Unsafe function +unsafe i32 dangerous() +{ + 42 +} +``` + +Function syntax: `[pub] [void | ] []() [override] { }` + +Functions use **Allman brace style** (opening brace on the next line). + +### Self Parameter + +Methods can take `self`, `&self`, `&mut self`, or `self` with lifetimes: + +```mist +pub void set_value(&mut self, i32 v) +{ + self.value = v; +} +``` diff --git a/content/docs/guide/generics.mdx b/content/docs/guide/generics.mdx new file mode 100644 index 0000000..458f696 --- /dev/null +++ b/content/docs/guide/generics.mdx @@ -0,0 +1,39 @@ +--- +title: Generics +description: Generic functions, structs, enums, trait bounds, and lifetimes. +icon: Braces +--- + +```mist +// Generic function +T id(T x) +{ + x +} + +// Generic struct +struct Pair { + A first, + B second, +} + +// Generic enum +enum Result { + Ok(T), + Err(E), +} + +// Generic with trait bounds +T max(T a, T b) +{ + if a > b { a } else { b } +} + +// Lifetime generics +void process<'a>(&'a str& data) +{ + // ... +} +``` + +Generic syntax uses `<` `>` delimiters. Lifetimes are prefixed with `'`. diff --git a/content/docs/guide/modules.mdx b/content/docs/guide/modules.mdx new file mode 100644 index 0000000..0f036a9 --- /dev/null +++ b/content/docs/guide/modules.mdx @@ -0,0 +1,29 @@ +--- +title: Modules & Imports +description: Module declarations, imports, re-exports, and module resolution rules. +icon: FolderTree +--- + +```mist +// Declare a submodule +pub module foo; + +// Import +use std::collections::HashMap; + +// Re-exporting import +pub use my_module::MyType; +``` + +### Module Resolution + +Mist maps the module tree to Rust's module system: + +| Mist Path | Rust Output | +|------------------------------|------------------------------| +| `src/main.mist` | `.mist/src/main.rs` | +| `src/utils/package.mist` | `.mist/src/utils/mod.rs` | +| `src/foo.mist` | `.mist/src/foo.rs` | +| `src/utils/helper.mist` | `.mist/src/utils/helper.rs` | + +A `package.mist` file acts as a directory's module root, analogous to `mod.rs`. diff --git a/content/docs/guide/operators.mdx b/content/docs/guide/operators.mdx new file mode 100644 index 0000000..d8e0f58 --- /dev/null +++ b/content/docs/guide/operators.mdx @@ -0,0 +1,98 @@ +--- +title: Operators, Macros & Closures +description: Binary, prefix, and postfix operators, macro calls, and closure expressions. +icon: Plus +--- + +### Binary Operators + +| Operator | Description | +|----------|----------------------| +| `+` | Addition | +| `-` | Subtraction | +| `*` | Multiplication | +| `/` | Division | +| `%` | Modulus | +| `==` | Equality | +| `!=` | Inequality | +| `<` | Less than | +| `>` | Greater than | +| `<=` | Less or equal | +| `>=` | Greater or equal | +| `&&` | Logical AND | +| `\|\|` | Logical OR | +| `&` | Bitwise AND | +| `\|` | Bitwise OR | +| `^` | Bitwise XOR | +| `<<` | Left shift | +| `>>` | Right shift | +| `=` | Assignment | +| `+=` | Add assign | +| `-=` | Subtract assign | +| `*=` | Multiply assign | +| `/=` | Divide assign | +| `%=` | Modulus assign | +| `&=` | Bitwise AND assign | +| `\|=` | Bitwise OR assign | +| `^=` | Bitwise XOR assign | +| `<<=` | Left shift assign | +| `>>=` | Right shift assign | +| `..` | Range (exclusive) | +| `..=` | Range (inclusive) | +| `->` | Pointer write | + +### Prefix Operators + +| Operator | Description | +|----------|-------------------| +| `*` | Dereference | +| `&` | Reference | +| `&mut` | Mutable reference | +| `!` | Logical NOT | +| `-` | Numeric negation | + +### Postfix Operators + +| Operator | Description | +|------------|----------------------| +| `.field` | Field access | +| `.0` | Tuple field access | +| `()` | Function call | +| `[]` | Index | +| `{ f: v }` | Struct literal | +| `as Type` | Type cast | +| `?` | Try (error prop) | +| `++` | Increment | +| `--` | Decrement | +| `!()` | Macro call (paren) | +| `![]` | Macro call (bracket) | +| `!{}` | Macro call (brace) | + +### Macros + +Mist reuses Rust's macro system directly: + +```mist +println!("hello"); +assert_eq!(a, b); +vec![1, 2, 3]; +``` + +Macro calls use `!` followed by parentheses, brackets, or braces. + +### Closures + +```mist +let add = (a, b) => a + b; +let result = add(2, 3); // 5 + +let square = f64 x => x * x; +``` + +Closures can have explicit return types: + +```mist +let transform = i32 x => { + x * 2 +}; +``` diff --git a/content/docs/guide/patterns.mdx b/content/docs/guide/patterns.mdx new file mode 100644 index 0000000..05f76d5 --- /dev/null +++ b/content/docs/guide/patterns.mdx @@ -0,0 +1,41 @@ +--- +title: Patterns +description: Pattern matching with literals, tuples, structs, named tuples, wildcards, and mutable bindings. +icon: Split +--- + +```mist +// Literal patterns +match x { + 1 => "one", + 2 => "two", + _ => "other", +} + +// Tuple patterns +let (a, b) = (1, 2); + +// Struct patterns +match value { + Point { x, y } => x + y, + Point { x: 0, y } => y, + _ => 0, +} + +// Named tuple patterns (newtype) +let MyType(value) = my_var; + +// Wildcard / etc +let _ = get_side_effect(); +match x { + 1 => ..., + .. => ..., // rest / etc +} + +// Mutable binding in pattern +let mut x = 42; +match ref_to_option { + Some(mut value) => value += 1, + None => {}, +} +``` diff --git a/content/docs/guide/structs-enums.mdx b/content/docs/guide/structs-enums.mdx new file mode 100644 index 0000000..e047bb3 --- /dev/null +++ b/content/docs/guide/structs-enums.mdx @@ -0,0 +1,47 @@ +--- +title: Structs & Enums +description: Defining structs and enums with named, tuple, and struct variants. +icon: Box +--- + +### Structs + +```mist +pub struct Point { + i32 x, + i32 y, +} + +pub struct Generic { + T value, +} +``` + +Fields can be public or private: + +```mist +struct User { + pub str& name, + i32 age, // private +} +``` + +### Enums + +```mist +pub enum Option { + Some(T), + None, +} + +pub enum Message { + Quit, + Move(i32, i32), + Write { str& content, i32 length }, +} +``` + +Enum variants can be: +- **Named** — `Variant` +- **Tuple** — `Variant(T1, T2)` +- **Struct** — `Variant { T1 field1, T2 field2 }` diff --git a/content/docs/guide/syntax-overview.mdx b/content/docs/guide/syntax-overview.mdx new file mode 100644 index 0000000..361de9c --- /dev/null +++ b/content/docs/guide/syntax-overview.mdx @@ -0,0 +1,45 @@ +--- +title: Syntax Overview +description: Comments, literals, identifiers, and keywords in Mist. +icon: Code +--- + +### Comments + +```mist +// Line comments only +``` + +### Literals + +```mist +42 // Integer +3.14 // Float +true // Boolean +false // Boolean +"hello" // String +(1, true, "x") // Tuple +``` + +### Identifiers & Keywords + +Keywords are reserved and cannot be used as identifiers: + +`if`, `else`, `fn`, `for`, `while`, `match`, `return`, `break`, `continue`, `struct`, `enum`, `class`, `trait`, `impl`, `use`, `pub`, `mut`, `let`, `true`, `false`, `dyn`, `loop`, `unsafe`, `override`, `const`, `type` + +Identifiers follow the pattern `[a-zA-Z_][a-zA-Z0-9_]*`. + +### Visibility + +```mist +// Private (default) +void internal() { } + +// Public +pub void external() { } + +// Public to specific path +pub(crate) void crate_only() { } +pub(super) void parent_only() { } +pub(in my::module) void module_only() { } +``` diff --git a/content/docs/guide/traits-impls.mdx b/content/docs/guide/traits-impls.mdx new file mode 100644 index 0000000..4a6370a --- /dev/null +++ b/content/docs/guide/traits-impls.mdx @@ -0,0 +1,42 @@ +--- +title: Traits & Impls +description: Defining traits, trait bounds, and implementation blocks. +icon: Puzzle +--- + +### Traits + +```mist +pub trait Drawable { + void draw(&self); +} + +pub trait Comparable : Eq { + i32 cmp(&self, T other); +} +``` + +Trait requirements are specified after `:`: + +```mist +trait MyTrait : SuperTrait + OtherTrait { + void required_method(&self); + void another(&self); +} +``` + +### Impl Blocks + +```mist +impl MyType { + void method(&self) { } +} + +impl Trait for MyType { + void method(&self) { } +} + +impl GenericTrait for MyType { + void method(&self, T value) { } +} +``` diff --git a/content/docs/guide/types.mdx b/content/docs/guide/types.mdx new file mode 100644 index 0000000..3b047fe --- /dev/null +++ b/content/docs/guide/types.mdx @@ -0,0 +1,47 @@ +--- +title: Types +description: The Mist type system including references, pointers, tuples, function types, and trait objects. +icon: Type +--- + +### Type System + +```mist +i32 // Path type +str& // Reference type +&mut i32 // Mutable reference +&'a i32 // Reference with lifetime +*const i32 // Const pointer +*mut i32 // Mutable pointer +(i32, bool) // Tuple type +fn(i32) -> bool // Function pointer type +Fn(i32) -> bool // Closure trait (Fn) +FnMut(i32) -> bool // Closure trait (FnMut) +FnOnce(i32) -> bool // Closure trait (FnOnce) +dyn Trait // Trait object +void // Unit type (maps to Rust's ()) +'lifetime // Lifetime +``` + +Type expressions can be composed: + +```mist +type_expr = { (void | path_type | tuple_type | dyn_type) ~ (unsafe_ref_type | ref_type | fn_type)* } +``` + +This means types are written left-to-right naturally: + +```mist +i32& // &i32 +i32&mut // &mut i32 +i32&'a // &'a i32 +i32 fn() -> bool // fn(i32) -> bool +``` + +### Type Aliases + +```mist +type MyInt = i32; + +type Result = std::result::Result; +``` diff --git a/content/docs/guide/variables.mdx b/content/docs/guide/variables.mdx new file mode 100644 index 0000000..e75148a --- /dev/null +++ b/content/docs/guide/variables.mdx @@ -0,0 +1,30 @@ +--- +title: Variables +description: Variable declarations, mutability, type annotations, and destructuring. +icon: Variable +--- + +```mist +let x = 42; // Type-inferred immutable +let mut y = 10; // Mutable variable +i32 z = 100; // Explicit type annotation +str& s = "hello"; // Typed string reference +bool b = true; // Typed boolean +f64 f = 3.14; // Typed float +let (a, b) = (1, "two"); // Destructuring +let (x, (y, z)) = (1, (2, 3)); // Nested destructuring +``` + +Variable declarations follow either of two forms: + +```mist +let [= ]; + [= ]; +``` + +### Const and Static + +```mist +const MAX: i32 = 100; +static NAME: str& = "hello"; +``` diff --git a/content/docs/internals/builder.mdx b/content/docs/internals/builder.mdx new file mode 100644 index 0000000..4d2f785 --- /dev/null +++ b/content/docs/internals/builder.mdx @@ -0,0 +1,34 @@ +--- +title: Builder & Error Remapping +description: How Mist invokes cargo and remaps Rust compiler errors back to Mist source locations. +icon: Bug +--- + +The builder module (`builder.mist` in the `mist_api` crate) wraps `cargo` to: + +1. Spawn `cargo` with `--message-format=json` +2. Parse JSON compiler messages +3. For each diagnostic span, look up the corresponding `.map.json` file +4. Remap Rust line/column to Mist line/column using the mapping +5. Display errors/warnings with Mist source locations and context lines + +```rust +pub fn build(args: Vec, root: PathBuf) -> bool { + // Spawn cargo with JSON output + // Parse CompilerMessage for each span + // Look up rev_mapper::Mapping from .map.json + // Remap to Mist positions + // Print diagnostics +} +``` + +### Diagnostic Output + +Errors and warnings are displayed with Mist source context: + +``` +src/main.mist:10:5 + Error: mismatched types + let x: i32 = "hello"; + ^^^^^^^ +``` diff --git a/content/docs/internals/codegen.mdx b/content/docs/internals/codegen.mdx new file mode 100644 index 0000000..56fba93 --- /dev/null +++ b/content/docs/internals/codegen.mdx @@ -0,0 +1,53 @@ +--- +title: Code Generation +description: How the Mist AST is translated into Rust source code, including class vtables and position mapping. +icon: Wrench +--- + +The code generator converts the Mist AST into Rust source code directly — no intermediate representation. + +### Key files + +- `src/lib.rs` — `RustCodegen` struct with output buffer, indentation tracking, `Mapping` for position translation +- `src/top_level.rs` — Generates Rust for top-level items (structs, enums, traits, functions, impls, imports, type aliases, const/static) +- `src/statement.rs` — Generates Rust for statements and blocks +- `src/expr.rs` — Generates Rust for expressions (literals, paths, binary ops, closures, arrays, prefix/postfix) +- `src/class_decl.rs` — Class-specific code generation (struct + vtable + impl blocks + Deref) + +### Codegen Design + +The `RustCodegen` maintains: +- A string `output` buffer +- An `indent` level (4 spaces per level) +- A `Mapping` that records `(RustMap, MistMap)` pairs for source position remapping +- A current `position` tracker (`RustMap(line, column)`) + +Generation follows the `GenRust` trait: + +```rust +pub trait GenRust { + fn gen_rust(&self, ctx: &mut Context, cg: &mut RustCodegen); +} +``` + +And `GetRust` for simple string-returning types: + +```rust +pub trait GetRust { + fn get_rust(&self) -> String; +} +``` + +The `Context` carries optional expression path information for class `super` / `Super` resolution. + +### Class Codegen + +Classes are the most complex codegen path. `ClassProcessedData` analyzes a class declaration and emits: + +1. **Struct declaration** — `struct ClassName { pub _super: Parent, pub field1: T1, ... }` (or `_vptr` for root classes) +2. **Vtable constants** — Index constants `__FN_METHOD` and a `__V_TABLE` static array of function pointers. For inherited classes, parent vtable entries are copied and overridden entries replaced. +3. **Constructor** — `pub fn new(...) -> Self` that creates an uninitialized instance via `MaybeUninit`, sets the vtable pointer, writes field defaults, calls `self.constructor(...)`, and returns `this` +4. **Method trampolines** — Public methods with `self` get wrapper functions `__m_method` stored in the vtable, plus virtual dispatch methods +5. **Override support** — Methods marked `override` are validated at compile time via generated test code +6. **Deref impls** — `impl Deref for Child` and `DerefMut` for inherited classes +7. **Impl declarations** — Inner `impl` blocks are rewritten to use the self type diff --git a/content/docs/internals/lsp.mdx b/content/docs/internals/lsp.mdx new file mode 100644 index 0000000..25b0092 --- /dev/null +++ b/content/docs/internals/lsp.mdx @@ -0,0 +1,55 @@ +--- +title: LSP Support +description: How the Mist Language Server bridges editor requests through rust-analyzer. +icon: Server +--- + +The Mist Language Server Protocol implementation runs a headless `rust-analyzer` instance and bridges Mist editor requests to Rust positions. + +### Architecture + +``` +Editor (LSP client) + │ + ▼ +┌─────────────────────┐ +│ mist-analyzer │ +│ (Rust + Mist) │ +└─────────────────────┘ + │ ▲ + │ JSON-RPC │ + ▼ │ +┌─────────────────────┐ +│ rust-analyzer │ +│ (headless child) │ +└─────────────────────┘ +``` + +### Flow + +1. Editor sends Mist file edits to `mist-analyzer` via LSP +2. `mist-analyzer` transpiles Mist to Rust +3. The transpiled Rust is forwarded to the headless `rust-analyzer` process +4. For goto-definition, hover, and completion, a unique marker token is injected at the cursor position +5. The transpiled output with the marker is sent to `rust-analyzer` +6. The marker position is located in the response +7. Results are mapped back to Mist positions using the `rev_mapper` + +### Key Features + +- **Full document sync** — Open, change, save, close +- **Go to definition** — Maps through transpiled Rust positions +- **Completions** — Supports trigger characters `:`, `.`, `'`, `(` +- **Hover** — Type information and docs +- **Diagnostics** — Real-time errors from transpilation failures and Rust compilation +- **Formatting** — Document formatting support +- **Auto-import** — Automatically inserts `pub module ;` when new `.mist` files are created +- **File watching** — Monitors `**/*.mist` for new files + +### Diagnostic Remapping + +When a transpilation error occurs in the `mist-analyzer`: + +1. Parse errors are converted to `Diagnostic` with Mist source positions +2. Semantic errors (uninitialized class fields) use the stored line/column +3. Rust compiler errors are remapped via the `Mapping` system back to Mist positions diff --git a/content/docs/internals/parsing.mdx b/content/docs/internals/parsing.mdx new file mode 100644 index 0000000..840abc6 --- /dev/null +++ b/content/docs/internals/parsing.mdx @@ -0,0 +1,76 @@ +--- +title: Parsing +description: PEG grammar structure, AST design, and how Mist source code is parsed. +icon: FileCode +--- + +The parser is built with [pest](https://pest.rs), a PEG parser generator. Grammar rules are defined in `grammar.pest` (693 rules). + +### Key files + +- `grammar.pest` — PEG grammar defining the entire language syntax +- `src/parser/common/` — Parse rule → AST conversions for expressions, statements, types, declarations +- `src/parser/items/` — Parse rule → AST conversions for top-level items (structs, enums, classes, functions, traits, impls, attributes) +- `src/ast/` — AST node types (expr.rs, statement.rs, top_level.rs) +- `src/error.rs` — Error types (PreAst parsing errors, Ast generation errors) + +### Parsing entry points + +```rust +// Parse a complete program +pub fn parse<'a>(source: &'a str) -> Result> + +// Parse only the module declaration +pub fn parse_module<'a>(source: &'a str) -> Result, ParseError<'a>> +``` + +### Grammar Structure + +The grammar follows a layered approach: + +1. **Lexical rules** (silent) — `WHITESPACE`, `COMMENT`, `identifier`, `integer`, `float`, `string_lit` +2. **Primary expressions** — literals, paths, tuples, arrays, closures, control flow +3. **Term** — prefix operators + primary + postfix operators +4. **Expression** — terms joined by binary operators (via Pratt parser) +5. **Statements** — variable declarations, control flow (`if`, `while`, `for`, `match`, `loop`), blocks +6. **Top-level items** — functions, structs, enums, classes, traits, impls, type aliases, imports, module declarations, constants + +### AST Structure + +The AST preserves source positions via `Spanned`: + +```rust +pub struct Spanned { + pub line: usize, + pub column: usize, + pub item: T, +} +``` + +Top-level items are wrapped as: + +```rust +pub struct TopLevel(pub Spanned, pub Vec); +``` + +Expressions use a fix-point representation for prefix/postfix operators: + +```rust +Expression::Fix { + initial: Box, + prefixes: Vec, + postfixes: Vec, +} +``` + +Binary expressions use Pratt parsing for correct precedence: + +```rust +Expression::Binary { + lhs: Box, + op: String, + rhs: Box, +} +``` + +All operators are left-associative with a single precedence level. diff --git a/content/docs/internals/position-mapping.mdx b/content/docs/internals/position-mapping.mdx new file mode 100644 index 0000000..58603fd --- /dev/null +++ b/content/docs/internals/position-mapping.mdx @@ -0,0 +1,31 @@ +--- +title: Position Mapping +description: Bidirectional mapping between Mist source positions and Rust output positions for error remapping and LSP features. +icon: Map +--- + +Mist maintains a bidirectional mapping between Mist source positions and Rust output positions. This is essential for: + +1. **Error remapping** — Rust compiler errors point to the correct Mist source location +2. **LSP features** — Go-to-definition, hover, and completion work from Mist source + +### Data Structure + +The mapping is stored as pairs: + +```rust +pub struct Mapping { + pub mist_path: PathBuf, + pub map: HashSet<(RustMap, MistMap)>, +} +``` + +- `RustMap(usize, usize)` — line/column in generated Rust +- `MistMap(usize, usize)` — line/column in original Mist + +### Mapping Lifecycle + +1. **Codegen** — Each `Spanned` AST node records its Mist position and the current Rust position in the codegen output buffer via `GenSpanTranslation` +2. **Persistence** — The mapping is serialized to `.map.json` alongside the transpiled `.rs` output +3. **Build-time remapping** — The builder reads `.map.json` to remap `cargo` diagnostics back to Mist positions +4. **LSP remapping** — Both Mist→Rust and Rust→Mist direction queries are supported via `find()` and `find_by_mist()` diff --git a/content/docs/internals/semantic-analysis.mdx b/content/docs/internals/semantic-analysis.mdx new file mode 100644 index 0000000..0359913 --- /dev/null +++ b/content/docs/internals/semantic-analysis.mdx @@ -0,0 +1,36 @@ +--- +title: Semantic Analysis +description: Class field initialization verification and the GetMutability analysis system. +icon: SearchCheck +--- + +The semantic checker (`semantics.rs`) performs class field initialization analysis. When a class has a constructor, it verifies that every declared field is mutated (directly or indirectly via method calls) within the constructor body. + +### Field Initialization + +The primary semantic check ensures all class fields are initialized in the constructor: + +```mist +class Player { + i32 health, + str& name, + + pub constructor(str& name) + { + self.name = name; + // Error: field 'health' is uninitialized + } +} +``` + +### How It Works + +The `GetMutability` trait tracks which identifiers are mutated: + +1. Collects all identifiers that get `&mut` references +2. Tracks `self.field = value` patterns +3. Follows method calls that might initialize fields (transitively) +4. Ensures all conditional branches initialize the same fields (intersection semantics) +5. `super` assignments count for `_super` field initialization + +If any field is uninitialized after the analysis, a `SemanticError` is reported with the field's source position. diff --git a/content/docs/internals/transpiler.mdx b/content/docs/internals/transpiler.mdx new file mode 100644 index 0000000..c7d6256 --- /dev/null +++ b/content/docs/internals/transpiler.mdx @@ -0,0 +1,61 @@ +--- +title: Transpilation Pipeline +description: How Mist source files are discovered, transpiled to Rust, and cached. +icon: Repeat +--- + +The transpiler (`transpiler.mist` in `mist_api`) orchestrates the full pipeline. + +### Pipeline + +1. Read `Mist.toml` configuration +2. Build a `Module` tree from the filesystem (discovering `package.mist` files) +3. For each module: + a. Parse Mist source + b. Run semantic checks + c. Generate Rust code and position mapping + d. Write `.rs` file and `.map.json` file +4. Recursively transpile included crates + +### Module Tree + +The `Module` struct represents the file-to-module mapping: + +```rust +pub class Module { + pub String name; + pub PathBuf path; + pub Vec children; + + pub constructor(PathBuf mist_path, MistConfig& config) { ... } + pub bool is_package(&self) { ... } + pub PathBuf output_dir(&self, PathBuf& parent_dir) { ... } + pub PathBuf output_path(&self, PathBuf& parent_dir, ...) { ... } +} +``` + +### Transpilation Mapping Rules + +| Mist Source | Rust Output | +|----------------------------|----------------------------------| +| `src/main.mist` | `.mist/src/main.rs` | +| `src/foo.mist` | `.mist/src/foo.rs` | +| `src/bar/package.mist` | `.mist/src/bar/mod.rs` | +| `src/bar/baz.mist` | `.mist/src/bar/baz.rs` | + +A `package.mist` file generates `mod.rs` in the output directory and contains `pub mod ;` declarations for its children. + +### Caching + +The transpiler implements basic caching via file modification times: + +```rust +fn is_source_newer(source: &Path, output: &Path) -> io::Result { + if !output.exists() { return Ok(true); } + let source_time = fs::metadata(source)?.modified()?; + let output_time = fs::metadata(output)?.modified()?; + Ok(source_time > output_time) +} +``` + +If a source file has not changed since the last transpilation, the `.rs` file is not regenerated. diff --git a/content/docs/meta.json b/content/docs/meta.json index 7be92bd..b70c4ff 100644 --- a/content/docs/meta.json +++ b/content/docs/meta.json @@ -1,8 +1,31 @@ { "pages": [ - "---[Rocket]Introduction---", + "---Introduction---", "index", "philosophy", - "limitations" + "limitations", + "---Language Guide---", + "guide/syntax-overview", + "guide/variables", + "guide/types", + "guide/functions", + "guide/control-flow", + "guide/structs-enums", + "guide/classes", + "guide/traits-impls", + "guide/generics", + "guide/modules", + "guide/patterns", + "guide/operators", + "guide/attributes", + "---Compiler Internals---", + "internals/architecture", + "internals/parsing", + "internals/semantic-analysis", + "internals/codegen", + "internals/transpiler", + "internals/builder", + "internals/lsp", + "internals/position-mapping" ] -} +} \ No newline at end of file diff --git a/docs.md b/docs.md deleted file mode 100644 index 3cabe0e..0000000 --- a/docs.md +++ /dev/null @@ -1,1131 +0,0 @@ -# Mist Language Documentation - -## Introduction - -Mist is a pragmatic systems programming language that compiles directly to Rust. It provides C/C++-style syntax with zero-cost abstractions, full interoperability with the Rust ecosystem, and additional language features like classes with inheritance. - -The compiler is written in Rust and Mist itself, and is organized into four main crates: - -- **mist-parser** — Parses Mist source code into an AST using pest -- **mist-codegen** — Generates Rust code from the Mist AST -- **mist-analyzer** — Language server built on top of rust-analyzer -- **mist-api** — Mist crate that orchestrates transpilation and building - ---- - -## Installation - -```bash -cargo install mist-lang -``` - -This installs the `mist` CLI binary and the `mist-analyzer` LSP binary. - ---- - -## Project Structure - -A Mist project requires the following structure: - -``` -my-project/ -├── Mist.toml # Project configuration -├── Cargo.toml # Standard Rust Cargo.toml (managed by mist) -├── src/ -│ ├── main.mist # Entry point (or lib.mist for libraries) -│ └── ... # Other .mist source files -└── .mist/ - └── src/ # Generated Rust output (do not edit) -``` - -### Mist.toml - -```toml -package = "main.mist" # Entry point file (relative to src/) -packages = ["utils"] # Subdirectories to treat as sub-modules -include = ["crates/*"] # Additional Mist crates to transpile -``` - -The `packages` field lists subdirectories under `src/` that should be treated as child modules. Each such directory should contain a `package.mist` file acting as the module root. - -The `include` field supports glob patterns for including external Mist crates. A trailing `*` treats all immediate subdirectories that contain a `Mist.toml` as independent crates. - ---- - -## CLI Commands - -| Command | Short | Description | -|------------------|-------|---------------------------------| -| `mist run` | `r` | Run the project | -| `mist build` | `b` | Build the project | -| `mist check` | `c` | Check without building | -| `mist test` | | Run tests | -| `mist publish` | | Publish the project | -| `mist bench` | | Benchmark the project | -| `mist doc` | | Build documentation | -| `mist fix` | | Auto-fix warnings | -| `mist clippy` | | Lint the project | -| `mist clean` | | Clean build artifacts | -| `mist transpile` | `t` | Transpile Mist to Rust only | -| `mist init` | | Init a project in current dir | -| `mist new` | | Create a new project | -| `mist version` | `-v` | Print compiler version | -| `mist help` | `-h` | Print usage information | - -All Cargo subcommands delegate to `cargo` with Mist error remapping. - ---- - -## Language Syntax - -### Comments - -```mist -// Line comments only -``` - -### Literals - -```mist -42 // Integer -3.14 // Float -true // Boolean -false // Boolean -"hello" // String -(1, true, "x") // Tuple -``` - -### Identifiers & Keywords - -Keywords are reserved and cannot be used as identifiers: - -`if`, `else`, `fn`, `for`, `while`, `match`, `return`, `break`, `continue`, `struct`, `enum`, `class`, `trait`, `impl`, `use`, `pub`, `mut`, `let`, `true`, `false`, `dyn`, `loop`, `unsafe`, `override`, `const`, `type` - -Identifiers follow the pattern `[a-zA-Z_][a-zA-Z0-9_]*`. - ---- - -### Variables - -```mist -let x = 42; // Type-inferred immutable -let mut y = 10; // Mutable variable -i32 z = 100; // Explicit type annotation -str& s = "hello"; // Typed string reference -bool b = true; // Typed boolean -f64 f = 3.14; // Typed float -let (a, b) = (1, "two"); // Destructuring -let (x, (y, z)) = (1, (2, 3)); // Nested destructuring -``` - -Variable declarations follow either of two forms: - -```mist -let [= ]; - [= ]; -``` - ---- - -### Functions - -```mist -// Return type is `void` (unit) -void greet(str& name) -{ - println!("Hello, {name}"); -} - -// With return type -i32 add(i32 a, i32 b) -{ - return a + b; -} - -// Expression body (last expression is the return value) -i32 square(i32 x) -{ - x * x -} - -// Public function -pub i32 multiply(i32 a, i32 b) -{ - a * b -} - -// Generic function -T identity(T value) -{ - value -} - -// Unsafe function -unsafe i32 dangerous() -{ - 42 -} -``` - -Function syntax: `[pub] [void | ] []() [override] { }` - -Functions use **Allman brace style** (opening brace on the next line). - -### Self Parameter - -Methods can take `self`, `&self`, `&mut self`, or `self` with lifetimes: - -```mist -pub void set_value(&mut self, i32 v) -{ - self.value = v; -} -``` - ---- - -### Control Flow - -#### If/Else - -```mist -if condition { - // body -} else if other_condition { - // body -} else { - // body -} -``` - -Conditions must be parenthesized: `if (expr) { }` or `if expr_no_struct { }` (struct literals require parentheses). - -Used as an expression: - -```mist -let x = if true { 1 } else { 2 }; -``` - -#### While - -```mist -while condition { - // body -} -``` - -#### For - -```mist -for pattern in iterator { - // body -} - -for i in 0 .. 10 { - // body -} -``` - -#### C-Style For - -```mist -for (let mut i = 0; i < 10; i++) { - // body -} -``` - -#### Loop - -```mist -loop { - // infinite loop -} -``` - -#### Match - -```mist -match value { - pattern1 => expr, - pattern2 => { - // block body - } - pattern3 | pattern4 => expr, -} -``` - -#### Break/Continue - -```mist -break; -continue; -``` - -#### Return - -```mist -return; -return value; -``` - ---- - -### Types - -#### Type System - -```mist -i32 // Path type -str& // Reference type -&mut i32 // Mutable reference -&'a i32 // Reference with lifetime -*const i32 // Const pointer -*mut i32 // Mutable pointer -(i32, bool) // Tuple type -fn(i32) -> bool // Function pointer type -Fn(i32) -> bool // Closure trait (Fn) -FnMut(i32) -> bool // Closure trait (FnMut) -FnOnce(i32) -> bool // Closure trait (FnOnce) -dyn Trait // Trait object -void // Unit type (maps to Rust's ()) -'i32 // Type ascription not to be confused with lifetime -``` - -Type expressions can be composed: - -```mist -type_expr = { (void | path_type | tuple_type | dyn_type) ~ (unsafe_ref_type | ref_type | fn_type)* } -``` - -This means types are written left-to-right naturally: - -```mist -i32& // &i32 -i32&mut // &mut i32 -i32&'a // &'a i32 -i32 fn() -> bool // fn(i32) -> bool -``` - -#### Type Aliases - -```mist -type MyInt = i32; - -type Result = std::result::Result; -``` - -#### Const and Static - -```mist -const MAX: i32 = 100; -static NAME: str& = "hello"; -``` - ---- - -### Structs - -```mist -pub struct Point { - i32 x, - i32 y, -} - -pub struct Generic { - T value, -} -``` - -Fields can be public or private: - -```mist -struct User { - pub str& name, - i32 age, // private -} -``` - ---- - -### Enums - -```mist -pub enum Option { - Some(T), - None, -} - -pub enum Message { - Quit, - Move(i32, i32), - Write { str& content, i32 length }, -} -``` - -Enum variants can be: -- **Named** — `Variant` -- **Tuple** — `Variant(T1, T2)` -- **Struct** — `Variant { T1 field1, T2 field2 }` - ---- - -### Traits - -```mist -pub trait Drawable { - void draw(&self); -} - -pub trait Comparable : Eq { - i32 cmp(&self, T other); -} -``` - -Trait requirements are specified after `:`: - -```mist -trait MyTrait : SuperTrait + OtherTrait { - void required_method(&self); - void another(&self); -} -``` - ---- - -### Impl Blocks - -```mist -impl MyType { - void method(&self) { } -} - -impl Trait for MyType { - void method(&self) { } -} - -impl GenericTrait for MyType { - void method(&self, T value) { } -} -``` - ---- - -### Classes - -Mist introduces `class` as syntactic sugar for a Rust struct with a virtual method table (vtable). - -```mist -pub class Animal { - str& name, - - pub constructor(str& name) - { - self.name = name; - } - - pub void speak(&self) - { - println!("..."); - } -} -``` - -Class fields can have default initializers: - -```mist -class Player { - i32 health = 100, - str& name, -} -``` - -#### Inheritance - -```mist -class Dog : Animal { - pub constructor(str& name) - { - super(name); - } - - override void speak(&self) - { - println!("Woof!"); - } -} -``` - -The `override` keyword supports explicit base class targeting: - -```mist -override(Animal) void speak(&self) -{ - println!("Woof!"); -} -``` - -#### Under the Hood - -A class `Dog : Animal` generates: - -1. A Rust struct with a `_super: Animal` field (or `_vptr: &'static [*const c_void]` for root classes) -2. A vtable constant with function pointers for each public method -3. An `impl` block with `Deref` and `DerefMut` -4. Method trampolines (`__m_`) that are dispatched through the vtable -5. A `new()` constructor that initializes via `MaybeUninit` and calls the user's `constructor(&mut self)` - -The vtable is unified: parent entries are copied, overridden entries replace parent slots, and new methods are appended. - ---- - -### Generics - -```mist -// Generic function -T id(T x) -{ - x -} - -// Generic struct -struct Pair { - A first, - B second, -} - -// Generic enum -enum Result { - Ok(T), - Err(E), -} - -// Generic with trait bounds -T max(T a, T b) -{ - if a > b { a } else { b } -} - -// Lifetime generics -void process<'a>(&'a str& data) -{ - // ... -} -``` - -Generic syntax uses `<` `>` delimiters. Lifetimes are prefixed with `'`. - ---- - -### Visibility - -```mist -// Private (default) -void internal() { } - -// Public -pub void external() { } - -// Public to specific path -pub(crate) void crate_only() { } -pub(super) void parent_only() { } -pub(in my::module) void module_only() { } -``` - ---- - -### Modules - -```mist -// Declare a submodule -pub module foo; - -// Import -use std::collections::HashMap; - -// Re-exporting import -pub use my_module::MyType; -``` - -#### Module Resolution - -Mist maps the module tree to Rust's module system: - -| Mist Path | Rust Output | -|------------------------------|------------------------------| -| `src/main.mist` | `.mist/src/main.rs` | -| `src/utils/package.mist` | `.mist/src/utils/mod.rs` | -| `src/foo.mist` | `.mist/src/foo.rs` | -| `src/utils/helper.mist` | `.mist/src/utils/helper.rs` | - -A `package.mist` file acts as a directory's module root, analogous to `mod.rs`. - ---- - -### Attributes - -Inner attributes apply to the containing module: - -```mist -#![allow(unused_variables)] -``` - -Outer attributes apply to the next item: - -```mist -#[derive(Debug, Clone)] -struct Point { - i32 x, - i32 y, -} - -#[test] -void my_test() -{ - assert_eq!(1, 1); -} -``` - -Attribute syntax: -- `#[path]` -- `#[path = literal]` -- `#[path(item1, item2, ...)]` - ---- - -### Patterns - -```mist -// Literal patterns -match x { - 1 => "one", - 2 => "two", - _ => "other", -} - -// Tuple patterns -let (a, b) = (1, 2); - -// Struct patterns -match value { - Point { x, y } => x + y, - Point { x: 0, y } => y, - _ => 0, -} - -// Named tuple patterns (newtype) -let MyType(value) = my_var; - -// Wildcard / etc -let _ = get_side_effect(); -match x { - 1 => ..., - .. => ..., // rest / etc -} - -// Mutable binding in pattern -let mut x = 42; -match ref_to_option { - Some(mut value) => value += 1, - None => {}, -} -``` - ---- - -### Operators - -#### Binary Operators - -| Operator | Description | -|----------|----------------------| -| `+` | Addition | -| `-` | Subtraction | -| `*` | Multiplication | -| `/` | Division | -| `%` | Modulus | -| `==` | Equality | -| `!=` | Inequality | -| `<` | Less than | -| `>` | Greater than | -| `<=` | Less or equal | -| `>=` | Greater or equal | -| `&&` | Logical AND | -| `\|\|` | Logical OR | -| `&` | Bitwise AND | -| `\|` | Bitwise OR | -| `^` | Bitwise XOR | -| `<<` | Left shift | -| `>>` | Right shift | -| `=` | Assignment | -| `+=` | Add assign | -| `-=` | Subtract assign | -| `*=` | Multiply assign | -| `/=` | Divide assign | -| `%=` | Modulus assign | -| `&=` | Bitwise AND assign | -| `\|=` | Bitwise OR assign | -| `^=` | Bitwise XOR assign | -| `<<=` | Left shift assign | -| `>>=` | Right shift assign | -| `..` | Range (exclusive) | -| `..=` | Range (inclusive) | -| `->` | Pointer write | - -#### Prefix Operators - -| Operator | Description | -|----------|-------------------| -| `*` | Dereference | -| `&` | Reference | -| `&mut` | Mutable reference | -| `!` | Logical NOT | -| `-` | Numeric negation | - -#### Postfix Operators - -| Operator | Description | -|------------|----------------------| -| `.field` | Field access | -| `.0` | Tuple field access | -| `()` | Function call | -| `[]` | Index | -| `{ f: v }` | Struct literal | -| `as Type` | Type cast | -| `?` | Try (error prop) | -| `++` | Increment | -| `--` | Decrement | -| `!()` | Macro call (paren) | -| `![]` | Macro call (bracket) | -| `!{}` | Macro call (brace) | - ---- - -### Macros - -Mist reuses Rust's macro system directly: - -```mist -println!("hello"); -assert_eq!(a, b); -vec![1, 2, 3]; -``` - -Macro calls use `!` followed by parentheses, brackets, or braces. - ---- - -### Closures - -```mist -let add = (a, b) => a + b; -let result = add(2, 3); // 5 - -let square = f64 x => x * x; -``` - -Closures can have explicit return types: - -```mist -let transform = i32 x => { - x * 2 -}; -``` - ---- - -### Attributes - -Mist supports Rust-style attributes at module and item level: - -```mist -#![crate_type = "lib"] - -#[derive(Clone)] -#[repr(C)] -pub struct External { } -``` - ---- - -## Compiler Architecture - -### Pipeline Overview - -The Mist compiler operates in distinct phases: - -``` -Source (.mist) - │ - ▼ -┌─────────────┐ -│ Lexer │ (pest PEG grammar) -│ & Parser │ -└─────────────┘ - │ AST - ▼ -┌─────────────┐ -│ Semantics │ (field init checking) -└─────────────┘ - │ - ▼ -┌─────────────┐ -│ Codegen │ (AST → Rust source) -└─────────────┘ - │ .rs + .map.json - ▼ -┌─────────────┐ -│ cargo │ (Rust compilation) -└─────────────┘ - │ - ▼ - Binary / Library -``` - -### 1. Parsing (`mist-parser`) - -The parser is built with [pest](https://pest.rs), a PEG parser generator. Grammar rules are defined in `grammar.pest` (693 rules). - -**Key files**: -- `grammar.pest` — PEG grammar defining the entire language syntax -- `src/parser/common/` — Parse rule → AST conversions for expressions, statements, types, declarations -- `src/parser/items/` — Parse rule → AST conversions for top-level items (structs, enums, classes, functions, traits, impls, attributes) -- `src/ast/` — AST node types (expr.rs, statement.rs, top_level.rs) -- `src/semantics.rs` — Semantic checks (e.g., ensuring all class fields are initialized in constructors) -- `src/error.rs` — Error types (PreAst parsing errors, Ast generation errors) -- `src/rev_mapper.rs` — Position mapping between Mist and Rust source - -**Parsing entry points**: - -```rust -// Parse a complete program -pub fn parse<'a>(source: &'a str) -> Result> - -// Parse only the module declaration -pub fn parse_module<'a>(source: &'a str) -> Result, ParseError<'a>> -``` - -#### Grammar Structure - -The grammar follows a layered approach: - -1. **Lexical rules** (silent) — `WHITESPACE`, `COMMENT`, `identifier`, `integer`, `float`, `string_lit` -2. **Primary expressions** — literals, paths, tuples, arrays, closures, control flow -3. **Term** — prefix operators + primary + postfix operators -4. **Expression** — terms joined by binary operators (via Pratt parser) -5. **Statements** — variable declarations, control flow (`if`, `while`, `for`, `match`, `loop`), blocks -6. **Top-level items** — functions, structs, enums, classes, traits, impls, type aliases, imports, module declarations, constants - -#### AST Structure - -The AST preserves source positions via `Spanned`: - -```rust -pub struct Spanned { - pub line: usize, - pub column: usize, - pub item: T, -} -``` - -Top-level items are wrapped as: - -```rust -pub struct TopLevel(pub Spanned, pub Vec); -``` - -Expressions use a fix-point representation for prefix/postfix operators: - -```rust -Expression::Fix { - initial: Box, - prefixes: Vec, - postfixes: Vec, -} -``` - -Binary expressions use Pratt parsing for correct precedence: - -```rust -Expression::Binary { - lhs: Box, - op: String, - rhs: Box, -} -``` - -All operators are left-associative with a single precedence level. - -### 2. Semantic Analysis (`mist-parser`) - -The semantic checker (`semantics.rs`) performs class field initialization analysis. When a class has a constructor, it verifies that every declared field is mutated (directly or indirectly via method calls) within the constructor body. - -**Key check**: `check_class_semantics()` — Collects all field names, then walks the constructor body tracking which identifiers are assigned. If any field is uninitialized, an error is reported. - -The mutability analysis (`GetMutability` trait) works by: -1. Collecting all identifiers that get `&mut` references -2. Tracking `self.field = value` patterns -3. Following method calls that might initialize fields -4. Ensuring all branches initialize the same fields (intersection semantics) - -### 3. Code Generation (`mist-codegen`) - -The code generator converts the Mist AST into Rust source code. It does not generate an intermediate representation — it produces Rust source text directly. - -**Key files**: -- `src/lib.rs` — `RustCodegen` struct with output buffer, indentation tracking, `Mapping` for position translation -- `src/top_level.rs` — Generates Rust for top-level items (structs, enums, traits, functions, impls, imports, type aliases, const/static) -- `src/statement.rs` — Generates Rust for statements and blocks -- `src/expr.rs` — Generates Rust for expressions (literals, paths, binary ops, closures, arrays, prefix/postfix) -- `src/class_decl.rs` — Class-specific code generation (struct + vtable + impl blocks + Deref) - -#### Codegen Design - -The `RustCodegen` maintains: -- A string `output` buffer -- An `indent` level (4 spaces per level) -- A `Mapping` that records `(RustMap, MistMap)` pairs for source position remapping -- A current `position` tracker (`RustMap(line, column)`) - -Generation follows the `GenRust` trait: - -```rust -pub trait GenRust { - fn gen_rust(&self, ctx: &mut Context, cg: &mut RustCodegen); -} -``` - -And `GetRust` for simple string-returning types: - -```rust -pub trait GetRust { - fn get_rust(&self) -> String; -} -``` - -The `Context` carries optional expression path information for class `super` / `Super` resolution. - -#### Class Codegen (detailed) - -Classes are the most complex codegen path. `ClassProcessedData` analyzes a class declaration and then emits: - -1. **Struct declaration** — `struct ClassName { pub _super: Parent, pub field1: T1, ... }` (or `_vptr` for root classes) - -2. **Vtable constants** — Index constants `__FN_METHOD` and a `__V_TABLE` static array of function pointers. For inherited classes, parent vtable entries are copied and overridden entries replaced. - -3. **Constructor** — `pub fn new(...) -> Self` that: - - Creates an uninitialized instance via `MaybeUninit::zeroed().assume_init()` - - Sets the vtable pointer - - Writes field defaults - - Calls `self.constructor(...)` - - Returns `this` - -4. **Method trampolines** — Public methods with `self` get wrapper functions `__m_method` that are stored in the vtable, plus virtual dispatch methods that look up the function pointer at runtime. - -5. **Override support** — Methods marked `override` are validated at compile time by generating test code that dereferences `&Self` to `&Target`, confirming the Deref chain works. - -6. **Deref impls** — `impl Deref for Child` and `DerefMut` for inherited classes. - -7. **Impl declarations** — Inner `impl` blocks are rewritten to use the self type. - -#### Mapping System - -The `rev_mapper` module provides bidirectional position mapping: - -```rust -pub struct Mapping { - pub mist_path: PathBuf, - pub map: HashSet<(RustMap, MistMap)>, -} -``` - -- `RustMap(usize, usize)` — line/column in generated Rust -- `MistMap(usize, usize)` — line/column in original Mist - -The mapping is populated during codegen via `GenSpanTranslation`: - -```rust -impl GenSpanTranslation for Spanned { - fn gen_span(&self, cg: &mut RustCodegen) { - cg.mapping.map.insert((cg.position, MistMap(self.line, self.column))); - } -} -``` - -This allows the builder to remap Rust compiler errors back to the original Mist source positions. - -### 4. Builder & Error Remapping (`mist-api`) - -The builder module (`builder.mist` in the `mist_api` crate) wraps `cargo` to: - -1. Spawn `cargo` with `--message-format=json` -2. Parse JSON compiler messages -3. For each diagnostic span, look up the corresponding `.map.json` file -4. Remap Rust line/column to Mist line/column using the mapping -5. Display errors/warnings with Mist source locations and context lines - -```rust -pub fn build(args: Vec, root: PathBuf) -> bool { - // Spawn cargo with JSON output - // Parse CompilerMessage for each span - // Look up rev_mapper::Mapping from .map.json - // Remap to Mist positions - // Print diagnostics -} -``` - -### 5. Transpilation Pipeline - -The transpiler (`transpiler.mist` in `mist_api`) orchestrates the full pipeline: - -1. Read `Mist.toml` configuration -2. Build a `Module` tree from the filesystem (discovering `package.mist` files) -3. For each module: - a. Parse Mist source - b. Run semantic checks - c. Generate Rust code and position mapping - d. Write `.rs` file and `.map.json` file -4. Recursively transpile included crates - -The `Module` struct represents the file-to-module mapping: - -```rust -pub class Module { - pub String name; - pub PathBuf path; - pub Vec children; - - pub constructor(PathBuf mist_path, MistConfig& config) { ... } - pub bool is_package(&self) { ... } - pub PathBuf output_dir(&self, PathBuf& parent_dir) { ... } - pub PathBuf output_path(&self, PathBuf& parent_dir, ...) { ... } -} -``` - -#### Transpilation Mapping Rules - -| Mist Source | Rust Output | -|-----------------|--------------------------------| -| `src/main.mist` | `.mist/src/main.rs` | -| `src/foo.mist` | `.mist/src/foo.rs` | -| `src/bar/package.mist` | `.mist/src/bar/mod.rs` | -| `src/bar/baz.mist` | `.mist/src/bar/baz.rs` | - -A `package.mist` file: -- Generates `mod.rs` in the output directory -- Contains `pub mod ;` declarations for its children -- Its own items are prepended to the output - -### 6. Caching - -The transpiler implements basic caching via file modification times: - -```rust -fn is_source_newer(source: &Path, output: &Path) -> io::Result { - if !output.exists() { return Ok(true); } - let source_time = fs::metadata(source)?.modified()?; - let output_time = fs::metadata(output)?.modified()?; - Ok(source_time > output_time) -} -``` - -If a source file has not changed since the last transpilation, the `.rs` file is not regenerated. - ---- - -## LSP Support (`mist-analyzer`) - -The Mist Language Server Protocol implementation runs a headless `rust-analyzer` instance and bridges Mist editor requests to Rust positions. - -### Architecture - -``` -Editor (LSP client) - │ - ▼ -┌─────────────────────┐ -│ mist-analyzer │ -│ (Rust + Mist) │ -└─────────────────────┘ - │ ▲ - │ JSON-RPC │ - ▼ │ -┌─────────────────────┐ -│ rust-analyzer │ -│ (headless child) │ -└─────────────────────┘ -``` - -### Flow - -1. Editor sends Mist file edits to `mist-analyzer` via LSP -2. `mist-analyzer` transpiles Mist to Rust -3. The transpiled Rust is forwarded to the headless `rust-analyzer` process -4. For goto-definition, hover, and completion, a unique marker token is injected at the cursor position -5. The transpiled output with the marker is sent to `rust-analyzer` -6. The marker position is located in the response -7. Results are mapped back to Mist positions using the `rev_mapper` - -### Key Features - -- **Full document sync** — Open, change, save, close -- **Go to definition** — Maps through transpiled Rust positions -- **Completions** — Supports trigger characters `:`, `.`, `'`, `(` -- **Hover** — Type information and docs -- **Diagnostics** — Real-time errors from transpilation failures and Rust compilation -- **Formatting** — Document formatting support (currently passthrough) -- **Auto-import** — Automatically inserts `pub module ;` when new `.mist` files are created -- **File watching** — Monitors `**/*.mist` for new files - -### Diagnostic Remapping - -When a transpilation error occurs in the `mist-analyzer`: - -1. Parse errors are converted to `Diagnostic` with Mist source positions -2. Semantic errors (uninitialized class fields) use the stored line/column -3. Rust compiler errors are remapped via the `Mapping` system back to Mist positions - ---- - -## Semantic Checks - -### Field Initialization - -The primary semantic check ensures all class fields are initialized in the constructor: - -```mist -class Player { - i32 health, - str& name, - - pub constructor(str& name) - { - self.name = name; - // Error: field 'health' is uninitialized - } -} -``` - -The analysis tracks: -- Direct field assignment: `self.field = value` -- `&mut self.field` patterns -- Method calls that might initialize fields (transitively) -- All conditional branches must initialize the same fields -- `super` assignments count for `_super` field initialization - ---- - -## Position Mapping - -Mist maintains a bidirectional mapping between Mist source positions and Rust output positions. This is essential for: - -1. **Error remapping** — Rust compiler errors point to the correct Mist source location -2. **LSP features** — Go-to-definition, hover, and completion work from Mist source - -The mapping is stored as `HashSet<(RustMap, MistMap)>` pairs and serialized to `.map.json` files alongside the transpiled `.rs` output. - -### Mapping Lifecycle - -1. **Codegen** — Each `Spanned` AST node records its Mist position and the current Rust position in the codegen output buffer -2. **Persistence** — The mapping is written to `.map.json` -3. **Build-time remapping** — The builder reads `.map.json` to remap `cargo` diagnostics back to Mist positions -4. **LSP remapping** — Both Mist→Rust and Rust→Mist direction queries are supported via `find()` and `find_by_mist()`