Docs rewrite
This commit is contained in:
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Builder & Error Remapping
|
||||
description: How Mist invokes cargo and remaps Rust compiler errors back to Mist source locations.
|
||||
icon: Bug
|
||||
---
|
||||
|
||||
The builder module (`builder.mist` in the `mist_api` crate) wraps `cargo` to:
|
||||
|
||||
1. Spawn `cargo` with `--message-format=json`
|
||||
2. Parse JSON compiler messages
|
||||
3. For each diagnostic span, look up the corresponding `.map.json` file
|
||||
4. Remap Rust line/column to Mist line/column using the mapping
|
||||
5. Display errors/warnings with Mist source locations and context lines
|
||||
|
||||
```rust
|
||||
pub fn build(args: Vec<String>, root: PathBuf) -> bool {
|
||||
// Spawn cargo with JSON output
|
||||
// Parse CompilerMessage for each span
|
||||
// Look up rev_mapper::Mapping from .map.json
|
||||
// Remap to Mist positions
|
||||
// Print diagnostics
|
||||
}
|
||||
```
|
||||
|
||||
### Diagnostic Output
|
||||
|
||||
Errors and warnings are displayed with Mist source context:
|
||||
|
||||
```
|
||||
src/main.mist:10:5
|
||||
Error: mismatched types
|
||||
let x: i32 = "hello";
|
||||
^^^^^^^
|
||||
```
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
title: Code Generation
|
||||
description: How the Mist AST is translated into Rust source code, including class vtables and position mapping.
|
||||
icon: Wrench
|
||||
---
|
||||
|
||||
The code generator converts the Mist AST into Rust source code directly — no intermediate representation.
|
||||
|
||||
### Key files
|
||||
|
||||
- `src/lib.rs` — `RustCodegen` struct with output buffer, indentation tracking, `Mapping` for position translation
|
||||
- `src/top_level.rs` — Generates Rust for top-level items (structs, enums, traits, functions, impls, imports, type aliases, const/static)
|
||||
- `src/statement.rs` — Generates Rust for statements and blocks
|
||||
- `src/expr.rs` — Generates Rust for expressions (literals, paths, binary ops, closures, arrays, prefix/postfix)
|
||||
- `src/class_decl.rs` — Class-specific code generation (struct + vtable + impl blocks + Deref)
|
||||
|
||||
### Codegen Design
|
||||
|
||||
The `RustCodegen` maintains:
|
||||
- A string `output` buffer
|
||||
- An `indent` level (4 spaces per level)
|
||||
- A `Mapping` that records `(RustMap, MistMap)` pairs for source position remapping
|
||||
- A current `position` tracker (`RustMap(line, column)`)
|
||||
|
||||
Generation follows the `GenRust` trait:
|
||||
|
||||
```rust
|
||||
pub trait GenRust {
|
||||
fn gen_rust(&self, ctx: &mut Context, cg: &mut RustCodegen);
|
||||
}
|
||||
```
|
||||
|
||||
And `GetRust` for simple string-returning types:
|
||||
|
||||
```rust
|
||||
pub trait GetRust {
|
||||
fn get_rust(&self) -> String;
|
||||
}
|
||||
```
|
||||
|
||||
The `Context` carries optional expression path information for class `super` / `Super` resolution.
|
||||
|
||||
### Class Codegen
|
||||
|
||||
Classes are the most complex codegen path. `ClassProcessedData` analyzes a class declaration and emits:
|
||||
|
||||
1. **Struct declaration** — `struct ClassName<G> { pub _super: Parent, pub field1: T1, ... }` (or `_vptr` for root classes)
|
||||
2. **Vtable constants** — Index constants `__FN_METHOD` and a `__V_TABLE` static array of function pointers. For inherited classes, parent vtable entries are copied and overridden entries replaced.
|
||||
3. **Constructor** — `pub fn new(...) -> Self` that creates an uninitialized instance via `MaybeUninit`, sets the vtable pointer, writes field defaults, calls `self.constructor(...)`, and returns `this`
|
||||
4. **Method trampolines** — Public methods with `self` get wrapper functions `__m_method` stored in the vtable, plus virtual dispatch methods
|
||||
5. **Override support** — Methods marked `override` are validated at compile time via generated test code
|
||||
6. **Deref impls** — `impl Deref<Target = Parent> for Child` and `DerefMut` for inherited classes
|
||||
7. **Impl declarations** — Inner `impl` blocks are rewritten to use the self type
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
title: LSP Support
|
||||
description: How the Mist Language Server bridges editor requests through rust-analyzer.
|
||||
icon: Server
|
||||
---
|
||||
|
||||
The Mist Language Server Protocol implementation runs a headless `rust-analyzer` instance and bridges Mist editor requests to Rust positions.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Editor (LSP client)
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ mist-analyzer │
|
||||
│ (Rust + Mist) │
|
||||
└─────────────────────┘
|
||||
│ ▲
|
||||
│ JSON-RPC │
|
||||
▼ │
|
||||
┌─────────────────────┐
|
||||
│ rust-analyzer │
|
||||
│ (headless child) │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
### Flow
|
||||
|
||||
1. Editor sends Mist file edits to `mist-analyzer` via LSP
|
||||
2. `mist-analyzer` transpiles Mist to Rust
|
||||
3. The transpiled Rust is forwarded to the headless `rust-analyzer` process
|
||||
4. For goto-definition, hover, and completion, a unique marker token is injected at the cursor position
|
||||
5. The transpiled output with the marker is sent to `rust-analyzer`
|
||||
6. The marker position is located in the response
|
||||
7. Results are mapped back to Mist positions using the `rev_mapper`
|
||||
|
||||
### Key Features
|
||||
|
||||
- **Full document sync** — Open, change, save, close
|
||||
- **Go to definition** — Maps through transpiled Rust positions
|
||||
- **Completions** — Supports trigger characters `:`, `.`, `'`, `(`
|
||||
- **Hover** — Type information and docs
|
||||
- **Diagnostics** — Real-time errors from transpilation failures and Rust compilation
|
||||
- **Formatting** — Document formatting support
|
||||
- **Auto-import** — Automatically inserts `pub module <name>;` when new `.mist` files are created
|
||||
- **File watching** — Monitors `**/*.mist` for new files
|
||||
|
||||
### Diagnostic Remapping
|
||||
|
||||
When a transpilation error occurs in the `mist-analyzer`:
|
||||
|
||||
1. Parse errors are converted to `Diagnostic` with Mist source positions
|
||||
2. Semantic errors (uninitialized class fields) use the stored line/column
|
||||
3. Rust compiler errors are remapped via the `Mapping` system back to Mist positions
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
title: Parsing
|
||||
description: PEG grammar structure, AST design, and how Mist source code is parsed.
|
||||
icon: FileCode
|
||||
---
|
||||
|
||||
The parser is built with [pest](https://pest.rs), a PEG parser generator. Grammar rules are defined in `grammar.pest` (693 rules).
|
||||
|
||||
### Key files
|
||||
|
||||
- `grammar.pest` — PEG grammar defining the entire language syntax
|
||||
- `src/parser/common/` — Parse rule → AST conversions for expressions, statements, types, declarations
|
||||
- `src/parser/items/` — Parse rule → AST conversions for top-level items (structs, enums, classes, functions, traits, impls, attributes)
|
||||
- `src/ast/` — AST node types (expr.rs, statement.rs, top_level.rs)
|
||||
- `src/error.rs` — Error types (PreAst parsing errors, Ast generation errors)
|
||||
|
||||
### Parsing entry points
|
||||
|
||||
```rust
|
||||
// Parse a complete program
|
||||
pub fn parse<'a>(source: &'a str) -> Result<Program, ParseError<'a>>
|
||||
|
||||
// Parse only the module declaration
|
||||
pub fn parse_module<'a>(source: &'a str) -> Result<Option<(Visibility, Identifier)>, ParseError<'a>>
|
||||
```
|
||||
|
||||
### Grammar Structure
|
||||
|
||||
The grammar follows a layered approach:
|
||||
|
||||
1. **Lexical rules** (silent) — `WHITESPACE`, `COMMENT`, `identifier`, `integer`, `float`, `string_lit`
|
||||
2. **Primary expressions** — literals, paths, tuples, arrays, closures, control flow
|
||||
3. **Term** — prefix operators + primary + postfix operators
|
||||
4. **Expression** — terms joined by binary operators (via Pratt parser)
|
||||
5. **Statements** — variable declarations, control flow (`if`, `while`, `for`, `match`, `loop`), blocks
|
||||
6. **Top-level items** — functions, structs, enums, classes, traits, impls, type aliases, imports, module declarations, constants
|
||||
|
||||
### AST Structure
|
||||
|
||||
The AST preserves source positions via `Spanned<T>`:
|
||||
|
||||
```rust
|
||||
pub struct Spanned<T> {
|
||||
pub line: usize,
|
||||
pub column: usize,
|
||||
pub item: T,
|
||||
}
|
||||
```
|
||||
|
||||
Top-level items are wrapped as:
|
||||
|
||||
```rust
|
||||
pub struct TopLevel(pub Spanned<TopLevelKind>, pub Vec<Attribute>);
|
||||
```
|
||||
|
||||
Expressions use a fix-point representation for prefix/postfix operators:
|
||||
|
||||
```rust
|
||||
Expression::Fix {
|
||||
initial: Box<Expression>,
|
||||
prefixes: Vec<Prefix>,
|
||||
postfixes: Vec<Postfix>,
|
||||
}
|
||||
```
|
||||
|
||||
Binary expressions use Pratt parsing for correct precedence:
|
||||
|
||||
```rust
|
||||
Expression::Binary {
|
||||
lhs: Box<Expression>,
|
||||
op: String,
|
||||
rhs: Box<Expression>,
|
||||
}
|
||||
```
|
||||
|
||||
All operators are left-associative with a single precedence level.
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
title: Position Mapping
|
||||
description: Bidirectional mapping between Mist source positions and Rust output positions for error remapping and LSP features.
|
||||
icon: Map
|
||||
---
|
||||
|
||||
Mist maintains a bidirectional mapping between Mist source positions and Rust output positions. This is essential for:
|
||||
|
||||
1. **Error remapping** — Rust compiler errors point to the correct Mist source location
|
||||
2. **LSP features** — Go-to-definition, hover, and completion work from Mist source
|
||||
|
||||
### Data Structure
|
||||
|
||||
The mapping is stored as pairs:
|
||||
|
||||
```rust
|
||||
pub struct Mapping {
|
||||
pub mist_path: PathBuf,
|
||||
pub map: HashSet<(RustMap, MistMap)>,
|
||||
}
|
||||
```
|
||||
|
||||
- `RustMap(usize, usize)` — line/column in generated Rust
|
||||
- `MistMap(usize, usize)` — line/column in original Mist
|
||||
|
||||
### Mapping Lifecycle
|
||||
|
||||
1. **Codegen** — Each `Spanned<T>` AST node records its Mist position and the current Rust position in the codegen output buffer via `GenSpanTranslation`
|
||||
2. **Persistence** — The mapping is serialized to `<output>.map.json` alongside the transpiled `.rs` output
|
||||
3. **Build-time remapping** — The builder reads `.map.json` to remap `cargo` diagnostics back to Mist positions
|
||||
4. **LSP remapping** — Both Mist→Rust and Rust→Mist direction queries are supported via `find()` and `find_by_mist()`
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: Semantic Analysis
|
||||
description: Class field initialization verification and the GetMutability analysis system.
|
||||
icon: SearchCheck
|
||||
---
|
||||
|
||||
The semantic checker (`semantics.rs`) performs class field initialization analysis. When a class has a constructor, it verifies that every declared field is mutated (directly or indirectly via method calls) within the constructor body.
|
||||
|
||||
### Field Initialization
|
||||
|
||||
The primary semantic check ensures all class fields are initialized in the constructor:
|
||||
|
||||
```mist
|
||||
class Player {
|
||||
i32 health,
|
||||
str& name,
|
||||
|
||||
pub constructor(str& name)
|
||||
{
|
||||
self.name = name;
|
||||
// Error: field 'health' is uninitialized
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
The `GetMutability` trait tracks which identifiers are mutated:
|
||||
|
||||
1. Collects all identifiers that get `&mut` references
|
||||
2. Tracks `self.field = value` patterns
|
||||
3. Follows method calls that might initialize fields (transitively)
|
||||
4. Ensures all conditional branches initialize the same fields (intersection semantics)
|
||||
5. `super` assignments count for `_super` field initialization
|
||||
|
||||
If any field is uninitialized after the analysis, a `SemanticError` is reported with the field's source position.
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
title: Transpilation Pipeline
|
||||
description: How Mist source files are discovered, transpiled to Rust, and cached.
|
||||
icon: Repeat
|
||||
---
|
||||
|
||||
The transpiler (`transpiler.mist` in `mist_api`) orchestrates the full pipeline.
|
||||
|
||||
### Pipeline
|
||||
|
||||
1. Read `Mist.toml` configuration
|
||||
2. Build a `Module` tree from the filesystem (discovering `package.mist` files)
|
||||
3. For each module:
|
||||
a. Parse Mist source
|
||||
b. Run semantic checks
|
||||
c. Generate Rust code and position mapping
|
||||
d. Write `.rs` file and `.map.json` file
|
||||
4. Recursively transpile included crates
|
||||
|
||||
### Module Tree
|
||||
|
||||
The `Module` struct represents the file-to-module mapping:
|
||||
|
||||
```rust
|
||||
pub class Module {
|
||||
pub String name;
|
||||
pub PathBuf path;
|
||||
pub Vec<Module> children;
|
||||
|
||||
pub constructor(PathBuf mist_path, MistConfig& config) { ... }
|
||||
pub bool is_package(&self) { ... }
|
||||
pub PathBuf output_dir(&self, PathBuf& parent_dir) { ... }
|
||||
pub PathBuf output_path(&self, PathBuf& parent_dir, ...) { ... }
|
||||
}
|
||||
```
|
||||
|
||||
### Transpilation Mapping Rules
|
||||
|
||||
| Mist Source | Rust Output |
|
||||
|----------------------------|----------------------------------|
|
||||
| `src/main.mist` | `.mist/src/main.rs` |
|
||||
| `src/foo.mist` | `.mist/src/foo.rs` |
|
||||
| `src/bar/package.mist` | `.mist/src/bar/mod.rs` |
|
||||
| `src/bar/baz.mist` | `.mist/src/bar/baz.rs` |
|
||||
|
||||
A `package.mist` file generates `mod.rs` in the output directory and contains `pub mod <child>;` declarations for its children.
|
||||
|
||||
### Caching
|
||||
|
||||
The transpiler implements basic caching via file modification times:
|
||||
|
||||
```rust
|
||||
fn is_source_newer(source: &Path, output: &Path) -> io::Result<bool> {
|
||||
if !output.exists() { return Ok(true); }
|
||||
let source_time = fs::metadata(source)?.modified()?;
|
||||
let output_time = fs::metadata(output)?.modified()?;
|
||||
Ok(source_time > output_time)
|
||||
}
|
||||
```
|
||||
|
||||
If a source file has not changed since the last transpilation, the `.rs` file is not regenerated.
|
||||
Reference in New Issue
Block a user