diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 3c2eee6a12..ac4341188a 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -41,7 +41,8 @@ code accordingly. We also use [clippy](https://github.com/rust-lang/rust-clippy) to detect rust code issues. Python code should follow the -[PEP 8](https://www.python.org/dev/peps/pep-0008/) style. +[PEP 8](https://www.python.org/dev/peps/pep-0008/) style. We also use +[flake8](http://flake8.pycqa.org/en/latest/) to check Python code style. ## Testing @@ -107,22 +108,46 @@ repository's structure: ## Understanding Internals -Rust crates - rustpython: top level crate - rustpython_parser - rustpython-vm (source is found in `vm/src`) +The RustPython package is the `rustpython` top-level crate. The `Cargo.toml` +file in the root of the repo provide configuration of the crate and the +implementation is found in the `src` directory (specifically, +`src/main.rs`). -Lexer, Parser, AST (Abstract Syntax Tree) +The top-level `rustpython` depends on several lower-level crates including: + +- `rustpython-parser` (source: `parser/src`) +- `rustpython-compiler` (source:`compiler/src`) +- `rustpython-vm` (source: `vm/src`) + +Together, these crates provide the functions of a programming language and +enable a line of code to go through a series of steps: + +- parse the line of source code into tokens +- determine if the tokens are valid syntax +- create an Abstract Syntax Tree (AST) +- compile the AST into bytecode +- execute the bytecode in the virtual machine (VM). + +### rustpython-parser + +This crate contains the lexer and parser to convert a line of code to +an Abstract Syntax Tree (AST): - Lexer: `parser/lexer.rs` converts Python source code into tokens - Parser: Takes the tokens generated by the lexer and parses the tokens into an AST (Abstract Syntax Tree) where the nodes of the syntax tree - are Rust structs and enums + are Rust structs and enums. - The Parser relies on `LALRPOP`, a Rust parser generator framework - More information on parsers and a tutorial can be found in the [LALRPOP book](https://lalrpop.github.io/lalrpop/README.html). +- AST: `parser/ast.rs` implements Python types and expressions + represented in by the AST nodes in Rust. + +### rustpython-compiler -Compiler Generates bytecode from the AST +(source is found in `compiler/src`) is primarily used to transform AST (Abstract Syntax Tree to bytecode) + +### rustpython-vm VM (Virtual Machine) Fetch and dispatch loop