forked from Rust-related/RustPython
Add subsections to Understanding Internals
This commit is contained in:
@@ -41,7 +41,8 @@ code accordingly. We also use [clippy](https://github.com/rust-lang/rust-clippy)
|
||||
to detect rust code issues.
|
||||
|
||||
Python code should follow the
|
||||
[PEP 8](https://www.python.org/dev/peps/pep-0008/) style.
|
||||
[PEP 8](https://www.python.org/dev/peps/pep-0008/) style. We also use
|
||||
[flake8](http://flake8.pycqa.org/en/latest/) to check Python code style.
|
||||
|
||||
## Testing
|
||||
|
||||
@@ -107,22 +108,46 @@ repository's structure:
|
||||
|
||||
## Understanding Internals
|
||||
|
||||
Rust crates
|
||||
rustpython: top level crate
|
||||
rustpython_parser
|
||||
rustpython-vm (source is found in `vm/src`)
|
||||
The RustPython package is the `rustpython` top-level crate. The `Cargo.toml`
|
||||
file in the root of the repo provide configuration of the crate and the
|
||||
implementation is found in the `src` directory (specifically,
|
||||
`src/main.rs`).
|
||||
|
||||
Lexer, Parser, AST (Abstract Syntax Tree)
|
||||
The top-level `rustpython` depends on several lower-level crates including:
|
||||
|
||||
- `rustpython-parser` (source: `parser/src`)
|
||||
- `rustpython-compiler` (source:`compiler/src`)
|
||||
- `rustpython-vm` (source: `vm/src`)
|
||||
|
||||
Together, these crates provide the functions of a programming language and
|
||||
enable a line of code to go through a series of steps:
|
||||
|
||||
- parse the line of source code into tokens
|
||||
- determine if the tokens are valid syntax
|
||||
- create an Abstract Syntax Tree (AST)
|
||||
- compile the AST into bytecode
|
||||
- execute the bytecode in the virtual machine (VM).
|
||||
|
||||
### rustpython-parser
|
||||
|
||||
This crate contains the lexer and parser to convert a line of code to
|
||||
an Abstract Syntax Tree (AST):
|
||||
- Lexer: `parser/lexer.rs` converts Python source code into tokens
|
||||
- Parser: Takes the tokens generated by the lexer and parses the tokens
|
||||
into an AST (Abstract Syntax Tree) where the nodes of the syntax tree
|
||||
are Rust structs and enums
|
||||
are Rust structs and enums.
|
||||
- The Parser relies on `LALRPOP`, a Rust parser generator framework
|
||||
- More information on parsers and a tutorial can be found in the
|
||||
[LALRPOP book](https://lalrpop.github.io/lalrpop/README.html).
|
||||
- AST: `parser/ast.rs` implements Python types and expressions
|
||||
represented in by the AST nodes in Rust.
|
||||
|
||||
### rustpython-compiler
|
||||
|
||||
Compiler
|
||||
Generates bytecode from the AST
|
||||
(source is found in `compiler/src`) is primarily used to transform AST (Abstract Syntax Tree to bytecode)
|
||||
|
||||
### rustpython-vm
|
||||
|
||||
VM (Virtual Machine)
|
||||
Fetch and dispatch loop
|
||||
|
||||
Reference in New Issue
Block a user