diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index d534515b2..c79bee470 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -19,13 +19,13 @@ The contents of the Development Guide include: RustPython requires the following: -- Rust latest stable version (e.g 1.38.0 at Oct 1st 2019) +- Rust latest stable version (e.g 1.51.0 as of Apr 2 2021) - To check Rust version: `rustc --version` - If you have `rustup` on your system, enter to update to the latest stable version: `rustup update stable` - If you do not have Rust installed, use [rustup](https://rustup.rs/) to do so. -- CPython version 3.7.4 or higher +- CPython version 3.8 or higher - CPython can be installed by your operating system's package manager, from the [Python website](https://www.python.org/downloads/), or using a third-party distribution, such as @@ -41,11 +41,12 @@ RustPython requires the following: The Rust code style used is the default [rustfmt](https://github.com/rust-lang/rustfmt) codestyle. Please format your -code accordingly. We also use [clippy](https://github.com/rust-lang/rust-clippy) -to detect rust code issues. +code accordingly, or run `cargo fmt` to autoformat it. We also use +[clippy](https://github.com/rust-lang/rust-clippy) to lint Rust code, which +you can check yourself with `cargo clippy`. -Python code should follow the -[PEP 8](https://www.python.org/dev/peps/pep-0008/) style. We also use +Custom Python code (i.e. code not copied from CPython's standard library) should +follow the [PEP 8](https://www.python.org/dev/peps/pep-0008/) style. We also use [flake8](http://flake8.pycqa.org/en/latest/) to check Python code style. ## Testing @@ -95,23 +96,17 @@ repository's structure: the python side of the standard library. - `test`: CPython test suite - `vm/src`: python virtual machine - - `builtins.rs`: Builtin functions - - `compile.rs`: the python compiler from ast to bytecode - - `obj`: python builtin types + - `builtins`: Builtin functions and types - `stdlib`: Standard library parts implemented in rust. - `src`: using the other subcrates to bring rustpython to life. -- `docs`: documentation (work in progress) -- `py_code_object`: CPython bytecode to rustpython bytecode converter (work in - progress) - `wasm`: Binary crate and resources for WebAssembly build -- `extra_tests`: extra integration test snippets as supplement of `Lib/test` +- `extra_tests`: extra integration test snippets as a supplement to `Lib/test` ## Understanding Internals The RustPython workspace includes the `rustpython` top-level crate. The `Cargo.toml` file in the root of the repo provide configuration of the crate and the -implementation is found in the `src` directory (specifically, -`src/main.rs`). +implementation is found in the `src` directory (specifically, `src/lib.rs`). The top-level `rustpython` binary depends on several lower-level crates including: @@ -133,25 +128,26 @@ enable a line of code to go through a series of steps: This crate contains the lexer and parser to convert a line of code to an Abstract Syntax Tree (AST): -- Lexer: `parser/lexer.rs` converts Python source code into tokens -- Parser: `parser/parser.rs` takes the tokens generated by the lexer and parses +- Lexer: `parser/src/lexer.rs` converts Python source code into tokens +- Parser: `parser/src/parser.rs` takes the tokens generated by the lexer and parses the tokens into an AST (Abstract Syntax Tree) where the nodes of the syntax tree are Rust structs and enums. - - The Parser relies on `LALRPOP`, a Rust parser generator framework. + - The Parser relies on `LALRPOP`, a Rust parser generator framework. The + LALRPOP definition of Python's grammar is in `parser/src/python.lalrpop`. - More information on parsers and a tutorial can be found in the [LALRPOP book](https://lalrpop.github.io/lalrpop/README.html). -- AST: `parser/ast.rs` implements in Rust the Python types and expressions +- AST: `ast/` implements in Rust the Python types and expressions represented by the AST nodes. ### rustpython-compiler The `rustpython-compiler` crate's purpose is to transform the AST (Abstract Syntax Tree) to bytecode. The implementation of the compiler is found in the -`compiler/src` directory. The compiler implements Python's peephole optimizer -implementation, Symbol table, and streams in Rust. +`compiler/src` directory. The compiler implements Python's symbol table, +ast->bytecode compiler, and bytecode optimizer in Rust. Implementation of bytecode structure in Rust is found in the `bytecode/src` -directory. The `bytecode/src/bytecode.rs` contains the representation of +directory. `bytecode/src/lib.rs` contains the representation of instructions and operations in Rust. Further information about Python's bytecode instructions can be found in the [Python documentation](https://docs.python.org/3/library/dis.html#bytecodes). @@ -163,8 +159,10 @@ executes Python's instructions. The `vm/src` directory contains code to implement the read and evaluation loop that fetches and dispatches instructions. This directory also contains the implementation of the Python Standard Library modules in Rust (`vm/src/stdlib`). In Python -everything can be represented as an Object. `vm/src/obj` directory holds -the Rust code used to represent a Python Object and its methods. +everything can be represented as an object. The `vm/src/builtins` directory holds +the Rust code used to represent different Python objects and their methods. The +core implementation of what a Python object is can be found in +`vm/src/pyobjectrc.rs`. ## Questions