diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 9fd818536e..3c2eee6a12 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,9 +1,95 @@ # RustPython Development Guide and Tips +RustPython attracts developers with interest and experience in Rust, Python, +or WebAssembly. Whether you are familiar with Rust, Python, or +WebAssembly, the goal of this Development Guide is to give you the basics to +get set up for developing RustPython and contributing to this project. + +The contents of the Development Guide include: + +- [Setting up a development environment](#setting-up-a-development-environment) +- [Code style](#code-style) +- [Testing](#testing) +- [Profiling](#profiling) +- [Code organization](#code-organization) +- [Understanding internals](#understanding-internals) +- [Questions](#questions) + +## Setting up a development environment + +RustPython requires the following: + +- Rust 1.36 or higher + - To check Rust version: `rustc --version` + - If you have `rustup` on your system, enter to update to the latest + stable version: `rustup update stable` + - If you do not have Rust installed, use [rustup](https://rustup.rs/) to + do so. +- CPython version 3.7.4 or higher + - CPython can be installed by your operating system's package manager, + from the [Python website](https://www.python.org/downloads/), or + using a third-party distribution, such as + [Anaconda](https://www.anaconda.com/distribution/). +- [Optional] The Python package, `pytest`, is used for testing Python code + snippets. To install, enter `python3 -m pip install pytest`. + +## Code style + +The Rust code style used is the default +[rustfmt](https://github.com/rust-lang/rustfmt) codestyle. Please format your +code accordingly. We also use [clippy](https://github.com/rust-lang/rust-clippy) +to detect rust code issues. + +Python code should follow the +[PEP 8](https://www.python.org/dev/peps/pep-0008/) style. + +## Testing + +To test RustPython's functionality, a collection of Python snippets are located +in the `tests/snippets` directory and can be run using `pytest`: + +```shell +$ cd tests +$ pytest -v +``` + +Rust unit tests can be run with `cargo`: + +```shell +$ cargo test --all +``` + +## Profiling + +To profile RustPython, build it in `release` mode with the `flame-it` feature. +This will generate a file `flamescope.json`, which can be viewed at +https://speedscope.app. + +```shell +$ cargo run --release --features flame-it script.py +``` + +To display the default output json file: + +```shell +$ cat flamescope.json +{} +``` + +You can specify another file name other than the default by using the +`--output-file` option to specify a file name (or `stdout` if you specify `-`). +The `--output-format` option determines the format of the output file. +The speedscope json format (default), text, or raw html can be passed. There +exists a raw html viewer which is currently broken, and we welcome a PR to fix it. + ## Code organization +Understanding a new codebase takes time. Here's a brief view of the +repository's structure: + - `bytecode/src`: python bytecode representation in rust structures - `compiler/src`: python compilation to bytecode +- `derive/src`: Rust language extensions and macros specific to rustpython - `parser/src`: python lexing, parsing and ast - `Lib`: Carefully selected / copied files from CPython sourcecode. This is the python side of the standard library. @@ -19,42 +105,33 @@ - `wasm`: Binary crate and resources for WebAssembly build - `tests`: integration test snippets -## Code style +## Understanding Internals -The code style used is the default -[rustfmt](https://github.com/rust-lang/rustfmt) codestyle. Please format your -code accordingly. We also use [clippy](https://github.com/rust-lang/rust-clippy) -to detect rust code issues. +Rust crates + rustpython: top level crate + rustpython_parser + rustpython-vm (source is found in `vm/src`) -## Testing +Lexer, Parser, AST (Abstract Syntax Tree) +- Lexer: `parser/lexer.rs` converts Python source code into tokens +- Parser: Takes the tokens generated by the lexer and parses the tokens + into an AST (Abstract Syntax Tree) where the nodes of the syntax tree + are Rust structs and enums + - The Parser relies on `LALRPOP`, a Rust parser generator framework + - More information on parsers and a tutorial can be found in the + [LALRPOP book](https://lalrpop.github.io/lalrpop/README.html). -To test rustpython, there is a collection of python snippets located in the -`tests/snippets` directory. To run those tests do the following: +Compiler +Generates bytecode from the AST -```shell -$ cd tests -$ pytest -v -``` +VM (Virtual Machine) +Fetch and dispatch loop -There also are some unit tests, you can run those with cargo: +Import system -```shell -$ cargo test --all -``` +Builtin Objects -## Profiling +## Questions -To profile rustpython, simply build in release mode with the `flame-it` feature. -This will generate a file `flamescope.json`, which you can then view at -https://speedscope.app. - -```sh -$ cargo run --release --features flame-it script.py -$ cat flamescope.json -{} -``` - -You can also pass the `--output-file` option to choose which file to output to -(or stdout if you specify `-`), and the `--output-format` option to choose if -you want to output in the speedscope json format (default), text, or a raw html -viewer (currently broken). +Have you tried these steps and have a question, please chat with us on +[gitter](https://gitter.im/rustpython/Lobby).