Rust-related/RustPython
1
0
Fork 1
mirror of https://github.com/RustPython/RustPython.git synced 2026-06-02 19:39:49 +09:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' of https://github.com/rustpython/rustpython

Browse Source
This commit is contained in:
MinGyo Jung
2019-08-16 15:39:10 +09:00
parent f4e19889a1 7764de311a
commit bc64016b02
82 changed files with 1774 additions and 908 deletions
Download Patch File Download Diff File Expand all files Collapse all files

480
Cargo.lock generated
View File

File diff suppressed because it is too large Load Diff

165
DEVELOPMENT.md
View File

@@ -1,9 +1,91 @@
# RustPython Development Guide and Tips
RustPython attracts developers with interest and experience in Rust, Python,
or WebAssembly. Whether you are familiar with Rust, Python, or
WebAssembly, the goal of this Development Guide is to give you the basics to
get set up for developing RustPython and contributing to this project.
The contents of the Development Guide include:
- [Setting up a development environment](#setting-up-a-development-environment)
- [Code style](#code-style)
- [Testing](#testing)
- [Profiling](#profiling)
- [Code organization](#code-organization)
- [Understanding internals](#understanding-internals)
- [Questions](#questions)
## Setting up a development environment
RustPython requires the following:
- Rust 1.36 or higher
- To check Rust version: `rustc --version`
- If you have `rustup` on your system, enter to update to the latest
stable version: `rustup update stable`
- If you do not have Rust installed, use [rustup](https://rustup.rs/) to
do so.
- CPython version 3.7.4 or higher
- CPython can be installed by your operating system's package manager,
from the [Python website](https://www.python.org/downloads/), or
using a third-party distribution, such as
[Anaconda](https://www.anaconda.com/distribution/).
- [Optional] The Python package, `pytest`, is used for testing Python code
snippets. To install, enter `python3 -m pip install pytest`.
## Code style
The Rust code style used is the default
[rustfmt](https://github.com/rust-lang/rustfmt) codestyle. Please format your
code accordingly. We also use [clippy](https://github.com/rust-lang/rust-clippy)
to detect rust code issues.
Python code should follow the
[PEP 8](https://www.python.org/dev/peps/pep-0008/) style. We also use
[flake8](http://flake8.pycqa.org/en/latest/) to check Python code style.
## Testing
To test RustPython's functionality, a collection of Python snippets is located
in the `tests/snippets` directory and can be run using `pytest`:
```shell
$ cd tests
$ pytest -v
```
Rust unit tests can be run with `cargo`:
```shell
$ cargo test --all
```
## Profiling
To profile RustPython, build it in `release` mode with the `flame-it` feature.
This will generate a file `flamescope.json`, which can be viewed at
https://speedscope.app.
```shell
$ cargo run --release --features flame-it script.py
$ cat flamescope.json
{<json>}
```
You can specify another file name other than the default by using the
`--output-file` option to specify a file name (or `stdout` if you specify `-`).
The `--output-format` option determines the format of the output file.
The speedscope json format (default), text, or raw html can be passed. There
exists a raw html viewer which is currently broken, and we welcome a PR to fix it.
## Code organization
Understanding a new codebase takes time. Here's a brief view of the
repository's structure:
- `bytecode/src`: python bytecode representation in rust structures
- `compiler/src`: python compilation to bytecode
- `derive/src`: Rust language extensions and macros specific to rustpython
- `parser/src`: python lexing, parsing and ast
- `Lib`: Carefully selected / copied files from CPython sourcecode. This is
the python side of the standard library.
@@ -19,42 +101,67 @@
- `wasm`: Binary crate and resources for WebAssembly build
- `tests`: integration test snippets
## Code style
## Understanding Internals
The code style used is the default
[rustfmt](https://github.com/rust-lang/rustfmt) codestyle. Please format your
code accordingly. We also use [clippy](https://github.com/rust-lang/rust-clippy)
to detect rust code issues.
The RustPython workspace includes the `rustpython` top-level crate. The `Cargo.toml`
file in the root of the repo provide configuration of the crate and the
implementation is found in the `src` directory (specifically,
`src/main.rs`).
## Testing
The top-level `rustpython` binary depends on several lower-level crates including:
To test rustpython, there is a collection of python snippets located in the
`tests/snippets` directory. To run those tests do the following:
- `rustpython-parser` (implementation in `parser/src`)
- `rustpython-compiler` (implementation in `compiler/src`)
- `rustpython-vm` (implementation in `vm/src`)
```shell
$ cd tests
$ pytest -v
```
Together, these crates provide the functions of a programming language and
enable a line of code to go through a series of steps:
There also are some unit tests, you can run those with cargo:
- parse the line of source code into tokens
- determine if the tokens are valid syntax
- create an Abstract Syntax Tree (AST)
- compile the AST into bytecode
- execute the bytecode in the virtual machine (VM).
```shell
$ cargo test --all
```
### rustpython-parser
## Profiling
This crate contains the lexer and parser to convert a line of code to
an Abstract Syntax Tree (AST):
To profile rustpython, simply build in release mode with the `flame-it` feature.
This will generate a file `flamescope.json`, which you can then view at
https://speedscope.app.
- Lexer: `parser/lexer.rs` converts Python source code into tokens
- Parser: `parser/parser.rs` takes the tokens generated by the lexer and parses
the tokens into an AST (Abstract Syntax Tree) where the nodes of the syntax
tree are Rust structs and enums.
- The Parser relies on `LALRPOP`, a Rust parser generator framework.
- More information on parsers and a tutorial can be found in the
[LALRPOP book](https://lalrpop.github.io/lalrpop/README.html).
- AST: `parser/ast.rs` implements in Rust the Python types and expressions
represented by the AST nodes.
```sh
$ cargo run --release --features flame-it script.py
$ cat flamescope.json
{<json>}
```
### rustpython-compiler
You can also pass the `--output-file` option to choose which file to output to
(or stdout if you specify `-`), and the `--output-format` option to choose if
you want to output in the speedscope json format (default), text, or a raw html
viewer (currently broken).
The `rustpython-compiler` crate's purpose is to transform the AST (Abstract Syntax
Tree) to bytecode. The implementation of the compiler is found in the
`compiler/src` directory. The compiler implements Python's peephole optimizer
implementation, Symbol table, and streams in Rust.
Implementation of bytecode structure in Rust is found in the `bytecode/src`
directory. The `bytecode/src/bytecode.rs` contains the representation of
instructions and operations in Rust. Further information about Python's
bytecode instructions can be found in the
[Python documentation](https://docs.python.org/3/library/dis.html#bytecodes).
### rustpython-vm
The `rustpython-vm` crate has the important job of running the virtual machine that
executes Python's instructions. The `vm/src` directory contains code to
implement the read and evaluation loop that fetches and dispatches
instructions. This directory also contains the implementation of the
Python Standard Library modules in Rust (`vm/src/stdlib`). In Python
everything can be represented as an Object. `vm/src/obj` directory holds
the Rust code used to represent a Python Object and its methods.
## Questions
Have you tried these steps and have a question, please chat with us on
[gitter](https://gitter.im/rustpython/Lobby).

3
Lib/unittest/__main__.py
View File

@@ -2,8 +2,7 @@
import sys
if sys.argv[0].endswith("__main__.py"):
# FIXME change to `import os.path` as it was for cpython once `import os.path` works
import os
import os.path
# We change sys.argv[0] to make help message more useful
# use executable without path, unquoted
# (it's just a hint anyway)

7
bytecode/src/bytecode.rs
View File

@@ -84,10 +84,7 @@ pub enum Instruction {
symbols: Vec<String>,
level: usize,
},
ImportStar {
name: Option<String>,
level: usize,
},
ImportStar,
ImportFrom {
name: String,
},
@@ -429,7 +426,7 @@ impl Instruction {
format!("{:?}", symbols),
level
),
ImportStar { name, level } => w!(ImportStar, format!("{:?}", name), level),
ImportStar => w!(ImportStar),
ImportFrom { name } => w!(ImportFrom, name),
LoadName { name, scope } => w!(LoadName, name, format!("{:?}", scope)),
StoreName { name, scope } => w!(StoreName, name, format!("{:?}", scope)),

107
compiler/src/compile.rs
View File

@@ -188,7 +188,19 @@ impl<O: OutputStream> Compiler<O> {
) -> Result<(), CompileError> {
let size_before = self.output_stack.len();
self.symbol_table_stack.push(symbol_table);
self.compile_statements(&program.statements)?;
let (statements, doc) = get_doc(&program.statements);
if let Some(value) = doc {
self.emit(Instruction::LoadConst {
value: bytecode::Constant::String { value },
});
self.emit(Instruction::StoreName {
name: "__doc__".to_owned(),
scope: bytecode::NameScope::Global,
});
}
self.compile_statements(statements)?;
assert_eq!(self.output_stack.len(), size_before);
// Emit None at end:
@@ -305,11 +317,15 @@ impl<O: OutputStream> Compiler<O> {
symbols: vec![],
level: 0,
});
if let Some(alias) = &name.alias {
for part in name.symbol.split('.').skip(1) {
self.emit(Instruction::LoadAttr {
name: part.to_owned(),
});
}
self.store_name(alias);
} else {
self.store_name(&name.symbol);
self.store_name(name.symbol.split('.').next().unwrap());
}
}
}
@@ -322,10 +338,12 @@ impl<O: OutputStream> Compiler<O> {
if import_star {
// from .... import *
self.emit(Instruction::ImportStar {
self.emit(Instruction::Import {
name: module.clone(),
symbols: vec!["*".to_owned()],
level: *level,
});
self.emit(Instruction::ImportStar);
} else {
// from mod import a, b as c
// First, determine the fromlist (for import lib):
@@ -1891,30 +1909,36 @@ impl<O: OutputStream> Compiler<O> {
}
fn compile_string(&mut self, string: &ast::StringGroup) -> Result<(), CompileError> {
match string {
ast::StringGroup::Joined { values } => {
for value in values {
self.compile_string(value)?;
if let Some(value) = try_get_constant_string(string) {
self.emit(Instruction::LoadConst {
value: bytecode::Constant::String { value },
});
} else {
match string {
ast::StringGroup::Joined { values } => {
for value in values {
self.compile_string(value)?;
}
self.emit(Instruction::BuildString { size: values.len() })
}
ast::StringGroup::Constant { value } => {
self.emit(Instruction::LoadConst {
value: bytecode::Constant::String {
value: value.to_string(),
},
});
}
ast::StringGroup::FormattedValue {
value,
conversion,
spec,
} => {
self.compile_expression(value)?;
self.emit(Instruction::FormatValue {
conversion: conversion.map(compile_conversion_flag),
spec: spec.clone(),
});
}
self.emit(Instruction::BuildString { size: values.len() })
}
ast::StringGroup::Constant { value } => {
self.emit(Instruction::LoadConst {
value: bytecode::Constant::String {
value: value.to_string(),
},
});
}
ast::StringGroup::FormattedValue {
value,
conversion,
spec,
} => {
self.compile_expression(value)?;
self.emit(Instruction::FormatValue {
conversion: conversion.map(compile_conversion_flag),
spec: spec.clone(),
});
}
}
Ok(())
@@ -1994,13 +2018,11 @@ impl<O: OutputStream> Compiler<O> {
}
fn get_doc(body: &[ast::Statement]) -> (&[ast::Statement], Option<String>) {
if let Some(val) = body.get(0) {
if let Some((val, body_rest)) = body.split_first() {
if let ast::StatementType::Expression { ref expression } = val.node {
if let ast::ExpressionType::String { value } = &expression.node {
if let ast::StringGroup::Constant { ref value } = value {
if let Some((_, body_rest)) = body.split_first() {
return (body_rest, Some(value.to_string()));
}
if let Some(value) = try_get_constant_string(value) {
return (body_rest, Some(value.to_string()));
}
}
}
@@ -2008,6 +2030,27 @@ fn get_doc(body: &[ast::Statement]) -> (&[ast::Statement], Option<String>) {
(body, None)
}
fn try_get_constant_string(string: &ast::StringGroup) -> Option<String> {
fn get_constant_string_inner(out_string: &mut String, string: &ast::StringGroup) -> bool {
match string {
ast::StringGroup::Constant { value } => {
out_string.push_str(&value);
true
}
ast::StringGroup::Joined { values } => values
.iter()
.all(|value| get_constant_string_inner(out_string, value)),
ast::StringGroup::FormattedValue { .. } => false,
}
}
let mut out_string = String::new();
if get_constant_string_inner(&mut out_string, string) {
Some(out_string)
} else {
None
}
}
fn compile_location(location: &ast::Location) -> bytecode::Location {
bytecode::Location::new(location.row(), location.column())
}

5
compiler/src/symboltable.rs
View File

@@ -388,7 +388,10 @@ impl SymbolTableBuilder {
self.register_name(alias, SymbolUsage::Assigned)?;
} else {
// `import module`
self.register_name(&name.symbol, SymbolUsage::Assigned)?;
self.register_name(
name.symbol.split('.').next().unwrap(),
SymbolUsage::Assigned,
)?;
}
}
}

4
src/main.rs
View File

@@ -312,7 +312,7 @@ fn run_rustpython(vm: &VirtualMachine, matches: &ArgMatches) -> PyResult<()> {
}
let scope = vm.new_scope_with_builtins();
let main_module = vm.ctx.new_module("__main__", scope.globals.clone());
let main_module = vm.new_module("__main__", scope.globals.clone());
vm.get_attribute(vm.sys_module.clone(), "modules")?
.set_item("__main__", main_module, vm)?;
@@ -359,7 +359,7 @@ fn run_module(vm: &VirtualMachine, module: &str) -> PyResult<()> {
debug!("Running module {}", module);
let runpy = vm.import("runpy", &vm.ctx.new_tuple(vec![]), 0)?;
let run_module_as_main = vm.get_attribute(runpy, "_run_module_as_main")?;
vm.invoke(run_module_as_main, vec![vm.new_str(module.to_owned())])?;
vm.invoke(&run_module_as_main, vec![vm.new_str(module.to_owned())])?;
Ok(())
}

4
tests/snippets/builtin_complex.py
View File

@@ -145,3 +145,7 @@ assert '(1+1j)' == str(1+1j)
assert '(1-1j)' == str(1-1j)
assert '(1+1j)' == repr(1+1j)
assert '(1-1j)' == repr(1-1j)
# __getnewargs__
assert (3 + 5j).__getnewargs__() == (3.0, 5.0)
assert (5j).__getnewargs__() == (0.0, 5.0)

21
tests/snippets/builtin_round.py Normal file
View File

@@ -0,0 +1,21 @@
from testutils import assertRaises
assert round(0) == 0
assert isinstance(round(0), int)
assert round(0.0) == 0
assert isinstance(round(0.0), int)
assert round(0, None) == 0
assert isinstance(round(0, None), int)
assert round(0.0, None) == 0
assert isinstance(round(0, None), int)
assert round(0, 0) == 0
assert isinstance(round(0, 0), int)
assert round(0.0, 0) == 0.0 # Cannot check the type
assert isinstance(round(0.0, 0), float)
with assertRaises(TypeError):
round(0, 0.0)
with assertRaises(TypeError):
round(0.0, 0.0)

17
tests/snippets/ellipsis.py
View File

@@ -11,6 +11,23 @@ assert b is c
assert b is d
assert d is e
assert Ellipsis.__repr__() == 'Ellipsis'
assert Ellipsis.__reduce__() == 'Ellipsis'
assert type(Ellipsis).__new__(type(Ellipsis)) == Ellipsis
assert type(Ellipsis).__reduce__(Ellipsis) == 'Ellipsis'
try:
type(Ellipsis).__new__(type(1))
except TypeError:
pass
else:
assert False, '`Ellipsis.__new__` should only accept `type(Ellipsis)` as argument'
try:
type(Ellipsis).__reduce__(1)
except TypeError:
pass
else:
assert False, '`Ellipsis.__reduce__` should only accept `Ellipsis` as argument'
assert Ellipsis is ...
Ellipsis = 2
assert Ellipsis is not ...

16
tests/snippets/floats.py
View File

@@ -151,12 +151,20 @@ assert float(1.2) == 1.2
assert math.trunc(1.2) == 1
assert_raises(OverflowError, float('inf').__trunc__)
assert_raises(ValueError, float('nan').__trunc__)
assert 0.5.__round__() == 0.0
assert 1.5.__round__() == 2.0
assert isinstance(0.5.__round__(), int)
assert isinstance(1.5.__round__(), int)
assert 0.5.__round__() == 0
assert 1.5.__round__() == 2
assert isinstance(0.5.__round__(0), float)
assert isinstance(1.5.__round__(0), float)
assert 0.5.__round__(0) == 0.0
assert 1.5.__round__(0) == 2.0
assert 0.5.__round__(None) == 0.0
assert 1.5.__round__(None) == 2.0
assert isinstance(0.5.__round__(None), int)
assert isinstance(1.5.__round__(None), int)
assert 0.5.__round__(None) == 0
assert 1.5.__round__(None) == 2
assert_raises(TypeError, lambda: 0.5.__round__(0.0))
assert_raises(TypeError, lambda: 1.5.__round__(0.0))
assert_raises(OverflowError, float('inf').__round__)
assert_raises(ValueError, float('nan').__round__)

13
tests/snippets/ints.py
View File

@@ -167,3 +167,16 @@ class F(float):
return 3
assert int(F(1.2)) == 3
assert isinstance((0).__round__(), int)
assert isinstance((1).__round__(), int)
assert (0).__round__() == 0
assert (1).__round__() == 1
assert isinstance((0).__round__(0), int)
assert isinstance((1).__round__(0), int)
assert (0).__round__(0) == 0
assert (1).__round__(0) == 1
assert_raises(TypeError, lambda: (0).__round__(None))
assert_raises(TypeError, lambda: (1).__round__(None))
assert_raises(TypeError, lambda: (0).__round__(0.0))
assert_raises(TypeError, lambda: (1).__round__(0.0))

67
tests/snippets/json_snippet.py
View File

@@ -1,5 +1,6 @@
from testutils import assert_raises
import json
from io import StringIO
def round_trip_test(obj):
# serde_json and Python's json module produce slightly differently spaced
@@ -7,63 +8,129 @@ def round_trip_test(obj):
# proxy
return obj == json.loads(json.dumps(obj))
def json_dump(obj):
f = StringIO()
json.dump(obj, f)
f.seek(0)
return f.getvalue()
def json_load(obj):
f = StringIO(obj)
return json.load(f)
assert '"string"' == json.dumps("string")
assert '"string"' == json_dump("string")
assert "1" == json.dumps(1)
assert "1" == json_dump(1)
assert "1.0" == json.dumps(1.0)
assert "1.0" == json_dump(1.0)
assert "true" == json.dumps(True)
assert "true" == json_dump(True)
assert "false" == json.dumps(False)
assert "false" == json_dump(False)
assert 'null' == json.dumps(None)
assert 'null' == json_dump(None)
assert '[]' == json.dumps([])
assert '[]' == json_dump([])
assert '[1]' == json.dumps([1])
assert '[1]' == json_dump([1])
assert '[[1]]' == json.dumps([[1]])
assert '[[1]]' == json_dump([[1]])
assert round_trip_test([1, "string", 1.0, True])
assert '[]' == json.dumps(())
assert '[]' == json_dump(())
assert '[1]' == json.dumps((1,))
assert '[1]' == json_dump((1,))
assert '[[1]]' == json.dumps(((1,),))
assert '[[1]]' == json_dump(((1,),))
# tuples don't round-trip through json
assert [1, "string", 1.0, True] == json.loads(json.dumps((1, "string", 1.0, True)))
assert '{}' == json.dumps({})
assert '{}' == json_dump({})
assert round_trip_test({'a': 'b'})
# should reject non-str keys in jsons
assert_raises(json.JSONDecodeError, lambda: json.loads('{3: "abc"}'))
assert_raises(json.JSONDecodeError, lambda: json_load('{3: "abc"}'))
# should serialize non-str keys as strings
assert json.dumps({'3': 'abc'}) == json.dumps({3: 'abc'})
assert 1 == json.loads("1")
assert 1 == json_load("1")
assert -1 == json.loads("-1")
assert -1 == json_load("-1")
assert 1.0 == json.loads("1.0")
assert 1.0 == json_load("1.0")
assert -1.0 == json.loads("-1.0")
assert -1.0 == json_load("-1.0")
assert "str" == json.loads('"str"')
assert "str" == json_load('"str"')
assert True is json.loads('true')
assert True is json_load('true')
assert False is json.loads('false')
assert False is json_load('false')
assert None is json.loads('null')
assert None is json_load('null')
assert [] == json.loads('[]')
assert [] == json_load('[]')
assert ['a'] == json.loads('["a"]')
assert ['a'] == json_load('["a"]')
assert [['a'], 'b'] == json.loads('[["a"], "b"]')
assert [['a'], 'b'] == json_load('[["a"], "b"]')
class String(str): pass
assert "string" == json.loads(String('"string"'))
assert "string" == json_load(String('"string"'))
assert '"string"' == json.dumps(String("string"))
assert '"string"' == json_dump(String("string"))
class Int(int): pass
class Float(float): pass
assert '1' == json.dumps(Int(1))
assert '1' == json_dump(Int(1))
assert '0.5' == json.dumps(Float(0.5))
assert '0.5' == json_dump(Float(0.5))
class List(list): pass
class Tuple(tuple): pass
class Dict(dict): pass
assert '[1]' == json.dumps(List([1]))
assert '[1]' == json_dump(List([1]))
assert json.dumps((1, "string", 1.0, True)) == json.dumps(Tuple((1, "string", 1.0, True)))
assert json_dump((1, "string", 1.0, True)) == json_dump(Tuple((1, "string", 1.0, True)))
assert json.dumps({'a': 'b'}) == json.dumps(Dict({'a': 'b'}))
assert json_dump({'a': 'b'}) == json_dump(Dict({'a': 'b'}))
# big ints should not crash VM
# TODO: test for correct output when actual serialization implemented and doesn’t throw

4
tests/snippets/object.py
View File

@@ -7,6 +7,10 @@ myobj = MyObject()
assert myobj == myobj
assert not myobj != myobj
object.__subclasshook__() == NotImplemented
object.__subclasshook__(1) == NotImplemented
object.__subclasshook__(1, 2) == NotImplemented
assert MyObject().__eq__(MyObject()) == NotImplemented
assert MyObject().__ne__(MyObject()) == NotImplemented
assert MyObject().__lt__(MyObject()) == NotImplemented

79
tests/snippets/stdlib_functools.py Normal file
View File

@@ -0,0 +1,79 @@
from functools import reduce
from testutils import assertRaises
class Squares:
def __init__(self, max):
self.max = max
self.sofar = []
def __len__(self):
return len(self.sofar)
def __getitem__(self, i):
if not 0 <= i < self.max: raise IndexError
n = len(self.sofar)
while n <= i:
self.sofar.append(n*n)
n += 1
return self.sofar[i]
def add(a, b):
return a + b
assert reduce(add, ['a', 'b', 'c']) == 'abc'
assert reduce(add, ['a', 'b', 'c'], str(42)) == '42abc'
assert reduce(add, [['a', 'c'], [], ['d', 'w']], []) == ['a','c','d','w']
assert reduce(add, [['a', 'c'], [], ['d', 'w']], []) == ['a','c','d','w']
assert reduce(lambda x, y: x*y, range(2, 21), 1) == 2432902008176640000
assert reduce(add, Squares(10)) == 285
assert reduce(add, Squares(10), 0) == 285
assert reduce(add, Squares(0), 0) == 0
assert reduce(42, "1") == "1"
assert reduce(42, "", "1") == "1"
with assertRaises(TypeError):
reduce()
with assertRaises(TypeError):
reduce(42, 42)
with assertRaises(TypeError):
reduce(42, 42, 42)
class TestFailingIter:
def __iter__(self):
raise RuntimeError
with assertRaises(RuntimeError):
reduce(add, TestFailingIter())
assert reduce(add, [], None) == None
assert reduce(add, [], 42) == 42
class BadSeq:
def __getitem__(self, index):
raise ValueError
with assertRaises(ValueError):
reduce(42, BadSeq())
# Test reduce()'s use of iterators.
class SequenceClass:
def __init__(self, n):
self.n = n
def __getitem__(self, i):
if 0 <= i < self.n:
return i
else:
raise IndexError
assert reduce(add, SequenceClass(5)) == 10
assert reduce(add, SequenceClass(5), 42) == 52
with assertRaises(TypeError):
reduce(add, SequenceClass(0))
assert reduce(add, SequenceClass(0), 42) == 42
assert reduce(add, SequenceClass(1)) == 0
assert reduce(add, SequenceClass(1), 42) == 42
d = {"one": 1, "two": 2, "three": 3}
assert reduce(add, d) == "".join(d.keys())

7
tests/snippets/stdlib_io.py
View File

@@ -1,5 +1,6 @@
from io import BufferedReader, FileIO, StringIO, BytesIO
import os
from testutils import assertRaises
fi = FileIO('README.md')
assert fi.seekable()
@@ -23,3 +24,9 @@ fd = os.open('README.md', os.O_RDONLY)
with FileIO(fd) as fio:
res2 = fio.read()
assert res == res2
fi = FileIO('README.md')
fi.read()
fi.close()
with assertRaises(ValueError):
fi.read()

6
tests/snippets/stdlib_socket.py
View File

@@ -119,6 +119,12 @@ with assertRaises(OSError):
with assertRaises(OSError):
socket.inet_aton("test")
with assertRaises(OverflowError):
socket.htonl(-1)
assert socket.htonl(0)==0
assert socket.htonl(10)==167772160
assert socket.inet_aton("127.0.0.1")==b"\x7f\x00\x00\x01"
assert socket.inet_aton("255.255.255.255")==b"\xff\xff\xff\xff"

3
vm/Cargo.toml
View File

@@ -79,3 +79,6 @@ libz-sys = "1.0.25"
gethostname = "0.2.0"
subprocess = "0.1.18"
num_cpus = "1.0"
[target."cfg(windows)".dependencies]
kernel32-sys = "0.2.2"

10
vm/Lib/_bootstrap_external.py
View File

@@ -26,6 +26,16 @@ _CASE_INSENSITIVE_PLATFORMS = (_CASE_INSENSITIVE_PLATFORMS_BYTES_KEY
+ _CASE_INSENSITIVE_PLATFORMS_STR_KEY)
def _w_long(x):
"""Convert a 32-bit integer to little-endian."""
return (int(x) & 0xFFFFFFFF).to_bytes(4, 'little')
def _r_long(int_bytes):
"""Convert 4 bytes in little-endian to an integer."""
return int.from_bytes(int_bytes, 'little')
def _make_relax_case():
if sys.platform.startswith(_CASE_INSENSITIVE_PLATFORMS):
if sys.platform.startswith(_CASE_INSENSITIVE_PLATFORMS_STR_KEY):

44
vm/src/builtins.rs
View File

@@ -37,7 +37,7 @@ fn builtin_abs(x: PyObjectRef, vm: &VirtualMachine) -> PyResult {
let method = vm.get_method_or_type_error(x.clone(), "__abs__", || {
format!("bad operand type for abs(): '{}'", x.class().name)
})?;
vm.invoke(method, PyFuncArgs::new(vec![], vec![]))
vm.invoke(&method, PyFuncArgs::new(vec![], vec![]))
}
fn builtin_all(iterable: PyIterable<bool>, vm: &VirtualMachine) -> PyResult<bool> {
@@ -395,7 +395,7 @@ fn builtin_len(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
let method = vm.get_method_or_type_error(obj.clone(), "__len__", || {
format!("object of type '{}' has no len()", obj.class().name)
})?;
vm.invoke(method, PyFuncArgs::default())
vm.invoke(&method, PyFuncArgs::default())
}
fn builtin_locals(vm: &VirtualMachine) -> PyDictRef {
@@ -428,15 +428,15 @@ fn builtin_max(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
let mut x = candidates_iter.next().unwrap();
// TODO: this key function looks pretty duplicate. Maybe we can create
// a local function?
let mut x_key = if let Some(f) = &key_func {
vm.invoke(f.clone(), vec![x.clone()])?
let mut x_key = if let Some(ref f) = &key_func {
vm.invoke(f, vec![x.clone()])?
} else {
x.clone()
};
for y in candidates_iter {
let y_key = if let Some(f) = &key_func {
vm.invoke(f.clone(), vec![y.clone()])?
let y_key = if let Some(ref f) = &key_func {
vm.invoke(f, vec![y.clone()])?
} else {
y.clone()
};
@@ -476,15 +476,15 @@ fn builtin_min(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
let mut x = candidates_iter.next().unwrap();
// TODO: this key function looks pretty duplicate. Maybe we can create
// a local function?
let mut x_key = if let Some(f) = &key_func {
vm.invoke(f.clone(), vec![x.clone()])?
let mut x_key = if let Some(ref f) = &key_func {
vm.invoke(f, vec![x.clone()])?
} else {
x.clone()
};
for y in candidates_iter {
let y_key = if let Some(f) = &key_func {
vm.invoke(f.clone(), vec![y.clone()])?
let y_key = if let Some(ref f) = &key_func {
vm.invoke(f, vec![y.clone()])?
} else {
y.clone()
};
@@ -711,7 +711,7 @@ fn builtin_reversed(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
arg_check!(vm, args, required = [(obj, None)]);
if let Some(reversed_method) = vm.get_method(obj.clone(), "__reversed__") {
vm.invoke(reversed_method?, PyFuncArgs::default())
vm.invoke(&reversed_method?, PyFuncArgs::default())
} else {
vm.get_method_or_type_error(obj.clone(), "__getitem__", || {
"argument to reversed() must be a sequence".to_string()
@@ -734,9 +734,19 @@ fn builtin_round(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
optional = [(ndigits, None)]
);
if let Some(ndigits) = ndigits {
let ndigits = vm.call_method(ndigits, "__int__", vec![])?;
let rounded = vm.call_method(number, "__round__", vec![ndigits])?;
Ok(rounded)
if objtype::isinstance(ndigits, &vm.ctx.int_type()) {
let ndigits = vm.call_method(ndigits, "__int__", vec![])?;
let rounded = vm.call_method(number, "__round__", vec![ndigits])?;
Ok(rounded)
} else if vm.ctx.none().is(ndigits) {
let rounded = &vm.call_method(number, "__round__", vec![])?;
Ok(vm.ctx.new_int(objint::get_value(rounded).clone()))
} else {
Err(vm.new_type_error(format!(
"'{}' object cannot be interpreted as an integer",
ndigits.class().name
)))
}
} else {
// without a parameter, the result type is coerced to int
let rounded = &vm.call_method(number, "__round__", vec![])?;
@@ -777,7 +787,7 @@ fn builtin_sum(iterable: PyIterable, start: OptionalArg, vm: &VirtualMachine) ->
// Should be renamed to builtin___import__?
fn builtin_import(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
vm.invoke(vm.import_func.borrow().clone(), args)
vm.invoke(&vm.import_func.borrow(), args)
}
fn builtin_vars(obj: OptionalArg, vm: &VirtualMachine) -> PyResult {
@@ -964,13 +974,13 @@ pub fn builtin_build_class_(
// Prepare uses full __getattribute__ resolution chain.
let prepare = vm.get_attribute(metaclass.clone().into_object(), "__prepare__")?;
let namespace = vm.invoke(prepare, vec![name_obj.clone(), bases.clone()])?;
let namespace = vm.invoke(&prepare, vec![name_obj.clone(), bases.clone()])?;
let namespace: PyDictRef = TryFromObject::try_from_object(vm, namespace)?;
let cells = vm.ctx.new_dict();
vm.invoke_with_locals(function, cells.clone(), namespace.clone())?;
vm.invoke_with_locals(&function, cells.clone(), namespace.clone())?;
namespace.set_item("__name__", name_obj.clone(), vm)?;
namespace.set_item("__qualname__", qualified_name.into_object(), vm)?;

3
vm/src/exceptions.rs
View File

@@ -3,7 +3,8 @@ use crate::obj::objsequence;
use crate::obj::objtuple::{PyTuple, PyTupleRef};
use crate::obj::objtype;
use crate::obj::objtype::PyClassRef;
use crate::pyobject::{create_type, IdProtocol, PyContext, PyObjectRef, PyResult, TypeProtocol};
use crate::pyobject::{IdProtocol, PyContext, PyObjectRef, PyResult, TypeProtocol};
use crate::types::create_type;
use crate::vm::VirtualMachine;
use itertools::Itertools;
use std::fs::File;

29
vm/src/frame.rs
View File

@@ -198,10 +198,7 @@ impl Frame {
ref symbols,
ref level,
} => self.import(vm, name, symbols, *level),
bytecode::Instruction::ImportStar {
ref name,
ref level,
} => self.import_star(vm, name, *level),
bytecode::Instruction::ImportStar => self.import_star(vm),
bytecode::Instruction::ImportFrom { ref name } => self.import_from(vm, name),
bytecode::Instruction::LoadName {
ref name,
@@ -488,7 +485,7 @@ impl Frame {
// Call function:
let func_ref = self.pop_value();
let value = vm.invoke(func_ref, args)?;
let value = vm.invoke(&func_ref, args)?;
self.push_value(value);
Ok(None)
}
@@ -599,7 +596,7 @@ impl Frame {
if !expr.is(&vm.get_none()) {
let repr = vm.to_repr(&expr)?;
// TODO: implement sys.displayhook
if let Ok(print) = vm.get_attribute(vm.builtins.clone(), "print") {
if let Ok(ref print) = vm.get_attribute(vm.builtins.clone(), "print") {
vm.invoke(print, vec![repr.into_object()])?;
}
}
@@ -740,25 +737,23 @@ impl Frame {
// Load attribute, and transform any error into import error.
let obj = vm
.get_attribute(module, name)
.map_err(|_| vm.new_import_error(format!("cannot import name '{}'", name)));
self.push_value(obj?);
.map_err(|_| vm.new_import_error(format!("cannot import name '{}'", name)))?;
self.push_value(obj);
Ok(None)
}
#[cfg_attr(feature = "flame-it", flame("Frame"))]
fn import_star(
&self,
vm: &VirtualMachine,
module: &Option<String>,
level: usize,
) -> FrameResult {
let module = module.clone().unwrap_or_default();
let module = vm.import(&module, &vm.ctx.new_tuple(vec![]), level)?;
fn import_star(&self, vm: &VirtualMachine) -> FrameResult {
let module = self.pop_value();
// Grab all the names from the module and put them in the context
if let Some(dict) = &module.dict {
for (k, v) in dict {
self.scope.store_name(&vm, &objstr::get_value(&k), v);
let k = vm.to_str(&k)?;
let k = k.as_str();
if !k.starts_with('_') {
self.scope.store_name(&vm, k, v);
}
}
}
Ok(None)

12
vm/src/function.rs
View File

@@ -393,6 +393,18 @@ impl<T> OptionalArg<T> {
}
}
pub type OptionalOption<T> = OptionalArg<Option<T>>;
impl<T> OptionalOption<T> {
#[inline]
pub fn flat_option(self) -> Option<T> {
match self {
Present(Some(value)) => Some(value),
_ => None,
}
}
}
impl<T> FromArgs for OptionalArg<T>
where
T: TryFromObject,

6
vm/src/import.rs
View File

@@ -17,14 +17,14 @@ pub fn init_importlib(vm: &VirtualMachine, external: bool) -> PyResult {
let importlib = import_frozen(vm, "_frozen_importlib")?;
let impmod = import_builtin(vm, "_imp")?;
let install = vm.get_attribute(importlib.clone(), "_install")?;
vm.invoke(install, vec![vm.sys_module.clone(), impmod])?;
vm.invoke(&install, vec![vm.sys_module.clone(), impmod])?;
vm.import_func
.replace(vm.get_attribute(importlib.clone(), "__import__")?);
if external && cfg!(feature = "rustpython-compiler") {
flame_guard!("install_external");
let install_external =
vm.get_attribute(importlib.clone(), "_install_external_importers")?;
vm.invoke(install_external, vec![])?;
vm.invoke(&install_external, vec![])?;
// Set pyc magic number to commit hash. Should be changed when bytecode will be more stable.
let importlib_external =
vm.import("_frozen_importlib_external", &vm.ctx.new_tuple(vec![]), 0)?;
@@ -87,7 +87,7 @@ pub fn import_codeobj(
if set_file_attr {
attrs.set_item("__file__", vm.new_str(code_obj.source_path.to_owned()), vm)?;
}
let module = vm.ctx.new_module(module_name, attrs.clone());
let module = vm.new_module(module_name, attrs.clone());
// Store module in cache to prevent infinite loop with mutual importing libs:
let sys_modules = vm.get_attribute(vm.sys_module.clone(), "modules")?;

1
vm/src/lib.rs
View File

@@ -68,6 +68,7 @@ pub mod scope;
pub mod stdlib;
mod sysmodule;
mod traceback;
pub mod types;
pub mod util;
mod version;
mod vm;

2
vm/src/macros.rs
View File

@@ -117,7 +117,7 @@ macro_rules! no_kwargs {
#[macro_export]
macro_rules! py_module {
( $vm:expr, $module_name:expr, { $($name:expr => $value:expr),* $(,)* }) => {{
let module = $vm.ctx.new_module($module_name, $vm.ctx.new_dict());
let module = $vm.new_module($module_name, $vm.ctx.new_dict());
$vm.set_attr(&module, "__name__", $vm.ctx.new_str($module_name.to_string())).unwrap();
$(
$vm.set_attr(&module, $name, $value).unwrap();

6
vm/src/obj/objbool.rs
View File

@@ -28,7 +28,7 @@ pub fn boolval(vm: &VirtualMachine, obj: PyObjectRef) -> PyResult<bool> {
Some(method_or_err) => {
// If descriptor returns Error, propagate it further
let method = method_or_err?;
let bool_obj = vm.invoke(method, PyFuncArgs::default())?;
let bool_obj = vm.invoke(&method, PyFuncArgs::default())?;
match bool_obj.payload::<PyInt>() {
Some(int_obj) => !int_obj.as_bigint().is_zero(),
None => {
@@ -42,7 +42,7 @@ pub fn boolval(vm: &VirtualMachine, obj: PyObjectRef) -> PyResult<bool> {
None => match vm.get_method(obj.clone(), "__len__") {
Some(method_or_err) => {
let method = method_or_err?;
let bool_obj = vm.invoke(method, PyFuncArgs::default())?;
let bool_obj = vm.invoke(&method, PyFuncArgs::default())?;
match bool_obj.payload::<PyInt>() {
Some(int_obj) => !int_obj.as_bigint().is_zero(),
None => {
@@ -66,7 +66,7 @@ Returns True when the argument x is true, False otherwise.
The builtins True and False are the only two instances of the class bool.
The class bool is a subclass of the class int, and cannot be subclassed.";
let bool_type = &context.bool_type;
let bool_type = &context.types.bool_type;
extend_class!(context, bool_type, {
"__new__" => context.new_rustfunc(bool_new),
"__repr__" => context.new_rustfunc(bool_repr),

6
vm/src/obj/objbytearray.rs
View File

@@ -78,14 +78,14 @@ impl PyValue for PyByteArray {
/// Fill bytearray class methods dictionary.
pub fn init(context: &PyContext) {
PyByteArrayRef::extend_class(context, &context.bytearray_type);
let bytearray_type = &context.bytearray_type;
PyByteArrayRef::extend_class(context, &context.types.bytearray_type);
let bytearray_type = &context.types.bytearray_type;
extend_class!(context, bytearray_type, {
"fromhex" => context.new_rustfunc(PyByteArrayRef::fromhex),
"maketrans" => context.new_rustfunc(PyByteInner::maketrans),
});
PyByteArrayIterator::extend_class(context, &context.bytearrayiterator_type);
PyByteArrayIterator::extend_class(context, &context.types.bytearrayiterator_type);
}
#[pyimpl]

6
vm/src/obj/objbytes.rs
View File

@@ -85,14 +85,14 @@ pub fn get_value<'a>(obj: &'a PyObjectRef) -> impl Deref<Target = Vec<u8>> + 'a
}
pub fn init(context: &PyContext) {
PyBytesRef::extend_class(context, &context.bytes_type);
let bytes_type = &context.bytes_type;
PyBytesRef::extend_class(context, &context.types.bytes_type);
let bytes_type = &context.types.bytes_type;
extend_class!(context, bytes_type, {
"fromhex" => context.new_rustfunc(PyBytesRef::fromhex),
"maketrans" => context.new_rustfunc(PyByteInner::maketrans),
});
PyBytesIterator::extend_class(context, &context.bytesiterator_type);
PyBytesIterator::extend_class(context, &context.types.bytesiterator_type);
}
#[pyimpl]

2
vm/src/obj/objclassmethod.rs
View File

@@ -65,5 +65,5 @@ impl PyClassMethod {
}
pub fn init(context: &PyContext) {
PyClassMethod::extend_class(context, &context.classmethod_type);
PyClassMethod::extend_class(context, &context.types.classmethod_type);
}

2
vm/src/obj/objcode.rs
View File

@@ -81,7 +81,7 @@ impl PyCodeRef {
}
pub fn init(context: &PyContext) {
extend_class!(context, &context.code_type, {
extend_class!(context, &context.types.code_type, {
"__new__" => context.new_rustfunc(PyCodeRef::new),
"__repr__" => context.new_rustfunc(PyCodeRef::repr),

10
vm/src/obj/objcomplex.rs
View File

@@ -41,7 +41,7 @@ impl From<Complex64> for PyComplex {
}
pub fn init(context: &PyContext) {
PyComplex::extend_class(context, &context.complex_type);
PyComplex::extend_class(context, &context.types.complex_type);
}
pub fn get_value(obj: &PyObjectRef) -> Complex64 {
@@ -258,4 +258,12 @@ impl PyComplex {
let ret = Wrapping(re_hash) + Wrapping(im_hash) * Wrapping(pyhash::IMAG);
ret.0
}
#[pymethod(name = "__getnewargs__")]
fn complex_getnewargs(&self, vm: &VirtualMachine) -> PyResult {
let Complex64 { re, im } = self.value;
Ok(vm
.ctx
.new_tuple(vec![vm.ctx.new_float(re), vm.ctx.new_float(im)]))
}
}

20
vm/src/obj/objdict.rs
View File

@@ -210,7 +210,7 @@ impl PyDictRef {
}
if let Some(method_or_err) = vm.get_method(self.clone().into_object(), "__missing__") {
let method = method_or_err?;
return vm.invoke(method, vec![key]);
return vm.invoke(&method, vec![key]);
}
Err(vm.new_key_error(key.clone()))
}
@@ -423,7 +423,7 @@ macro_rules! dict_iterator {
impl PyValue for $name {
fn class(vm: &VirtualMachine) -> PyClassRef {
vm.ctx.$class.clone()
vm.ctx.types.$class.clone()
}
}
@@ -473,7 +473,7 @@ macro_rules! dict_iterator {
impl PyValue for $iter_name {
fn class(vm: &VirtualMachine) -> PyClassRef {
vm.ctx.$iter_class.clone()
vm.ctx.types.$iter_class.clone()
}
}
};
@@ -511,7 +511,7 @@ dict_iterator! {
}
pub fn init(context: &PyContext) {
extend_class!(context, &context.dict_type, {
extend_class!(context, &context.types.dict_type, {
"__bool__" => context.new_rustfunc(PyDictRef::bool),
"__len__" => context.new_rustfunc(PyDictRef::len),
"__contains__" => context.new_rustfunc(PyDictRef::contains),
@@ -536,10 +536,10 @@ pub fn init(context: &PyContext) {
"popitem" => context.new_rustfunc(PyDictRef::popitem),
});
PyDictKeys::extend_class(context, &context.dictkeys_type);
PyDictKeyIterator::extend_class(context, &context.dictkeyiterator_type);
PyDictValues::extend_class(context, &context.dictvalues_type);
PyDictValueIterator::extend_class(context, &context.dictvalueiterator_type);
PyDictItems::extend_class(context, &context.dictitems_type);
PyDictItemIterator::extend_class(context, &context.dictitemiterator_type);
PyDictKeys::extend_class(context, &context.types.dictkeys_type);
PyDictKeyIterator::extend_class(context, &context.types.dictkeyiterator_type);
PyDictValues::extend_class(context, &context.types.dictvalues_type);
PyDictValueIterator::extend_class(context, &context.types.dictvalueiterator_type);
PyDictItems::extend_class(context, &context.types.dictitems_type);
PyDictItemIterator::extend_class(context, &context.types.dictitemiterator_type);
}

28
vm/src/obj/objellipsis.rs
View File

@@ -1,20 +1,30 @@
use crate::function::PyFuncArgs;
use crate::pyobject::{PyContext, PyResult};
use crate::obj::objtype::{issubclass, PyClassRef};
use crate::pyobject::{PyContext, PyEllipsisRef, PyResult};
use crate::vm::VirtualMachine;
pub fn init(context: &PyContext) {
extend_class!(context, &context.ellipsis_type, {
"__new__" => context.new_rustfunc(ellipsis_new),
"__repr__" => context.new_rustfunc(ellipsis_repr)
"__repr__" => context.new_rustfunc(ellipsis_repr),
"__reduce__" => context.new_rustfunc(ellipsis_reduce),
});
}
fn ellipsis_new(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
arg_check!(vm, args, required = [(_cls, None)]);
Ok(vm.ctx.ellipsis())
fn ellipsis_new(cls: PyClassRef, vm: &VirtualMachine) -> PyResult {
if issubclass(&cls, &vm.ctx.ellipsis_type) {
Ok(vm.ctx.ellipsis())
} else {
Err(vm.new_type_error(format!(
"ellipsis.__new__({ty}): {ty} is not a subtype of ellipsis",
ty = cls,
)))
}
}
fn ellipsis_repr(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
arg_check!(vm, args, required = [(_cls, None)]);
Ok(vm.new_str("Ellipsis".to_string()))
fn ellipsis_repr(_self: PyEllipsisRef, _vm: &VirtualMachine) -> String {
"Ellipsis".to_string()
}
fn ellipsis_reduce(_self: PyEllipsisRef, _vm: &VirtualMachine) -> String {
"Ellipsis".to_string()
}

4
vm/src/obj/objenumerate.rs
View File

@@ -68,8 +68,8 @@ impl PyEnumerate {
}
pub fn init(context: &PyContext) {
PyEnumerate::extend_class(context, &context.enumerate_type);
extend_class!(context, &context.enumerate_type, {
PyEnumerate::extend_class(context, &context.types.enumerate_type);
extend_class!(context, &context.types.enumerate_type, {
"__new__" => context.new_rustfunc(enumerate_new),
});
}

6
vm/src/obj/objfilter.rs
View File

@@ -52,7 +52,7 @@ impl PyFilter {
} else {
// the predicate itself can raise StopIteration which does stop the filter
// iteration
vm.invoke(predicate.clone(), vec![next_obj.clone()])?
vm.invoke(&predicate, vec![next_obj.clone()])?
};
if objbool::boolval(vm, predicate_value)? {
return Ok(next_obj);
@@ -67,8 +67,8 @@ impl PyFilter {
}
pub fn init(context: &PyContext) {
PyFilter::extend_class(context, &context.filter_type);
extend_class!(context, &context.filter_type, {
PyFilter::extend_class(context, &context.types.filter_type);
extend_class!(context, &context.types.filter_type, {
"__new__" => context.new_rustfunc(filter_new),
});
}

39
vm/src/obj/objfloat.rs
View File

@@ -440,25 +440,38 @@ impl PyFloat {
OptionalArg::Missing => None,
OptionalArg::Present(ref value) => {
if !vm.get_none().is(value) {
let ndigits = if objtype::isinstance(value, &vm.ctx.int_type()) {
objint::get_value(value)
} else {
if !objtype::isinstance(value, &vm.ctx.int_type()) {
return Err(vm.new_type_error(format!(
"TypeError: '{}' object cannot be interpreted as an integer",
"'{}' object cannot be interpreted as an integer",
value.class().name
)));
};
if ndigits.is_zero() {
None
} else {
Some(ndigits)
}
// Only accept int type ndigits
let ndigits = objint::get_value(value);
Some(ndigits)
} else {
None
}
}
};
if ndigits.is_none() {
if let Some(ndigits) = ndigits {
if ndigits.is_zero() {
let fract = self.value.fract();
let value = if (fract.abs() - 0.5).abs() < std::f64::EPSILON {
if self.value.trunc() % 2.0 == 0.0 {
self.value - fract
} else {
self.value + fract
}
} else {
self.value.round()
};
Ok(vm.ctx.new_float(value))
} else {
Ok(vm.ctx.not_implemented())
}
} else {
let fract = self.value.fract();
let value = if (fract.abs() - 0.5).abs() < std::f64::EPSILON {
if self.value.trunc() % 2.0 == 0.0 {
@@ -471,8 +484,6 @@ impl PyFloat {
};
let int = try_to_bigint(value, vm)?;
Ok(vm.ctx.new_int(int))
} else {
Ok(vm.ctx.not_implemented())
}
}
@@ -612,12 +623,12 @@ pub fn make_float(vm: &VirtualMachine, obj: &PyObjectRef) -> PyResult<f64> {
obj.class().name
)
})?;
let result = vm.invoke(method, vec![])?;
let result = vm.invoke(&method, vec![])?;
Ok(get_value(&result))
}
}
#[rustfmt::skip] // to avoid line splitting
pub fn init(context: &PyContext) {
PyFloat::extend_class(context, &context.float_type);
PyFloat::extend_class(context, &context.types.float_type);
}

2
vm/src/obj/objframe.rs
View File

@@ -9,7 +9,7 @@ use crate::pyobject::{PyContext, PyObjectRef, PyResult};
use crate::vm::VirtualMachine;
pub fn init(context: &PyContext) {
extend_class!(context, &context.frame_type, {
extend_class!(context, &context.types.frame_type, {
"__new__" => context.new_rustfunc(FrameRef::new),
"__repr__" => context.new_rustfunc(FrameRef::repr),
"f_locals" => context.new_property(FrameRef::flocals),

6
vm/src/obj/objfunction.rs
View File

@@ -42,7 +42,7 @@ impl PyValue for PyFunction {
impl PyFunctionRef {
fn call(self, args: Args, kwargs: KwArgs, vm: &VirtualMachine) -> PyResult {
vm.invoke(self.into_object(), (&args, &kwargs))
vm.invoke(&self.into_object(), (&args, &kwargs))
}
fn code(self, _vm: &VirtualMachine) -> PyCodeRef {
@@ -78,7 +78,7 @@ impl PyValue for PyMethod {
}
pub fn init(context: &PyContext) {
let function_type = &context.function_type;
let function_type = &context.types.function_type;
extend_class!(context, function_type, {
"__get__" => context.new_rustfunc(bind_method),
"__call__" => context.new_rustfunc(PyFunctionRef::call),
@@ -87,7 +87,7 @@ pub fn init(context: &PyContext) {
"__kwdefaults__" => context.new_property(PyFunctionRef::kwdefaults),
});
let builtin_function_or_method_type = &context.builtin_function_or_method_type;
let builtin_function_or_method_type = &context.types.builtin_function_or_method_type;
extend_class!(context, builtin_function_or_method_type, {
"__get__" => context.new_rustfunc(bind_method)
});

2
vm/src/obj/objgenerator.rs
View File

@@ -75,5 +75,5 @@ fn handle_execution_result(result: ExecutionResult, vm: &VirtualMachine) -> PyRe
}
pub fn init(ctx: &PyContext) {
PyGenerator::extend_class(ctx, &ctx.generator_type);
PyGenerator::extend_class(ctx, &ctx.types.generator_type);
}

38
vm/src/obj/objint.rs
View File

@@ -9,13 +9,14 @@ use crate::function::{KwArgs, OptionalArg, PyFuncArgs};
use crate::obj::objtype::PyClassRef;
use crate::pyhash;
use crate::pyobject::{
IntoPyObject, PyClassImpl, PyContext, PyObjectRef, PyRef, PyResult, PyValue, TryFromObject,
TypeProtocol,
IdProtocol, IntoPyObject, PyClassImpl, PyContext, PyObjectRef, PyRef, PyResult, PyValue,
TryFromObject, TypeProtocol,
};
use crate::vm::VirtualMachine;
use super::objbyteinner::PyByteInner;
use super::objbytes::PyBytes;
use super::objint;
use super::objstr::{PyString, PyStringRef};
use super::objtype;
@@ -468,8 +469,29 @@ impl PyInt {
zelf: PyRef<Self>,
_precision: OptionalArg<PyObjectRef>,
_vm: &VirtualMachine,
) -> PyIntRef {
zelf
) -> PyResult<PyIntRef> {
let _ndigits = match _precision {
OptionalArg::Missing => None,
OptionalArg::Present(ref value) => {
if !_vm.get_none().is(value) {
if !objtype::isinstance(value, &_vm.ctx.int_type()) {
return Err(_vm.new_type_error(format!(
"'{}' object cannot be interpreted as an integer",
value.class().name
)));
};
// Only accept int type _ndigits
let _ndigits = objint::get_value(value);
Some(_ndigits)
} else {
return Err(_vm.new_type_error(format!(
"'{}' object cannot be interpreted as an integer",
value.class().name
)));
}
}
};
Ok(zelf)
}
#[pymethod(name = "__int__")]
@@ -681,7 +703,7 @@ impl IntOptions {
fn get_int_value(self, vm: &VirtualMachine) -> PyResult<BigInt> {
if let OptionalArg::Present(val) = self.val_options {
let base = if let OptionalArg::Present(base) = self.base {
if !objtype::isinstance(&val, &vm.ctx.str_type) {
if !objtype::isinstance(&val, &vm.ctx.str_type()) {
return Err(vm.new_type_error(
"int() can't convert non-string with explicit base".to_string(),
));
@@ -724,7 +746,7 @@ pub fn to_int(vm: &VirtualMachine, obj: &PyObjectRef, mut base: u32) -> PyResult
let method = vm.get_method_or_type_error(obj.clone(), "__int__", || {
format!("int() argument must be a string or a number, not '{}'", obj.class().name)
})?;
let result = vm.invoke(method, PyFuncArgs::default())?;
let result = vm.invoke(&method, PyFuncArgs::default())?;
match result.payload::<PyInt>() {
Some(int_obj) => Ok(int_obj.as_bigint().clone()),
None => Err(vm.new_type_error(format!(
@@ -776,8 +798,8 @@ fn div_ints(vm: &VirtualMachine, i1: &BigInt, i2: &BigInt) -> PyResult {
}
pub fn init(context: &PyContext) {
PyInt::extend_class(context, &context.int_type);
extend_class!(context, &context.int_type, {
PyInt::extend_class(context, &context.types.int_type);
extend_class!(context, &context.types.int_type, {
"__new__" => context.new_rustfunc(int_new),
});
}

4
vm/src/obj/objiter.rs
View File

@@ -20,7 +20,7 @@ use super::objtype::PyClassRef;
pub fn get_iter(vm: &VirtualMachine, iter_target: &PyObjectRef) -> PyResult {
if let Some(method_or_err) = vm.get_method(iter_target.clone(), "__iter__") {
let method = method_or_err?;
vm.invoke(method, vec![])
vm.invoke(&method, vec![])
} else {
vm.get_method_or_type_error(iter_target.clone(), "__getitem__", || {
format!("Cannot iterate over {}", iter_target.class().name)
@@ -122,5 +122,5 @@ impl PySequenceIterator {
}
pub fn init(context: &PyContext) {
PySequenceIterator::extend_class(context, &context.iter_type);
PySequenceIterator::extend_class(context, &context.types.iter_type);
}

8
vm/src/obj/objlist.rs
View File

@@ -755,7 +755,7 @@ fn do_sort(
for x in values.iter() {
keys.push(match &key_func {
None => x.clone(),
Some(ref func) => vm.invoke((*func).clone(), vec![x.clone()])?,
Some(ref func) => vm.invoke(func, vec![x.clone()])?,
});
}
@@ -859,7 +859,7 @@ impl PyListReverseIterator {
#[rustfmt::skip] // to avoid line splitting
pub fn init(context: &PyContext) {
let list_type = &context.list_type;
let list_type = &context.types.list_type;
let list_doc = "Built-in mutable sequence.\n\n\
If no argument is given, the constructor creates a new empty list.\n\
@@ -901,6 +901,6 @@ pub fn init(context: &PyContext) {
"remove" => context.new_rustfunc(PyListRef::remove)
});
PyListIterator::extend_class(context, &context.listiterator_type);
PyListReverseIterator::extend_class(context, &context.listreverseiterator_type);
PyListIterator::extend_class(context, &context.types.listiterator_type);
PyListReverseIterator::extend_class(context, &context.types.listreverseiterator_type);
}

6
vm/src/obj/objmap.rs
View File

@@ -51,7 +51,7 @@ impl PyMap {
.collect::<Result<Vec<_>, _>>()?;
// the mapper itself can raise StopIteration which does stop the map iteration
vm.invoke(self.mapper.clone(), next_objs)
vm.invoke(&self.mapper, next_objs)
}
#[pymethod(name = "__iter__")]
@@ -61,8 +61,8 @@ impl PyMap {
}
pub fn init(context: &PyContext) {
PyMap::extend_class(context, &context.map_type);
extend_class!(context, &context.map_type, {
PyMap::extend_class(context, &context.types.map_type);
extend_class!(context, &context.types.map_type, {
"__new__" => context.new_rustfunc(map_new),
});
}

4
vm/src/obj/objmappingproxy.rs
View File

@@ -13,7 +13,7 @@ pub type PyMappingProxyRef = PyRef<PyMappingProxy>;
impl PyValue for PyMappingProxy {
fn class(vm: &VirtualMachine) -> PyClassRef {
vm.ctx.mappingproxy_type.clone()
vm.ctx.types.mappingproxy_type.clone()
}
}
@@ -38,5 +38,5 @@ impl PyMappingProxy {
}
pub fn init(context: &PyContext) {
PyMappingProxy::extend_class(context, &context.mappingproxy_type)
PyMappingProxy::extend_class(context, &context.types.mappingproxy_type)
}

2
vm/src/obj/objmemory.rs
View File

@@ -47,5 +47,5 @@ impl PyValue for PyMemoryView {
}
pub fn init(ctx: &PyContext) {
PyMemoryView::extend_class(ctx, &ctx.memoryview_type)
PyMemoryView::extend_class(ctx, &ctx.types.memoryview_type)
}

70
vm/src/obj/objmodule.rs
View File

@@ -1,6 +1,8 @@
use crate::function::OptionalOption;
use crate::obj::objdict::PyDictRef;
use crate::obj::objstr::PyStringRef;
use crate::obj::objtype::PyClassRef;
use crate::pyobject::{PyContext, PyRef, PyResult, PyValue};
use crate::pyobject::{ItemProtocol, PyContext, PyObjectRef, PyRef, PyResult, PyValue};
use crate::vm::VirtualMachine;
#[derive(Debug)]
@@ -17,15 +19,71 @@ impl PyValue for PyModule {
}
}
pub fn init_module_dict(
vm: &VirtualMachine,
module_dict: &PyDictRef,
name: PyObjectRef,
doc: PyObjectRef,
) {
module_dict
.set_item("__name__", name, vm)
.expect("Failed to set __name__ on module");
module_dict
.set_item("__doc__", doc, vm)
.expect("Failed to set __doc__ on module");
module_dict
.set_item("__package__", vm.get_none(), vm)
.expect("Failed to set __package__ on module");
module_dict
.set_item("__loader__", vm.get_none(), vm)
.expect("Failed to set __loader__ on module");
module_dict
.set_item("__spec__", vm.get_none(), vm)
.expect("Failed to set __spec__ on module");
}
impl PyModuleRef {
fn init(self, name: PyStringRef, vm: &VirtualMachine) -> PyResult {
vm.set_attr(&self.into_object(), "__name__", name)?;
Ok(vm.get_none())
fn new(
cls: PyClassRef,
name: PyStringRef,
doc: OptionalOption<PyStringRef>,
vm: &VirtualMachine,
) -> PyResult<PyModuleRef> {
let zelf = PyModule {
name: name.as_str().to_owned(),
}
.into_ref_with_type(vm, cls)?;
init_module_dict(
vm,
zelf.as_object().dict.as_ref().unwrap(),
name.into_object(),
doc.flat_option()
.map_or_else(|| vm.get_none(), PyRef::into_object),
);
Ok(zelf)
}
fn getattribute(self, name: PyStringRef, vm: &VirtualMachine) -> PyResult {
vm.generic_getattribute(self.as_object().clone(), name.clone())?
.ok_or_else(|| {
vm.new_attribute_error(format!(
"module '{}' has no attribute '{}'",
self.name, name,
))
})
}
fn repr(self, vm: &VirtualMachine) -> PyResult {
let importlib = vm.import("_frozen_importlib", &vm.ctx.new_tuple(vec![]), 0)?;
let module_repr = vm.get_attribute(importlib, "_module_repr")?;
vm.invoke(&module_repr, vec![self.into_object()])
}
}
pub fn init(context: &PyContext) {
extend_class!(&context, &context.module_type, {
"__init__" => context.new_rustfunc(PyModuleRef::init),
extend_class!(&context, &context.types.module_type, {
"__new__" => context.new_rustfunc(PyModuleRef::new),
"__getattribute__" => context.new_rustfunc(PyModuleRef::getattribute),
"__repr__" => context.new_rustfunc(PyModuleRef::repr),
});
}

2
vm/src/obj/objnamespace.rs
View File

@@ -28,5 +28,5 @@ impl PyNamespace {
}
pub fn init(context: &PyContext) {
PyNamespace::extend_class(context, &context.namespace_type);
PyNamespace::extend_class(context, &context.types.namespace_type);
}

4
vm/src/obj/objnone.rs
View File

@@ -64,7 +64,7 @@ impl PyNoneRef {
if let Ok(property) = PyPropertyRef::try_from_object(vm, descriptor.clone()) {
property.instance_binding_get(obj, vm)
} else {
vm.invoke(get_func, vec![descriptor, obj, cls])
vm.invoke(&get_func, vec![descriptor, obj, cls])
}
}
@@ -95,7 +95,7 @@ impl PyNoneRef {
Ok(attr)
}
} else if let Some(getter) = class_get_attr(&cls, "__getattr__") {
vm.invoke(getter, vec![self.into_object(), name.into_object()])
vm.invoke(&getter, vec![self.into_object(), name.into_object()])
} else {
Err(vm.new_attribute_error(format!("{} has no attribute '{}'", self.as_object(), name)))
}

54
vm/src/obj/objobject.rs
View File

@@ -24,7 +24,7 @@ impl PyValue for PyInstance {
pub fn new_instance(vm: &VirtualMachine, mut args: PyFuncArgs) -> PyResult {
// more or less __new__ operator
let cls = PyClassRef::try_from_object(vm, args.shift())?;
let dict = if cls.is(&vm.ctx.object) {
let dict = if cls.is(&vm.ctx.object()) {
None
} else {
Some(vm.ctx.new_dict())
@@ -72,7 +72,7 @@ fn object_setattr(
if let Some(attr) = objtype::class_get_attr(&cls, &attr_name.value) {
if let Some(descriptor) = objtype::class_get_attr(&attr.class(), "__set__") {
return vm
.invoke(descriptor, vec![attr, obj.clone(), value])
.invoke(&descriptor, vec![attr, obj.clone(), value])
.map(|_| ());
}
}
@@ -94,7 +94,7 @@ fn object_delattr(obj: PyObjectRef, attr_name: PyStringRef, vm: &VirtualMachine)
if let Some(attr) = objtype::class_get_attr(&cls, &attr_name.value) {
if let Some(descriptor) = objtype::class_get_attr(&attr.class(), "__delete__") {
return vm.invoke(descriptor, vec![attr, obj.clone()]).map(|_| ());
return vm.invoke(&descriptor, vec![attr, obj.clone()]).map(|_| ());
}
}
@@ -118,6 +118,10 @@ fn object_repr(zelf: PyObjectRef, _vm: &VirtualMachine) -> String {
format!("<{} object at 0x{:x}>", zelf.class().name, zelf.get_id())
}
fn object_subclasshook(vm: &VirtualMachine, _args: PyFuncArgs) -> PyResult {
Ok(vm.ctx.not_implemented())
}
pub fn object_dir(obj: PyObjectRef, vm: &VirtualMachine) -> PyResult<PyList> {
let attributes: PyAttributes = objtype::get_attributes(obj.class());
@@ -126,7 +130,7 @@ pub fn object_dir(obj: PyObjectRef, vm: &VirtualMachine) -> PyResult<PyList> {
// Get instance attributes:
if let Some(object_dict) = &obj.dict {
vm.invoke(
vm.get_attribute(dict.clone().into_object(), "update")?,
&vm.get_attribute(dict.clone().into_object(), "update")?,
object_dict.clone().into_object(),
)?;
}
@@ -149,7 +153,7 @@ fn object_format(
}
pub fn init(context: &PyContext) {
let object = &context.object;
let object = &context.types.object_type;
let object_doc = "The most base type";
extend_class!(context, object, {
@@ -179,7 +183,8 @@ pub fn init(context: &PyContext) {
"__repr__" => context.new_rustfunc(object_repr),
"__format__" => context.new_rustfunc(object_format),
"__getattribute__" => context.new_rustfunc(object_getattribute),
"__doc__" => context.new_str(object_doc.to_string())
"__subclasshook__" => context.new_classmethod(object_subclasshook),
"__doc__" => context.new_str(object_doc.to_string()),
});
}
@@ -218,39 +223,8 @@ fn object_dict_setter(
))
}
fn object_getattribute(obj: PyObjectRef, name_str: PyStringRef, vm: &VirtualMachine) -> PyResult {
let name = &name_str.value;
fn object_getattribute(obj: PyObjectRef, name: PyStringRef, vm: &VirtualMachine) -> PyResult {
vm_trace!("object.__getattribute__({:?}, {:?})", obj, name);
let cls = obj.class();
if let Some(attr) = objtype::class_get_attr(&cls, &name) {
let attr_class = attr.class();
if objtype::class_has_attr(&attr_class, "__set__") {
if let Some(descriptor) = objtype::class_get_attr(&attr_class, "__get__") {
return vm.invoke(descriptor, vec![attr, obj, cls.into_object()]);
}
}
}
if let Some(obj_attr) = object_getattr(&obj, &name, &vm)? {
Ok(obj_attr)
} else if let Some(attr) = objtype::class_get_attr(&cls, &name) {
vm.call_get_descriptor(attr, obj)
} else if let Some(getter) = objtype::class_get_attr(&cls, "__getattr__") {
vm.invoke(getter, vec![obj, name_str.into_object()])
} else {
Err(vm.new_attribute_error(format!("{} has no attribute '{}'", obj, name)))
}
}
fn object_getattr(
obj: &PyObjectRef,
attr_name: &str,
vm: &VirtualMachine,
) -> PyResult<Option<PyObjectRef>> {
if let Some(ref dict) = obj.dict {
dict.get_item_option(attr_name, vm)
} else {
Ok(None)
}
vm.generic_getattribute(obj.clone(), name.clone())?
.ok_or_else(|| vm.new_attribute_error(format!("{} has no attribute '{}'", obj, name)))
}

22
vm/src/obj/objproperty.rs
View File

@@ -37,7 +37,7 @@ impl PyReadOnlyProperty {
if obj.is(vm.ctx.none.as_object()) {
Ok(zelf.into_object())
} else {
vm.invoke(zelf.getter.clone(), obj)
vm.invoke(&zelf.getter, obj)
}
}
}
@@ -123,8 +123,8 @@ impl PyProperty {
// specialised version that doesn't check for None
pub(crate) fn instance_binding_get(&self, obj: PyObjectRef, vm: &VirtualMachine) -> PyResult {
if let Some(getter) = self.getter.as_ref() {
vm.invoke(getter.clone(), obj)
if let Some(ref getter) = self.getter.as_ref() {
vm.invoke(getter, obj)
} else {
Err(vm.new_attribute_error("unreadable attribute".to_string()))
}
@@ -141,7 +141,7 @@ impl PyProperty {
if obj.is(vm.ctx.none.as_object()) {
Ok(zelf.into_object())
} else {
vm.invoke(getter.clone(), obj)
vm.invoke(&getter, obj)
}
} else {
Err(vm.new_attribute_error("unreadable attribute".to_string()))
@@ -150,8 +150,8 @@ impl PyProperty {
#[pymethod(name = "__set__")]
fn set(&self, obj: PyObjectRef, value: PyObjectRef, vm: &VirtualMachine) -> PyResult {
if let Some(setter) = self.setter.as_ref() {
vm.invoke(setter.clone(), vec![obj, value])
if let Some(ref setter) = self.setter.as_ref() {
vm.invoke(setter, vec![obj, value])
} else {
Err(vm.new_attribute_error("can't set attribute".to_string()))
}
@@ -159,8 +159,8 @@ impl PyProperty {
#[pymethod(name = "__delete__")]
fn delete(&self, obj: PyObjectRef, vm: &VirtualMachine) -> PyResult {
if let Some(deleter) = self.deleter.as_ref() {
vm.invoke(deleter.clone(), obj)
if let Some(ref deleter) = self.deleter.as_ref() {
vm.invoke(deleter, obj)
} else {
Err(vm.new_attribute_error("can't delete attribute".to_string()))
}
@@ -305,12 +305,12 @@ impl<'a> PropertyBuilder<'a> {
}
pub fn init(context: &PyContext) {
PyReadOnlyProperty::extend_class(context, &context.readonly_property_type);
PyProperty::extend_class(context, &context.property_type);
PyReadOnlyProperty::extend_class(context, &context.types.readonly_property_type);
PyProperty::extend_class(context, &context.types.property_type);
// This is a bit unfortunate, but this instance attribute overlaps with the
// class __doc__ string..
extend_class!(context, &context.property_type, {
extend_class!(context, &context.types.property_type, {
"__doc__" =>
PropertyBuilder::new(context)
.add_getter(PyProperty::doc_getter)

4
vm/src/obj/objrange.rs
View File

@@ -105,8 +105,8 @@ pub fn get_value(obj: &PyObjectRef) -> PyRange {
}
pub fn init(context: &PyContext) {
PyRange::extend_class(context, &context.range_type);
PyRangeIterator::extend_class(context, &context.rangeiterator_type);
PyRange::extend_class(context, &context.types.range_type);
PyRangeIterator::extend_class(context, &context.types.rangeiterator_type);
}
type PyRangeRef = PyRef<PyRange>;

4
vm/src/obj/objset.rs
View File

@@ -748,6 +748,6 @@ impl TryFromObject for SetIterable {
}
pub fn init(context: &PyContext) {
PySet::extend_class(context, &context.set_type);
PyFrozenSet::extend_class(context, &context.frozenset_type);
PySet::extend_class(context, &context.types.set_type);
PyFrozenSet::extend_class(context, &context.types.frozenset_type);
}

2
vm/src/obj/objslice.rs
View File

@@ -114,7 +114,7 @@ fn to_index_value(vm: &VirtualMachine, obj: &PyObjectRef) -> PyResult<Option<Big
}
pub fn init(context: &PyContext) {
let slice_type = &context.slice_type;
let slice_type = &context.types.slice_type;
extend_class!(context, slice_type, {
"__new__" => context.new_rustfunc(slice_new),

2
vm/src/obj/objstaticmethod.rs
View File

@@ -32,7 +32,7 @@ impl PyStaticMethodRef {
}
pub fn init(context: &PyContext) {
let staticmethod_type = &context.staticmethod_type;
let staticmethod_type = &context.types.staticmethod_type;
extend_class!(context, staticmethod_type, {
"__get__" => context.new_rustfunc(PyStaticMethodRef::get),
"__new__" => context.new_rustfunc(PyStaticMethodRef::new),

12
vm/src/obj/objstr.rs
View File

@@ -1146,10 +1146,10 @@ impl IntoPyObject for &String {
}
pub fn init(ctx: &PyContext) {
PyString::extend_class(ctx, &ctx.str_type);
PyString::extend_class(ctx, &ctx.types.str_type);
PyStringIterator::extend_class(ctx, &ctx.striterator_type);
PyStringReverseIterator::extend_class(ctx, &ctx.strreverseiterator_type);
PyStringIterator::extend_class(ctx, &ctx.types.striterator_type);
PyStringReverseIterator::extend_class(ctx, &ctx.types.strreverseiterator_type);
}
pub fn get_value(obj: &PyObjectRef) -> String {
@@ -1328,9 +1328,9 @@ fn do_cformat(
} else {
// check for only literal parts, in which case only dict or empty tuple is allowed
if num_specifiers == 0
&& !(objtype::isinstance(&values_obj, &vm.ctx.tuple_type)
&& !(objtype::isinstance(&values_obj, &vm.ctx.types.tuple_type)
&& objtuple::get_value(&values_obj).is_empty())
&& !objtype::isinstance(&values_obj, &vm.ctx.dict_type)
&& !objtype::isinstance(&values_obj, &vm.ctx.types.dict_type)
{
return Err(vm.new_type_error(
"not all arguments converted during string formatting".to_string(),
@@ -1396,7 +1396,7 @@ fn do_cformat(
.into_iter()
.nth(tuple_index)
.is_some())
&& !objtype::isinstance(&values_obj, &vm.ctx.dict_type)
&& !objtype::isinstance(&values_obj, &vm.ctx.types.dict_type)
{
return Err(
vm.new_type_error("not all arguments converted during string formatting".to_string())

2
vm/src/obj/objsuper.rs
View File

@@ -33,7 +33,7 @@ impl PyValue for PySuper {
}
pub fn init(context: &PyContext) {
let super_type = &context.super_type;
let super_type = &context.types.super_type;
let super_doc = "super() -> same as super(__class__, <first argument>)\n\
super(type) -> unbound super object\n\

4
vm/src/obj/objtuple.rs
View File

@@ -276,7 +276,7 @@ impl PyTupleIterator {
#[rustfmt::skip] // to avoid line splitting
pub fn init(context: &PyContext) {
let tuple_type = &context.tuple_type;
let tuple_type = &context.types.tuple_type;
let tuple_doc = "tuple() -> empty tuple
tuple(iterable) -> tuple initialized from iterable's items
@@ -303,5 +303,5 @@ If the argument is a tuple, the return value is the same object.";
"index" => context.new_rustfunc(PyTupleRef::index)
});
PyTupleIterator::extend_class(context, &context.tupleiterator_type);
PyTupleIterator::extend_class(context, &context.types.tupleiterator_type);
}

18
vm/src/obj/objtype.rs
View File

@@ -137,7 +137,7 @@ impl PyClassRef {
if let Some(attr) = class_get_attr(&mcl, &name) {
let attr_class = attr.class();
if class_has_attr(&attr_class, "__set__") {
if let Some(descriptor) = class_get_attr(&attr_class, "__get__") {
if let Some(ref descriptor) = class_get_attr(&attr_class, "__get__") {
return vm.invoke(
descriptor,
vec![attr, self.into_object(), mcl.into_object()],
@@ -148,7 +148,7 @@ impl PyClassRef {
if let Some(attr) = class_get_attr(&self, &name) {
let attr_class = attr.class();
if let Some(descriptor) = class_get_attr(&attr_class, "__get__") {
if let Some(ref descriptor) = class_get_attr(&attr_class, "__get__") {
let none = vm.get_none();
return vm.invoke(descriptor, vec![attr, none, self.into_object()]);
}
@@ -158,7 +158,7 @@ impl PyClassRef {
Ok(cls_attr)
} else if let Some(attr) = class_get_attr(&mcl, &name) {
vm.call_get_descriptor(attr, self.into_object())
} else if let Some(getter) = class_get_attr(&self, "__getattr__") {
} else if let Some(ref getter) = class_get_attr(&self, "__getattr__") {
vm.invoke(getter, vec![mcl.into_object(), name_ref.into_object()])
} else {
Err(vm.new_attribute_error(format!("{} has no attribute '{}'", self, name)))
@@ -172,7 +172,7 @@ impl PyClassRef {
vm: &VirtualMachine,
) -> PyResult<()> {
if let Some(attr) = class_get_attr(&self.class(), &attr_name.value) {
if let Some(descriptor) = class_get_attr(&attr.class(), "__set__") {
if let Some(ref descriptor) = class_get_attr(&attr.class(), "__set__") {
vm.invoke(descriptor, vec![attr, self.into_object(), value])?;
return Ok(());
}
@@ -212,7 +212,7 @@ pub fn init(ctx: &PyContext) {
type(object) -> the object's type\n\
type(name, bases, dict) -> a new type";
extend_class!(&ctx, &ctx.type_type, {
extend_class!(&ctx, &ctx.types.type_type, {
"__call__" => ctx.new_rustfunc(type_call),
"__dict__" =>
PropertyBuilder::new(ctx)
@@ -288,11 +288,11 @@ pub fn type_call(class: PyClassRef, args: Args, kwargs: KwArgs, vm: &VirtualMach
vm_trace!("type_call: {:?}", class);
let new = class_get_attr(&class, "__new__").expect("All types should have a __new__.");
let new_wrapped = vm.call_get_descriptor(new, class.into_object())?;
let obj = vm.invoke(new_wrapped, (&args, &kwargs))?;
let obj = vm.invoke(&new_wrapped, (&args, &kwargs))?;
if let Some(init_method_or_err) = vm.get_method(obj.clone(), "__init__") {
let init_method = init_method_or_err?;
let res = vm.invoke(init_method, (&args, &kwargs))?;
let res = vm.invoke(&init_method, (&args, &kwargs))?;
if !res.is(&vm.get_none()) {
return Err(vm.new_type_error("__init__ must return None".to_string()));
}
@@ -433,8 +433,8 @@ mod tests {
#[test]
fn test_linearise() {
let context = PyContext::new();
let object: PyClassRef = context.object.clone();
let type_type = &context.type_type;
let object: PyClassRef = context.object();
let type_type = &context.types.type_type;
let a = new(type_type.clone(), "A", vec![object.clone()], HashMap::new()).unwrap();
let b = new(type_type.clone(), "B", vec![object.clone()], HashMap::new()).unwrap();

2
vm/src/obj/objweakproxy.rs
View File

@@ -61,5 +61,5 @@ impl PyWeakProxy {
}
pub fn init(context: &PyContext) {
PyWeakProxy::extend_class(&context, &context.weakproxy_type);
PyWeakProxy::extend_class(&context, &context.types.weakproxy_type);
}

2
vm/src/obj/objweakref.rs
View File

@@ -48,7 +48,7 @@ impl PyWeakRef {
}
pub fn init(context: &PyContext) {
extend_class!(context, &context.weakref_type, {
extend_class!(context, &context.types.weakref_type, {
"__new__" => context.new_rustfunc(PyWeakRef::create),
"__call__" => context.new_rustfunc(PyWeakRef::call)
});

4
vm/src/obj/objzip.rs
View File

@@ -51,8 +51,8 @@ impl PyZip {
}
pub fn init(context: &PyContext) {
PyZip::extend_class(context, &context.zip_type);
extend_class!(context, &context.zip_type, {
PyZip::extend_class(context, &context.types.zip_type);
extend_class!(context, &context.types.zip_type, {
"__new__" => context.new_rustfunc(zip_new),
});
}

430
vm/src/pyobject.rs
View File

@@ -1,12 +1,9 @@
use std::any::Any;
use std::cell::Cell;
use std::cell::RefCell;
use std::collections::HashMap;
use std::fmt;
use std::marker::PhantomData;
use std::mem;
use std::ops::Deref;
use std::ptr;
use std::rc::Rc;
use num_bigint::BigInt;
@@ -16,46 +13,30 @@ use num_traits::{One, Zero};
use crate::bytecode;
use crate::exceptions;
use crate::function::{IntoPyNativeFunc, PyFuncArgs};
use crate::obj::objbool;
use crate::obj::objbuiltinfunc::PyBuiltinFunction;
use crate::obj::objbytearray;
use crate::obj::objbytes;
use crate::obj::objclassmethod::{self, PyClassMethod};
use crate::obj::objclassmethod::PyClassMethod;
use crate::obj::objcode;
use crate::obj::objcode::PyCodeRef;
use crate::obj::objcomplex::{self, PyComplex};
use crate::obj::objdict::{self, PyDict, PyDictRef};
use crate::obj::objellipsis;
use crate::obj::objenumerate;
use crate::obj::objfilter;
use crate::obj::objfloat::{self, PyFloat};
use crate::obj::objframe;
use crate::obj::objfunction::{self, PyFunction, PyMethod};
use crate::obj::objgenerator;
use crate::obj::objint::{self, PyInt, PyIntRef};
use crate::obj::objcomplex::PyComplex;
use crate::obj::objdict::{PyDict, PyDictRef};
use crate::obj::objfloat::PyFloat;
use crate::obj::objfunction::{PyFunction, PyMethod};
use crate::obj::objint::{PyInt, PyIntRef};
use crate::obj::objiter;
use crate::obj::objlist::{self, PyList};
use crate::obj::objmap;
use crate::obj::objmappingproxy;
use crate::obj::objmemory;
use crate::obj::objmodule::{self, PyModule};
use crate::obj::objnamespace::{self, PyNamespace};
use crate::obj::objnone::{self, PyNone, PyNoneRef};
use crate::obj::objlist::PyList;
use crate::obj::objmodule::PyModule;
use crate::obj::objnamespace::PyNamespace;
use crate::obj::objnone::{PyNone, PyNoneRef};
use crate::obj::objobject;
use crate::obj::objproperty;
use crate::obj::objproperty::PropertyBuilder;
use crate::obj::objrange;
use crate::obj::objset::{self, PySet};
use crate::obj::objslice;
use crate::obj::objstaticmethod;
use crate::obj::objset::PySet;
use crate::obj::objstr;
use crate::obj::objsuper;
use crate::obj::objtuple::{self, PyTuple, PyTupleRef};
use crate::obj::objtuple::{PyTuple, PyTupleRef};
use crate::obj::objtype::{self, PyClass, PyClassRef};
use crate::obj::objweakproxy;
use crate::obj::objweakref;
use crate::obj::objzip;
use crate::scope::Scope;
use crate::types::{create_type, initialize_types, TypeZoo};
use crate::vm::VirtualMachine;
use indexmap::IndexMap;
@@ -112,71 +93,16 @@ impl fmt::Display for PyObject<dyn PyObjectPayload> {
#[derive(Debug)]
pub struct PyContext {
pub bytes_type: PyClassRef,
pub bytesiterator_type: PyClassRef,
pub bytearray_type: PyClassRef,
pub bytearrayiterator_type: PyClassRef,
pub bool_type: PyClassRef,
pub classmethod_type: PyClassRef,
pub code_type: PyClassRef,
pub dict_type: PyClassRef,
pub ellipsis_type: PyClassRef,
pub enumerate_type: PyClassRef,
pub filter_type: PyClassRef,
pub float_type: PyClassRef,
pub frame_type: PyClassRef,
pub frozenset_type: PyClassRef,
pub generator_type: PyClassRef,
pub int_type: PyClassRef,
pub iter_type: PyClassRef,
pub complex_type: PyClassRef,
pub true_value: PyIntRef,
pub false_value: PyIntRef,
pub list_type: PyClassRef,
pub listiterator_type: PyClassRef,
pub listreverseiterator_type: PyClassRef,
pub striterator_type: PyClassRef,
pub strreverseiterator_type: PyClassRef,
pub dictkeyiterator_type: PyClassRef,
pub dictvalueiterator_type: PyClassRef,
pub dictitemiterator_type: PyClassRef,
pub dictkeys_type: PyClassRef,
pub dictvalues_type: PyClassRef,
pub dictitems_type: PyClassRef,
pub map_type: PyClassRef,
pub memoryview_type: PyClassRef,
pub none: PyNoneRef,
pub empty_tuple: PyTupleRef,
pub ellipsis_type: PyClassRef,
pub ellipsis: PyEllipsisRef,
pub not_implemented: PyNotImplementedRef,
pub empty_tuple: PyTupleRef,
pub tuple_type: PyClassRef,
pub tupleiterator_type: PyClassRef,
pub set_type: PyClassRef,
pub staticmethod_type: PyClassRef,
pub super_type: PyClassRef,
pub str_type: PyClassRef,
pub range_type: PyClassRef,
pub rangeiterator_type: PyClassRef,
pub slice_type: PyClassRef,
pub type_type: PyClassRef,
pub zip_type: PyClassRef,
pub function_type: PyClassRef,
pub builtin_function_or_method_type: PyClassRef,
pub property_type: PyClassRef,
pub readonly_property_type: PyClassRef,
pub module_type: PyClassRef,
pub namespace_type: PyClassRef,
pub bound_method_type: PyClassRef,
pub weakref_type: PyClassRef,
pub weakproxy_type: PyClassRef,
pub mappingproxy_type: PyClassRef,
pub object: PyClassRef,
pub exceptions: exceptions::ExceptionZoo,
}
pub fn create_type(name: &str, type_type: &PyClassRef, base: &PyClassRef) -> PyClassRef {
let dict = PyAttributes::new();
objtype::new(type_type.clone(), name, vec![base.clone()], dict).unwrap()
pub types: TypeZoo,
pub exceptions: exceptions::ExceptionZoo,
}
pub type PyNotImplementedRef = PyRef<PyNotImplemented>;
@@ -201,115 +127,12 @@ impl PyValue for PyEllipsis {
}
}
fn init_type_hierarchy() -> (PyClassRef, PyClassRef) {
// `type` inherits from `object`
// and both `type` and `object are instances of `type`.
// to produce this circular dependency, we need an unsafe block.
// (and yes, this will never get dropped. TODO?)
let (type_type, object_type) = unsafe {
let object_type = PyObject {
typ: mem::uninitialized(), // !
dict: None,
payload: PyClass {
name: String::from("object"),
mro: vec![],
subclasses: RefCell::new(vec![]),
attributes: RefCell::new(PyAttributes::new()),
},
}
.into_ref();
let type_type = PyObject {
typ: mem::uninitialized(), // !
dict: None,
payload: PyClass {
name: String::from("type"),
mro: vec![object_type.clone().downcast().unwrap()],
subclasses: RefCell::new(vec![]),
attributes: RefCell::new(PyAttributes::new()),
},
}
.into_ref();
let object_type_ptr = PyObjectRef::into_raw(object_type.clone()) as *mut PyObject<PyClass>;
let type_type_ptr = PyObjectRef::into_raw(type_type.clone()) as *mut PyObject<PyClass>;
let type_type: PyClassRef = type_type.downcast().unwrap();
let object_type: PyClassRef = object_type.downcast().unwrap();
ptr::write(&mut (*object_type_ptr).typ, type_type.clone());
ptr::write(&mut (*type_type_ptr).typ, type_type.clone());
(type_type, object_type)
};
object_type
.subclasses
.borrow_mut()
.push(objweakref::PyWeak::downgrade(&type_type.as_object()));
(type_type, object_type)
}
// Basic objects:
impl PyContext {
pub fn new() -> Self {
flame_guard!("init PyContext");
let (type_type, object_type) = init_type_hierarchy();
let dict_type = create_type("dict", &type_type, &object_type);
let module_type = create_type("module", &type_type, &object_type);
let namespace_type = create_type("SimpleNamespace", &type_type, &object_type);
let classmethod_type = create_type("classmethod", &type_type, &object_type);
let staticmethod_type = create_type("staticmethod", &type_type, &object_type);
let function_type = create_type("function", &type_type, &object_type);
let builtin_function_or_method_type =
create_type("builtin_function_or_method", &type_type, &object_type);
let property_type = create_type("property", &type_type, &object_type);
let readonly_property_type = create_type("readonly_property", &type_type, &object_type);
let super_type = create_type("super", &type_type, &object_type);
let weakref_type = create_type("ref", &type_type, &object_type);
let weakproxy_type = create_type("weakproxy", &type_type, &object_type);
let generator_type = create_type("generator", &type_type, &object_type);
let bound_method_type = create_type("method", &type_type, &object_type);
let str_type = create_type("str", &type_type, &object_type);
let list_type = create_type("list", &type_type, &object_type);
let listiterator_type = create_type("list_iterator", &type_type, &object_type);
let listreverseiterator_type =
create_type("list_reverseiterator", &type_type, &object_type);
let striterator_type = create_type("str_iterator", &type_type, &object_type);
let strreverseiterator_type = create_type("str_reverseiterator", &type_type, &object_type);
let dictkeys_type = create_type("dict_keys", &type_type, &object_type);
let dictvalues_type = create_type("dict_values", &type_type, &object_type);
let dictitems_type = create_type("dict_items", &type_type, &object_type);
let dictkeyiterator_type = create_type("dict_keyiterator", &type_type, &object_type);
let dictvalueiterator_type = create_type("dict_valueiterator", &type_type, &object_type);
let dictitemiterator_type = create_type("dict_itemiterator", &type_type, &object_type);
let set_type = create_type("set", &type_type, &object_type);
let frozenset_type = create_type("frozenset", &type_type, &object_type);
let int_type = create_type("int", &type_type, &object_type);
let float_type = create_type("float", &type_type, &object_type);
let frame_type = create_type("frame", &type_type, &object_type);
let complex_type = create_type("complex", &type_type, &object_type);
let bytes_type = create_type("bytes", &type_type, &object_type);
let bytesiterator_type = create_type("bytes_iterator", &type_type, &object_type);
let bytearray_type = create_type("bytearray", &type_type, &object_type);
let bytearrayiterator_type = create_type("bytearray_iterator", &type_type, &object_type);
let tuple_type = create_type("tuple", &type_type, &object_type);
let tupleiterator_type = create_type("tuple_iterator", &type_type, &object_type);
let iter_type = create_type("iter", &type_type, &object_type);
let enumerate_type = create_type("enumerate", &type_type, &object_type);
let filter_type = create_type("filter", &type_type, &object_type);
let map_type = create_type("map", &type_type, &object_type);
let zip_type = create_type("zip", &type_type, &object_type);
let bool_type = create_type("bool", &type_type, &int_type);
let memoryview_type = create_type("memoryview", &type_type, &object_type);
let code_type = create_type("code", &type_type, &object_type);
let range_type = create_type("range", &type_type, &object_type);
let rangeiterator_type = create_type("range_iterator", &type_type, &object_type);
let slice_type = create_type("slice", &type_type, &object_type);
let mappingproxy_type = create_type("mappingproxy", &type_type, &object_type);
let exceptions = exceptions::ExceptionZoo::new(&type_type, &object_type);
let types = TypeZoo::new();
let exceptions = exceptions::ExceptionZoo::new(&types.type_type, &types.object_type);
fn create_object<T: PyObjectPayload>(payload: T, cls: &PyClassRef) -> PyRef<T> {
PyRef {
@@ -318,297 +141,213 @@ impl PyContext {
}
}
let none_type = create_type("NoneType", &type_type, &object_type);
let none_type = create_type("NoneType", &types.type_type, &types.object_type);
let none = create_object(PyNone, &none_type);
let ellipsis_type = create_type("EllipsisType", &type_type, &object_type);
let ellipsis_type = create_type("EllipsisType", &types.type_type, &types.object_type);
let ellipsis = create_object(PyEllipsis, &ellipsis_type);
let not_implemented_type = create_type("NotImplementedType", &type_type, &object_type);
let not_implemented_type =
create_type("NotImplementedType", &types.type_type, &types.object_type);
let not_implemented = create_object(PyNotImplemented, &not_implemented_type);
let true_value = create_object(PyInt::new(BigInt::one()), &bool_type);
let false_value = create_object(PyInt::new(BigInt::zero()), &bool_type);
let true_value = create_object(PyInt::new(BigInt::one()), &types.bool_type);
let false_value = create_object(PyInt::new(BigInt::zero()), &types.bool_type);
let empty_tuple = create_object(PyTuple::from(vec![]), &tuple_type);
let empty_tuple = create_object(PyTuple::from(vec![]), &types.tuple_type);
let context = PyContext {
bool_type,
memoryview_type,
bytearray_type,
bytearrayiterator_type,
bytes_type,
bytesiterator_type,
code_type,
complex_type,
classmethod_type,
int_type,
float_type,
frame_type,
staticmethod_type,
list_type,
listiterator_type,
listreverseiterator_type,
striterator_type,
strreverseiterator_type,
dictkeys_type,
dictvalues_type,
dictitems_type,
dictkeyiterator_type,
dictvalueiterator_type,
dictitemiterator_type,
set_type,
frozenset_type,
true_value,
false_value,
tuple_type,
tupleiterator_type,
iter_type,
ellipsis_type,
enumerate_type,
filter_type,
map_type,
zip_type,
dict_type,
not_implemented,
none,
ellipsis,
not_implemented,
str_type,
range_type,
rangeiterator_type,
slice_type,
object: object_type,
function_type,
builtin_function_or_method_type,
super_type,
mappingproxy_type,
property_type,
readonly_property_type,
generator_type,
module_type,
namespace_type,
bound_method_type,
weakref_type,
weakproxy_type,
type_type,
ellipsis_type,
types,
exceptions,
empty_tuple,
};
objtype::init(&context);
objlist::init(&context);
objset::init(&context);
objtuple::init(&context);
objobject::init(&context);
objdict::init(&context);
objfunction::init(&context);
objstaticmethod::init(&context);
objclassmethod::init(&context);
objgenerator::init(&context);
objint::init(&context);
objfloat::init(&context);
objcomplex::init(&context);
objbytes::init(&context);
objbytearray::init(&context);
objproperty::init(&context);
objmemory::init(&context);
objstr::init(&context);
objrange::init(&context);
objslice::init(&context);
objsuper::init(&context);
objtuple::init(&context);
objiter::init(&context);
objellipsis::init(&context);
objenumerate::init(&context);
objfilter::init(&context);
objmap::init(&context);
objzip::init(&context);
objbool::init(&context);
objcode::init(&context);
objframe::init(&context);
objweakref::init(&context);
objweakproxy::init(&context);
objnone::init(&context);
objmodule::init(&context);
objnamespace::init(&context);
objmappingproxy::init(&context);
initialize_types(&context);
exceptions::init(&context);
context
}
pub fn bytearray_type(&self) -> PyClassRef {
self.bytearray_type.clone()
self.types.bytearray_type.clone()
}
pub fn bytearrayiterator_type(&self) -> PyClassRef {
self.bytearrayiterator_type.clone()
self.types.bytearrayiterator_type.clone()
}
pub fn bytes_type(&self) -> PyClassRef {
self.bytes_type.clone()
self.types.bytes_type.clone()
}
pub fn bytesiterator_type(&self) -> PyClassRef {
self.bytesiterator_type.clone()
self.types.bytesiterator_type.clone()
}
pub fn code_type(&self) -> PyClassRef {
self.code_type.clone()
self.types.code_type.clone()
}
pub fn complex_type(&self) -> PyClassRef {
self.complex_type.clone()
self.types.complex_type.clone()
}
pub fn dict_type(&self) -> PyClassRef {
self.dict_type.clone()
self.types.dict_type.clone()
}
pub fn float_type(&self) -> PyClassRef {
self.float_type.clone()
self.types.float_type.clone()
}
pub fn frame_type(&self) -> PyClassRef {
self.frame_type.clone()
self.types.frame_type.clone()
}
pub fn int_type(&self) -> PyClassRef {
self.int_type.clone()
self.types.int_type.clone()
}
pub fn list_type(&self) -> PyClassRef {
self.list_type.clone()
self.types.list_type.clone()
}
pub fn listiterator_type(&self) -> PyClassRef {
self.listiterator_type.clone()
self.types.listiterator_type.clone()
}
pub fn listreverseiterator_type(&self) -> PyClassRef {
self.listreverseiterator_type.clone()
self.types.listreverseiterator_type.clone()
}
pub fn striterator_type(&self) -> PyClassRef {
self.striterator_type.clone()
self.types.striterator_type.clone()
}
pub fn strreverseiterator_type(&self) -> PyClassRef {
self.strreverseiterator_type.clone()
self.types.strreverseiterator_type.clone()
}
pub fn module_type(&self) -> PyClassRef {
self.module_type.clone()
self.types.module_type.clone()
}
pub fn namespace_type(&self) -> PyClassRef {
self.namespace_type.clone()
self.types.namespace_type.clone()
}
pub fn set_type(&self) -> PyClassRef {
self.set_type.clone()
self.types.set_type.clone()
}
pub fn range_type(&self) -> PyClassRef {
self.range_type.clone()
self.types.range_type.clone()
}
pub fn rangeiterator_type(&self) -> PyClassRef {
self.rangeiterator_type.clone()
self.types.rangeiterator_type.clone()
}
pub fn slice_type(&self) -> PyClassRef {
self.slice_type.clone()
self.types.slice_type.clone()
}
pub fn frozenset_type(&self) -> PyClassRef {
self.frozenset_type.clone()
self.types.frozenset_type.clone()
}
pub fn bool_type(&self) -> PyClassRef {
self.bool_type.clone()
self.types.bool_type.clone()
}
pub fn memoryview_type(&self) -> PyClassRef {
self.memoryview_type.clone()
self.types.memoryview_type.clone()
}
pub fn tuple_type(&self) -> PyClassRef {
self.tuple_type.clone()
self.types.tuple_type.clone()
}
pub fn tupleiterator_type(&self) -> PyClassRef {
self.tupleiterator_type.clone()
self.types.tupleiterator_type.clone()
}
pub fn iter_type(&self) -> PyClassRef {
self.iter_type.clone()
self.types.iter_type.clone()
}
pub fn enumerate_type(&self) -> PyClassRef {
self.enumerate_type.clone()
self.types.enumerate_type.clone()
}
pub fn filter_type(&self) -> PyClassRef {
self.filter_type.clone()
self.types.filter_type.clone()
}
pub fn map_type(&self) -> PyClassRef {
self.map_type.clone()
self.types.map_type.clone()
}
pub fn zip_type(&self) -> PyClassRef {
self.zip_type.clone()
self.types.zip_type.clone()
}
pub fn str_type(&self) -> PyClassRef {
self.str_type.clone()
self.types.str_type.clone()
}
pub fn super_type(&self) -> PyClassRef {
self.super_type.clone()
self.types.super_type.clone()
}
pub fn function_type(&self) -> PyClassRef {
self.function_type.clone()
self.types.function_type.clone()
}
pub fn builtin_function_or_method_type(&self) -> PyClassRef {
self.builtin_function_or_method_type.clone()
self.types.builtin_function_or_method_type.clone()
}
pub fn property_type(&self) -> PyClassRef {
self.property_type.clone()
self.types.property_type.clone()
}
pub fn readonly_property_type(&self) -> PyClassRef {
self.readonly_property_type.clone()
self.types.readonly_property_type.clone()
}
pub fn classmethod_type(&self) -> PyClassRef {
self.classmethod_type.clone()
self.types.classmethod_type.clone()
}
pub fn staticmethod_type(&self) -> PyClassRef {
self.staticmethod_type.clone()
self.types.staticmethod_type.clone()
}
pub fn generator_type(&self) -> PyClassRef {
self.generator_type.clone()
self.types.generator_type.clone()
}
pub fn bound_method_type(&self) -> PyClassRef {
self.bound_method_type.clone()
self.types.bound_method_type.clone()
}
pub fn weakref_type(&self) -> PyClassRef {
self.weakref_type.clone()
self.types.weakref_type.clone()
}
pub fn weakproxy_type(&self) -> PyClassRef {
self.weakproxy_type.clone()
self.types.weakproxy_type.clone()
}
pub fn type_type(&self) -> PyClassRef {
self.type_type.clone()
self.types.type_type.clone()
}
pub fn none(&self) -> PyObjectRef {
@@ -624,7 +363,7 @@ impl PyContext {
}
pub fn object(&self) -> PyClassRef {
self.object.clone()
self.types.object_type.clone()
}
pub fn new_int<T: Into<BigInt>>(&self, i: T) -> PyObjectRef {
@@ -691,16 +430,6 @@ impl PyContext {
objtype::new(self.type_type(), name, vec![base], PyAttributes::new()).unwrap()
}
pub fn new_module(&self, name: &str, dict: PyDictRef) -> PyObjectRef {
PyObject::new(
PyModule {
name: name.to_string(),
},
self.module_type.clone(),
Some(dict),
)
}
pub fn new_namespace(&self) -> PyObjectRef {
PyObject::new(PyNamespace, self.namespace_type(), Some(self.new_dict()))
}
@@ -948,7 +677,7 @@ pub struct PyCallable {
impl PyCallable {
#[inline]
pub fn invoke(&self, args: impl Into<PyFuncArgs>, vm: &VirtualMachine) -> PyResult {
vm.invoke(self.obj.clone(), args)
vm.invoke(&self.obj, args)
}
#[inline]
@@ -1103,8 +832,9 @@ impl<T> PyIterable<T> {
/// This operation may fail if an exception is raised while invoking the
/// `__iter__` method of the iterable object.
pub fn iter<'a>(&self, vm: &'a VirtualMachine) -> PyResult<PyIterator<'a, T>> {
let method = &self.method;
let iter_obj = vm.invoke(
self.method.clone(),
method,
PyFuncArgs {
args: vec![],
kwargs: IndexMap::new(),

92
vm/src/stdlib/errno.rs
View File

@@ -149,6 +149,96 @@ const ERROR_CODES: &[(&str, i32)] = &[
("ERFKILL", libc::ERFKILL),
];
#[cfg(all(unix, not(target_os = "linux")))]
const ERROR_CODES: &[(&str, i32)] = &[
("ENODEV", libc::ENODEV),
("EHOSTUNREACH", libc::EHOSTUNREACH),
("ENOMSG", libc::ENOMSG),
("ENODATA", libc::ENODATA),
("ENOTBLK", libc::ENOTBLK),
("ENOSYS", libc::ENOSYS),
("EPIPE", libc::EPIPE),
("EINVAL", libc::EINVAL),
("EOVERFLOW", libc::EOVERFLOW),
("EINTR", libc::EINTR),
("EUSERS", libc::EUSERS),
("ENOTEMPTY", libc::ENOTEMPTY),
("ENOBUFS", libc::ENOBUFS),
("EPROTO", libc::EPROTO),
("EREMOTE", libc::EREMOTE),
("ECHILD", libc::ECHILD),
("ELOOP", libc::ELOOP),
("EXDEV", libc::EXDEV),
("E2BIG", libc::E2BIG),
("ESRCH", libc::ESRCH),
("EMSGSIZE", libc::EMSGSIZE),
("EAFNOSUPPORT", libc::EAFNOSUPPORT),
("EHOSTDOWN", libc::EHOSTDOWN),
("EPFNOSUPPORT", libc::EPFNOSUPPORT),
("ENOPROTOOPT", libc::ENOPROTOOPT),
("EBUSY", libc::EBUSY),
("EAGAIN", libc::EAGAIN),
("EISCONN", libc::EISCONN),
("ESHUTDOWN", libc::ESHUTDOWN),
("EBADF", libc::EBADF),
("EMULTIHOP", libc::EMULTIHOP),
("EIO", libc::EIO),
("EPROTOTYPE", libc::EPROTOTYPE),
("ENOSPC", libc::ENOSPC),
("ENOEXEC", libc::ENOEXEC),
("EALREADY", libc::EALREADY),
("ENETDOWN", libc::ENETDOWN),
("EACCES", libc::EACCES),
("EILSEQ", libc::EILSEQ),
("ENOTDIR", libc::ENOTDIR),
("EPERM", libc::EPERM),
("EDOM", libc::EDOM),
("ECONNREFUSED", libc::ECONNREFUSED),
("EISDIR", libc::EISDIR),
("EPROTONOSUPPORT", libc::EPROTONOSUPPORT),
("EROFS", libc::EROFS),
("EADDRNOTAVAIL", libc::EADDRNOTAVAIL),
("EIDRM", libc::EIDRM),
("EBADMSG", libc::EBADMSG),
("ENFILE", libc::ENFILE),
("ESPIPE", libc::ESPIPE),
("ENOLINK", libc::ENOLINK),
("ENETRESET", libc::ENETRESET),
("ETIMEDOUT", libc::ETIMEDOUT),
("ENOENT", libc::ENOENT),
("EEXIST", libc::EEXIST),
("EDQUOT", libc::EDQUOT),
("ENOSTR", libc::ENOSTR),
("EFAULT", libc::EFAULT),
("EFBIG", libc::EFBIG),
("ENOTCONN", libc::ENOTCONN),
("EDESTADDRREQ", libc::EDESTADDRREQ),
("ENOLCK", libc::ENOLCK),
("ECONNABORTED", libc::ECONNABORTED),
("ENETUNREACH", libc::ENETUNREACH),
("ESTALE", libc::ESTALE),
("ENOSR", libc::ENOSR),
("ENOMEM", libc::ENOMEM),
("ENOTSOCK", libc::ENOTSOCK),
("EMLINK", libc::EMLINK),
("ERANGE", libc::ERANGE),
("ECONNRESET", libc::ECONNRESET),
("EADDRINUSE", libc::EADDRINUSE),
("ENOTSUP", libc::ENOTSUP),
("ENAMETOOLONG", libc::ENAMETOOLONG),
("ENOTTY", libc::ENOTTY),
("ESOCKTNOSUPPORT", libc::ESOCKTNOSUPPORT),
("ETIME", libc::ETIME),
("ETOOMANYREFS", libc::ETOOMANYREFS),
("EMFILE", libc::EMFILE),
("ETXTBSY", libc::ETXTBSY),
("EINPROGRESS", libc::EINPROGRESS),
("ENXIO", libc::ENXIO),
("ECANCELED", libc::ECANCELED),
("EOWNERDEAD", libc::EOWNERDEAD),
("ENOTRECOVERABLE", libc::ENOTRECOVERABLE),
];
#[cfg(windows)]
const ERROR_CODES: &[(&str, i32)] = &[
("ENODEV", 19),
@@ -254,5 +344,5 @@ const ERROR_CODES: &[(&str, i32)] = &[
("ENOTSUP", 129),
];
#[cfg(not(any(target_os = "linux", windows)))]
#[cfg(not(any(unix, windows)))]
const ERROR_CODES: &[(&str, i32)] = &[];

46
vm/src/stdlib/functools.rs Normal file
View File

@@ -0,0 +1,46 @@
use crate::function::OptionalArg;
use crate::obj::objiter;
use crate::obj::objtype;
use crate::pyobject::{PyObjectRef, PyResult};
use crate::vm::VirtualMachine;
pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
let ctx = &vm.ctx;
py_module!(vm, "_functools", {
"reduce" => ctx.new_rustfunc(functools_reduce),
})
}
fn functools_reduce(
function: PyObjectRef,
sequence: PyObjectRef,
start_value: OptionalArg<PyObjectRef>,
vm: &VirtualMachine,
) -> PyResult {
let iterator = objiter::get_iter(vm, &sequence)?;
let start_value = if let OptionalArg::Present(val) = start_value {
val
} else {
objiter::call_next(vm, &iterator).map_err(|err| {
if objtype::isinstance(&err, &vm.ctx.exceptions.stop_iteration) {
let exc_type = vm.ctx.exceptions.type_error.clone();
vm.new_exception(
exc_type,
"reduce() of empty sequence with no initial value".to_string(),
)
} else {
err
}
})?
};
let mut accumulator = start_value;
while let Ok(next_obj) = objiter::call_next(vm, &iterator) {
accumulator = vm.invoke(&function, vec![accumulator, next_obj])?
}
Ok(accumulator)
}

34
vm/src/stdlib/io.rs
View File

@@ -431,6 +431,31 @@ fn file_io_write(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
}
}
#[cfg(windows)]
fn file_io_close(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
use std::os::windows::io::IntoRawHandle;
arg_check!(vm, args, required = [(file_io, None)]);
let file_no = vm.get_attribute(file_io.clone(), "fileno")?;
let raw_fd = objint::get_value(&file_no).to_i64().unwrap();
let handle = os::rust_file(raw_fd);
let raw_handle = handle.into_raw_handle();
unsafe {
kernel32::CloseHandle(raw_handle);
}
Ok(vm.ctx.none())
}
#[cfg(unix)]
fn file_io_close(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
arg_check!(vm, args, required = [(file_io, None)]);
let file_no = vm.get_attribute(file_io.clone(), "fileno")?;
let raw_fd = objint::get_value(&file_no).to_i32().unwrap();
unsafe {
libc::close(raw_fd);
}
Ok(vm.ctx.none())
}
fn file_io_seekable(vm: &VirtualMachine, _args: PyFuncArgs) -> PyResult {
Ok(vm.ctx.new_bool(true))
}
@@ -607,7 +632,7 @@ pub fn io_open(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
// This is subsequently consumed by a Buffered Class.
let file_io_class = vm.get_attribute(io_module.clone(), "FileIO").unwrap();
let file_io_obj = vm.invoke(
file_io_class,
&file_io_class,
vec![file.clone(), vm.ctx.new_str(mode.clone())],
)?;
@@ -620,13 +645,13 @@ pub fn io_open(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
let buffered_writer_class = vm
.get_attribute(io_module.clone(), "BufferedWriter")
.unwrap();
vm.invoke(buffered_writer_class, vec![file_io_obj.clone()])
vm.invoke(&buffered_writer_class, vec![file_io_obj.clone()])
}
'r' => {
let buffered_reader_class = vm
.get_attribute(io_module.clone(), "BufferedReader")
.unwrap();
vm.invoke(buffered_reader_class, vec![file_io_obj.clone()])
vm.invoke(&buffered_reader_class, vec![file_io_obj.clone()])
}
//TODO: updating => PyBufferedRandom
_ => unimplemented!("'a' mode is not yet implemented"),
@@ -637,7 +662,7 @@ pub fn io_open(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
// a TextIOWrapper which is subsequently returned.
't' => {
let text_io_wrapper_class = vm.get_attribute(io_module, "TextIOWrapper").unwrap();
vm.invoke(text_io_wrapper_class, vec![buffered.unwrap()])
vm.invoke(&text_io_wrapper_class, vec![buffered.unwrap()])
}
// If the mode is binary this Buffered class is returned directly at
// this point.
@@ -678,6 +703,7 @@ pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
"read" => ctx.new_rustfunc(file_io_read),
"readinto" => ctx.new_rustfunc(file_io_readinto),
"write" => ctx.new_rustfunc(file_io_write),
"close" => ctx.new_rustfunc(file_io_close),
"seekable" => ctx.new_rustfunc(file_io_seekable)
});

6
vm/src/stdlib/itertools.rs
View File

@@ -218,8 +218,9 @@ impl PyItertoolsStarmap {
#[pymethod(name = "__next__")]
fn next(&self, vm: &VirtualMachine) -> PyResult {
let obj = call_next(vm, &self.iter)?;
let function = &self.function;
vm.invoke(self.function.clone(), vm.extract_elements(&obj)?)
vm.invoke(function, vm.extract_elements(&obj)?)
}
#[pymethod(name = "__iter__")]
@@ -271,8 +272,9 @@ impl PyItertoolsTakewhile {
// might be StopIteration or anything else, which is propaged upwwards
let obj = call_next(vm, &self.iterable)?;
let predicate = &self.predicate;
let verdict = vm.invoke(self.predicate.clone(), vec![obj.clone()])?;
let verdict = vm.invoke(predicate, vec![obj.clone()])?;
let verdict = objbool::boolval(vm, verdict)?;
if verdict {
Ok(obj)

18
vm/src/stdlib/json.rs
View File

@@ -1,6 +1,7 @@
use crate::obj::objstr::PyStringRef;
use crate::py_serde;
use crate::pyobject::{create_type, ItemProtocol, PyObjectRef, PyResult};
use crate::pyobject::{ItemProtocol, PyObjectRef, PyResult};
use crate::types::create_type;
use crate::VirtualMachine;
use serde_json;
@@ -10,6 +11,12 @@ pub fn json_dumps(obj: PyObjectRef, vm: &VirtualMachine) -> PyResult<String> {
serde_json::to_string(&serializer).map_err(|err| vm.new_type_error(err.to_string()))
}
pub fn json_dump(obj: PyObjectRef, fs: PyObjectRef, vm: &VirtualMachine) -> PyResult {
let result = json_dumps(obj, vm)?;
vm.call_method(&fs, "write", vec![vm.new_str(result)])?;
Ok(vm.get_none())
}
/// Implement json.loads
pub fn json_loads(string: PyStringRef, vm: &VirtualMachine) -> PyResult {
// TODO: Implement non-trivial deserialization case
@@ -33,19 +40,26 @@ pub fn json_loads(string: PyStringRef, vm: &VirtualMachine) -> PyResult {
})
}
pub fn json_load(fp: PyObjectRef, vm: &VirtualMachine) -> PyResult {
let result = vm.call_method(&fp, "read", vec![])?;
json_loads(result.downcast()?, vm)
}
pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
let ctx = &vm.ctx;
// TODO: Make this a proper type with a constructor
let json_decode_error = create_type(
"JSONDecodeError",
&ctx.type_type,
&ctx.types.type_type,
&ctx.exceptions.exception_type,
);
py_module!(vm, "json", {
"dumps" => ctx.new_rustfunc(json_dumps),
"dump" => ctx.new_rustfunc(json_dump),
"loads" => ctx.new_rustfunc(json_loads),
"load" => ctx.new_rustfunc(json_load),
"JSONDecodeError" => json_decode_error
})
}

16
vm/src/stdlib/marshal.rs
View File

@@ -1,17 +1,17 @@
use crate::bytecode;
use crate::obj::objbytes::{PyBytes, PyBytesRef};
use crate::obj::objcode::{PyCode, PyCodeRef};
use crate::pyobject::{IntoPyObject, PyObjectRef, PyResult};
use crate::pyobject::{PyObjectRef, PyResult};
use crate::vm::VirtualMachine;
fn marshal_dumps(co: PyCodeRef, vm: &VirtualMachine) -> PyResult {
PyBytes::new(bincode::serialize(&co.code).unwrap()).into_pyobject(vm)
fn marshal_dumps(co: PyCodeRef, _vm: &VirtualMachine) -> PyBytes {
PyBytes::new(bincode::serialize(&co.code).unwrap())
}
fn marshal_loads(code_bytes: PyBytesRef, vm: &VirtualMachine) -> PyResult {
let code = bincode::deserialize::<bytecode::CodeObject>(&code_bytes).unwrap();
let pycode = PyCode { code };
pycode.into_pyobject(vm)
fn marshal_loads(code_bytes: PyBytesRef, vm: &VirtualMachine) -> PyResult<PyCode> {
let code = bincode::deserialize::<bytecode::CodeObject>(&code_bytes)
.map_err(|_| vm.new_value_error("Couldn't deserialize python bytecode".to_owned()))?;
Ok(PyCode { code })
}
pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
@@ -19,6 +19,6 @@ pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
py_module!(vm, "marshal", {
"loads" => ctx.new_rustfunc(marshal_loads),
"dumps" => ctx.new_rustfunc(marshal_dumps)
"dumps" => ctx.new_rustfunc(marshal_dumps),
})
}

6
vm/src/stdlib/math.rs
View File

@@ -181,7 +181,7 @@ fn try_magic_method(func_name: &str, vm: &VirtualMachine, value: &PyObjectRef) -
func_name,
)
})?;
vm.invoke(method, vec![])
vm.invoke(&method, vec![])
}
fn math_trunc(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
@@ -191,7 +191,7 @@ fn math_trunc(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
fn math_ceil(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
arg_check!(vm, args, required = [(value, None)]);
if objtype::isinstance(value, &vm.ctx.float_type) {
if objtype::isinstance(value, &vm.ctx.float_type()) {
let v = objfloat::get_value(value);
Ok(vm.ctx.new_float(v.ceil()))
} else {
@@ -201,7 +201,7 @@ fn math_ceil(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
fn math_floor(vm: &VirtualMachine, args: PyFuncArgs) -> PyResult {
arg_check!(vm, args, required = [(value, None)]);
if objtype::isinstance(value, &vm.ctx.float_type) {
if objtype::isinstance(value, &vm.ctx.float_type()) {
let v = objfloat::get_value(value);
Ok(vm.ctx.new_float(v.floor()))
} else {

2
vm/src/stdlib/mod.rs
View File

@@ -5,6 +5,7 @@ mod codecs;
mod collections;
mod dis;
mod errno;
mod functools;
mod hashlib;
mod imp;
mod itertools;
@@ -57,6 +58,7 @@ pub fn get_module_inits() -> HashMap<String, StdlibInitFunc> {
"dis".to_string() => Box::new(dis::make_module),
"_codecs".to_string() => Box::new(codecs::make_module),
"_collections".to_string() => Box::new(collections::make_module),
"_functools".to_string() => Box::new(functools::make_module),
"errno".to_string() => Box::new(errno::make_module),
"hashlib".to_string() => Box::new(hashlib::make_module),
"itertools".to_string() => Box::new(itertools::make_module),

11
vm/src/stdlib/os.rs
View File

@@ -316,6 +316,15 @@ fn _os_environ(vm: &VirtualMachine) -> PyDictRef {
environ
}
fn os_readlink(path: PyStringRef, dir_fd: DirFd, vm: &VirtualMachine) -> PyResult {
let path = make_path(vm, path, &dir_fd);
let path = fs::read_link(path.as_str()).map_err(|err| convert_io_error(vm, err))?;
let path = path.into_os_string().into_string().map_err(|_osstr| {
vm.new_unicode_decode_error("Can't convert OS path to valid UTF-8 string".into())
})?;
Ok(vm.ctx.new_str(path))
}
#[derive(Debug)]
struct DirEntry {
entry: fs::DirEntry,
@@ -964,7 +973,7 @@ pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
// mkfifo Some Some None
// mknod Some Some None
// pathconf Some None None
// readlink Some Some None
SupportFunc::new(vm, "readlink", os_readlink, Some(false), Some(false), None),
SupportFunc::new(vm, "remove", os_remove, Some(false), Some(false), None),
SupportFunc::new(vm, "rename", os_rename, Some(false), Some(false), None),
SupportFunc::new(vm, "replace", os_rename, Some(false), Some(false), None), // TODO: Fix replace

2
vm/src/stdlib/signal.rs
View File

@@ -92,7 +92,7 @@ pub fn check_signals(vm: &VirtualMachine) {
.get(&(signum as i32))
.expect("Handler should be set")
.clone();
vm.invoke(handler, vec![vm.new_int(signum), vm.get_none()])
vm.invoke(&handler, vec![vm.new_int(signum), vm.get_none()])
.expect("Test");
}
}

13
vm/src/stdlib/socket.rs
View File

@@ -21,6 +21,7 @@ use crate::vm::VirtualMachine;
use crate::obj::objtype::PyClassRef;
#[cfg(unix)]
use crate::stdlib::os::convert_nix_error;
use num_bigint::Sign;
use num_traits::ToPrimitive;
#[derive(Debug, Copy, Clone)]
@@ -409,6 +410,17 @@ fn socket_inet_ntoa(packed_ip: PyBytesRef, vm: &VirtualMachine) -> PyResult {
Ok(vm.new_str(Ipv4Addr::from(ip_num).to_string()))
}
fn socket_htonl(host: PyIntRef, vm: &VirtualMachine) -> PyResult {
if host.as_bigint().sign() == Sign::Minus {
return Err(
vm.new_overflow_error("can't convert negative value to unsigned int".to_string())
);
}
let host = host.as_bigint().to_u32().unwrap();
Ok(vm.new_int(host.to_be()))
}
pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
let ctx = &vm.ctx;
@@ -435,6 +447,7 @@ pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
"inet_aton" => ctx.new_rustfunc(socket_inet_aton),
"inet_ntoa" => ctx.new_rustfunc(socket_inet_ntoa),
"gethostname" => ctx.new_rustfunc(socket_gethostname),
"htonl" => ctx.new_rustfunc(socket_htonl),
});
extend_module_platform_specific(vm, module)

2
vm/src/stdlib/thread.rs
View File

@@ -37,7 +37,7 @@ fn get_ident(_vm: &VirtualMachine) -> u32 {
fn allocate_lock(vm: &VirtualMachine) -> PyResult {
let lock_class = vm.class("_thread", "RLock");
vm.invoke(lock_class.into_object(), vec![])
vm.invoke(&lock_class.into_object(), vec![])
}
pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {

9
vm/src/stdlib/zlib.rs
View File

@@ -1,6 +1,7 @@
use crate::function::OptionalArg;
use crate::obj::{objbytes::PyBytesRef, objint::PyIntRef};
use crate::pyobject::{create_type, ItemProtocol, PyObjectRef, PyResult};
use crate::pyobject::{ItemProtocol, PyObjectRef, PyResult};
use crate::types::create_type;
use crate::vm::VirtualMachine;
use adler32::RollingAdler32 as Adler32;
@@ -18,7 +19,11 @@ const DEF_BUF_SIZE: usize = 16 * 1024;
pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
let ctx = &vm.ctx;
let zlib_error = create_type("error", &ctx.type_type, &ctx.exceptions.exception_type);
let zlib_error = create_type(
"error",
&ctx.types.type_type,
&ctx.exceptions.exception_type,
);
py_module!(vm, "zlib", {
"crc32" => ctx.new_rustfunc(zlib_crc32),

313
vm/src/types.rs Normal file
View File

@@ -0,0 +1,313 @@
use crate::obj::objbool;
use crate::obj::objbytearray;
use crate::obj::objbytes;
use crate::obj::objclassmethod;
use crate::obj::objcode;
use crate::obj::objcomplex;
use crate::obj::objdict;
use crate::obj::objellipsis;
use crate::obj::objenumerate;
use crate::obj::objfilter;
use crate::obj::objfloat;
use crate::obj::objframe;
use crate::obj::objfunction;
use crate::obj::objgenerator;
use crate::obj::objint;
use crate::obj::objiter;
use crate::obj::objlist;
use crate::obj::objmap;
use crate::obj::objmappingproxy;
use crate::obj::objmemory;
use crate::obj::objmodule;
use crate::obj::objnamespace;
use crate::obj::objnone;
use crate::obj::objobject;
use crate::obj::objproperty;
use crate::obj::objrange;
use crate::obj::objset;
use crate::obj::objslice;
use crate::obj::objstaticmethod;
use crate::obj::objstr;
use crate::obj::objsuper;
use crate::obj::objtuple;
use crate::obj::objtype::{self, PyClass, PyClassRef};
use crate::obj::objweakproxy;
use crate::obj::objweakref;
use crate::obj::objzip;
use crate::pyobject::{PyAttributes, PyContext, PyObject, PyObjectRef};
use std::cell::RefCell;
use std::mem;
use std::ptr;
/// Holder of references to builtin types.
#[derive(Debug)]
pub struct TypeZoo {
pub bytes_type: PyClassRef,
pub bytesiterator_type: PyClassRef,
pub bytearray_type: PyClassRef,
pub bytearrayiterator_type: PyClassRef,
pub bool_type: PyClassRef,
pub classmethod_type: PyClassRef,
pub code_type: PyClassRef,
pub dict_type: PyClassRef,
pub enumerate_type: PyClassRef,
pub filter_type: PyClassRef,
pub float_type: PyClassRef,
pub frame_type: PyClassRef,
pub frozenset_type: PyClassRef,
pub generator_type: PyClassRef,
pub int_type: PyClassRef,
pub iter_type: PyClassRef,
pub complex_type: PyClassRef,
pub list_type: PyClassRef,
pub listiterator_type: PyClassRef,
pub listreverseiterator_type: PyClassRef,
pub striterator_type: PyClassRef,
pub strreverseiterator_type: PyClassRef,
pub dictkeyiterator_type: PyClassRef,
pub dictvalueiterator_type: PyClassRef,
pub dictitemiterator_type: PyClassRef,
pub dictkeys_type: PyClassRef,
pub dictvalues_type: PyClassRef,
pub dictitems_type: PyClassRef,
pub map_type: PyClassRef,
pub memoryview_type: PyClassRef,
pub tuple_type: PyClassRef,
pub tupleiterator_type: PyClassRef,
pub set_type: PyClassRef,
pub staticmethod_type: PyClassRef,
pub super_type: PyClassRef,
pub str_type: PyClassRef,
pub range_type: PyClassRef,
pub rangeiterator_type: PyClassRef,
pub slice_type: PyClassRef,
pub type_type: PyClassRef,
pub zip_type: PyClassRef,
pub function_type: PyClassRef,
pub builtin_function_or_method_type: PyClassRef,
pub property_type: PyClassRef,
pub readonly_property_type: PyClassRef,
pub module_type: PyClassRef,
pub namespace_type: PyClassRef,
pub bound_method_type: PyClassRef,
pub weakref_type: PyClassRef,
pub weakproxy_type: PyClassRef,
pub mappingproxy_type: PyClassRef,
pub object_type: PyClassRef,
}
impl Default for TypeZoo {
fn default() -> Self {
Self::new()
}
}
impl TypeZoo {
pub fn new() -> Self {
let (type_type, object_type) = init_type_hierarchy();
let dict_type = create_type("dict", &type_type, &object_type);
let module_type = create_type("module", &type_type, &object_type);
let namespace_type = create_type("SimpleNamespace", &type_type, &object_type);
let classmethod_type = create_type("classmethod", &type_type, &object_type);
let staticmethod_type = create_type("staticmethod", &type_type, &object_type);
let function_type = create_type("function", &type_type, &object_type);
let builtin_function_or_method_type =
create_type("builtin_function_or_method", &type_type, &object_type);
let property_type = create_type("property", &type_type, &object_type);
let readonly_property_type = create_type("readonly_property", &type_type, &object_type);
let super_type = create_type("super", &type_type, &object_type);
let weakref_type = create_type("ref", &type_type, &object_type);
let weakproxy_type = create_type("weakproxy", &type_type, &object_type);
let generator_type = create_type("generator", &type_type, &object_type);
let bound_method_type = create_type("method", &type_type, &object_type);
let str_type = create_type("str", &type_type, &object_type);
let list_type = create_type("list", &type_type, &object_type);
let listiterator_type = create_type("list_iterator", &type_type, &object_type);
let listreverseiterator_type =
create_type("list_reverseiterator", &type_type, &object_type);
let striterator_type = create_type("str_iterator", &type_type, &object_type);
let strreverseiterator_type = create_type("str_reverseiterator", &type_type, &object_type);
let dictkeys_type = create_type("dict_keys", &type_type, &object_type);
let dictvalues_type = create_type("dict_values", &type_type, &object_type);
let dictitems_type = create_type("dict_items", &type_type, &object_type);
let dictkeyiterator_type = create_type("dict_keyiterator", &type_type, &object_type);
let dictvalueiterator_type = create_type("dict_valueiterator", &type_type, &object_type);
let dictitemiterator_type = create_type("dict_itemiterator", &type_type, &object_type);
let set_type = create_type("set", &type_type, &object_type);
let frozenset_type = create_type("frozenset", &type_type, &object_type);
let int_type = create_type("int", &type_type, &object_type);
let float_type = create_type("float", &type_type, &object_type);
let frame_type = create_type("frame", &type_type, &object_type);
let complex_type = create_type("complex", &type_type, &object_type);
let bytes_type = create_type("bytes", &type_type, &object_type);
let bytesiterator_type = create_type("bytes_iterator", &type_type, &object_type);
let bytearray_type = create_type("bytearray", &type_type, &object_type);
let bytearrayiterator_type = create_type("bytearray_iterator", &type_type, &object_type);
let tuple_type = create_type("tuple", &type_type, &object_type);
let tupleiterator_type = create_type("tuple_iterator", &type_type, &object_type);
let iter_type = create_type("iter", &type_type, &object_type);
let enumerate_type = create_type("enumerate", &type_type, &object_type);
let filter_type = create_type("filter", &type_type, &object_type);
let map_type = create_type("map", &type_type, &object_type);
let zip_type = create_type("zip", &type_type, &object_type);
let bool_type = create_type("bool", &type_type, &int_type);
let memoryview_type = create_type("memoryview", &type_type, &object_type);
let code_type = create_type("code", &type_type, &object_type);
let range_type = create_type("range", &type_type, &object_type);
let rangeiterator_type = create_type("range_iterator", &type_type, &object_type);
let slice_type = create_type("slice", &type_type, &object_type);
let mappingproxy_type = create_type("mappingproxy", &type_type, &object_type);
Self {
bool_type,
memoryview_type,
bytearray_type,
bytearrayiterator_type,
bytes_type,
bytesiterator_type,
code_type,
complex_type,
classmethod_type,
int_type,
float_type,
frame_type,
staticmethod_type,
list_type,
listiterator_type,
listreverseiterator_type,
striterator_type,
strreverseiterator_type,
dictkeys_type,
dictvalues_type,
dictitems_type,
dictkeyiterator_type,
dictvalueiterator_type,
dictitemiterator_type,
set_type,
frozenset_type,
tuple_type,
tupleiterator_type,
iter_type,
enumerate_type,
filter_type,
map_type,
zip_type,
dict_type,
str_type,
range_type,
rangeiterator_type,
slice_type,
object_type,
function_type,
builtin_function_or_method_type,
super_type,
mappingproxy_type,
property_type,
readonly_property_type,
generator_type,
module_type,
namespace_type,
bound_method_type,
weakref_type,
weakproxy_type,
type_type,
}
}
}
pub fn create_type(name: &str, type_type: &PyClassRef, base: &PyClassRef) -> PyClassRef {
let dict = PyAttributes::new();
objtype::new(type_type.clone(), name, vec![base.clone()], dict).unwrap()
}
fn init_type_hierarchy() -> (PyClassRef, PyClassRef) {
// `type` inherits from `object`
// and both `type` and `object are instances of `type`.
// to produce this circular dependency, we need an unsafe block.
// (and yes, this will never get dropped. TODO?)
let (type_type, object_type) = unsafe {
let object_type = PyObject {
typ: mem::uninitialized(), // !
dict: None,
payload: PyClass {
name: String::from("object"),
mro: vec![],
subclasses: RefCell::new(vec![]),
attributes: RefCell::new(PyAttributes::new()),
},
}
.into_ref();
let type_type = PyObject {
typ: mem::uninitialized(), // !
dict: None,
payload: PyClass {
name: String::from("type"),
mro: vec![object_type.clone().downcast().unwrap()],
subclasses: RefCell::new(vec![]),
attributes: RefCell::new(PyAttributes::new()),
},
}
.into_ref();
let object_type_ptr = PyObjectRef::into_raw(object_type.clone()) as *mut PyObject<PyClass>;
let type_type_ptr = PyObjectRef::into_raw(type_type.clone()) as *mut PyObject<PyClass>;
let type_type: PyClassRef = type_type.downcast().unwrap();
let object_type: PyClassRef = object_type.downcast().unwrap();
ptr::write(&mut (*object_type_ptr).typ, type_type.clone());
ptr::write(&mut (*type_type_ptr).typ, type_type.clone());
(type_type, object_type)
};
object_type
.subclasses
.borrow_mut()
.push(objweakref::PyWeak::downgrade(&type_type.as_object()));
(type_type, object_type)
}
/// Fill attributes of builtin types.
pub fn initialize_types(context: &PyContext) {
objtype::init(&context);
objlist::init(&context);
objset::init(&context);
objtuple::init(&context);
objobject::init(&context);
objdict::init(&context);
objfunction::init(&context);
objstaticmethod::init(&context);
objclassmethod::init(&context);
objgenerator::init(&context);
objint::init(&context);
objfloat::init(&context);
objcomplex::init(&context);
objbytes::init(&context);
objbytearray::init(&context);
objproperty::init(&context);
objmemory::init(&context);
objstr::init(&context);
objrange::init(&context);
objslice::init(&context);
objsuper::init(&context);
objtuple::init(&context);
objiter::init(&context);
objellipsis::init(&context);
objenumerate::init(&context);
objfilter::init(&context);
objmap::init(&context);
objzip::init(&context);
objbool::init(&context);
objcode::init(&context);
objframe::init(&context);
objweakref::init(&context);
objweakproxy::init(&context);
objnone::init(&context);
objmodule::init(&context);
objnamespace::init(&context);
objmappingproxy::init(&context);
}

139
vm/src/vm.rs
View File

@@ -25,6 +25,7 @@ use crate::obj::objfunction::{PyFunction, PyMethod};
use crate::obj::objgenerator::PyGenerator;
use crate::obj::objint::PyInt;
use crate::obj::objiter;
use crate::obj::objmodule::{self, PyModule};
use crate::obj::objsequence;
use crate::obj::objstr::{PyString, PyStringRef};
use crate::obj::objtuple::PyTupleRef;
@@ -32,8 +33,8 @@ use crate::obj::objtype;
use crate::obj::objtype::PyClassRef;
use crate::pyhash;
use crate::pyobject::{
IdProtocol, ItemProtocol, PyContext, PyObjectRef, PyResult, PyValue, TryFromObject, TryIntoRef,
TypeProtocol,
IdProtocol, ItemProtocol, PyContext, PyObject, PyObjectRef, PyResult, PyValue, TryFromObject,
TryIntoRef, TypeProtocol,
};
use crate::scope::Scope;
use crate::stdlib;
@@ -142,9 +143,23 @@ impl VirtualMachine {
flame_guard!("init VirtualMachine");
let ctx = PyContext::new();
// make a new module without access to the vm; doesn't
// set __spec__, __loader__, etc. attributes
let new_module = |name: &str, dict| {
PyObject::new(
PyModule {
name: name.to_owned(),
},
ctx.types.module_type.clone(),
Some(dict),
)
};
// Hard-core modules:
let builtins = ctx.new_module("builtins", ctx.new_dict());
let sysmod = ctx.new_module("sys", ctx.new_dict());
let builtins_dict = ctx.new_dict();
let builtins = new_module("builtins", builtins_dict.clone());
let sysmod_dict = ctx.new_dict();
let sysmod = new_module("sys", sysmod_dict.clone());
let stdlib_inits = RefCell::new(stdlib::get_module_inits());
let frozen = RefCell::new(frozen::get_module_inits());
@@ -168,6 +183,19 @@ impl VirtualMachine {
signal_handlers: Default::default(),
};
objmodule::init_module_dict(
&vm,
&builtins_dict,
vm.new_str("builtins".to_owned()),
vm.get_none(),
);
objmodule::init_module_dict(
&vm,
&sysmod_dict,
vm.new_str("sys".to_owned()),
vm.get_none(),
);
builtins::make_module(&vm, builtins.clone());
sysmodule::make_module(&vm, sysmod, builtins);
vm
@@ -254,11 +282,22 @@ impl VirtualMachine {
self.ctx.new_bool(b)
}
pub fn new_module(&self, name: &str, dict: PyDictRef) -> PyObjectRef {
objmodule::init_module_dict(self, &dict, self.new_str(name.to_owned()), self.get_none());
PyObject::new(
PyModule {
name: name.to_owned(),
},
self.ctx.types.module_type.clone(),
Some(dict),
)
}
#[cfg_attr(feature = "flame-it", flame("VirtualMachine"))]
fn new_exception_obj(&self, exc_type: PyClassRef, args: Vec<PyObjectRef>) -> PyResult {
// TODO: add repr of args into logging?
vm_trace!("New exception created: {}", exc_type.name);
self.invoke(exc_type.into_object(), args)
self.invoke(&exc_type.into_object(), args)
}
pub fn new_empty_exception(&self, exc_type: PyClassRef) -> PyResult {
@@ -407,10 +446,23 @@ impl VirtualMachine {
}
pub fn import(&self, module: &str, from_list: &PyObjectRef, level: usize) -> PyResult {
let sys_modules = self.get_attribute(self.sys_module.clone(), "modules")?;
sys_modules
.get_item(module.to_string(), self)
.or_else(|_| {
// if the import inputs seem weird, e.g a package import or something, rather than just
// a straight `import ident`
let weird = module.contains('.')
|| level != 0
|| objbool::boolval(self, from_list.clone()).unwrap_or(true);
let module = self.new_str(module.to_owned());
let cached_module = if weird {
None
} else {
let sys_modules = self.get_attribute(self.sys_module.clone(), "modules")?;
sys_modules.get_item(module.clone(), self).ok()
};
match cached_module {
Some(module) => Ok(module),
None => {
let import_func = self
.get_attribute(self.builtins.clone(), "__import__")
.map_err(|_| self.new_import_error("__import__ not found".to_string()))?;
@@ -424,17 +476,18 @@ impl VirtualMachine {
(self.get_none(), self.get_none())
};
self.invoke(
import_func,
&import_func,
vec![
self.ctx.new_str(module.to_string()),
module,
globals,
locals,
from_list.clone(),
self.ctx.new_int(level),
],
)
})
.map_err(|exc| import::remove_importlib_frames(self, &exc))
.map_err(|exc| import::remove_importlib_frames(self, &exc))
}
}
}
/// Determines if `obj` is an instance of `cls`, either directly, indirectly or virtually via
@@ -463,7 +516,7 @@ impl VirtualMachine {
pub fn call_get_descriptor(&self, attr: PyObjectRef, obj: PyObjectRef) -> PyResult {
let attr_class = attr.class();
if let Some(descriptor) = objtype::class_get_attr(&attr_class, "__get__") {
if let Some(ref descriptor) = objtype::class_get_attr(&attr_class, "__get__") {
let cls = obj.class();
self.invoke(descriptor, vec![attr, obj.clone(), cls.into_object()])
} else {
@@ -487,14 +540,14 @@ impl VirtualMachine {
func
);
let wrapped = self.call_get_descriptor(func, obj.clone())?;
self.invoke(wrapped, args)
self.invoke(&wrapped, args)
}
None => Err(self.new_type_error(format!("Unsupported method: {}", method_name))),
}
}
#[cfg_attr(feature = "flame-it", flame("VirtualMachine"))]
fn _invoke(&self, func_ref: PyObjectRef, args: PyFuncArgs) -> PyResult {
fn _invoke(&self, func_ref: &PyObjectRef, args: PyFuncArgs) -> PyResult {
vm_trace!("Invoke: {:?} {:?}", func_ref, args);
if let Some(PyFunction {
@@ -513,7 +566,7 @@ impl VirtualMachine {
ref object,
}) = func_ref.payload()
{
self.invoke(function.clone(), args.insert(object.clone()))
self.invoke(&function, args.insert(object.clone()))
} else if let Some(PyBuiltinFunction { ref value }) = func_ref.payload() {
value(self, args)
} else {
@@ -523,9 +576,8 @@ impl VirtualMachine {
}
}
// TODO: make func_ref an &PyObjectRef
#[inline]
pub fn invoke<T>(&self, func_ref: PyObjectRef, args: T) -> PyResult
pub fn invoke<T>(&self, func_ref: &PyObjectRef, args: T) -> PyResult
where
T: Into<PyFuncArgs>,
{
@@ -546,7 +598,7 @@ impl VirtualMachine {
let trace_func = self.trace_func.borrow().clone();
if !self.is_none(&trace_func) {
self.use_tracing.replace(false);
let res = self.invoke(trace_func, args.clone());
let res = self.invoke(&trace_func, args.clone());
self.use_tracing.replace(true);
res?;
}
@@ -554,7 +606,7 @@ impl VirtualMachine {
let profile_func = self.profile_func.borrow().clone();
if !self.is_none(&profile_func) {
self.use_tracing.replace(false);
let res = self.invoke(profile_func, args);
let res = self.invoke(&profile_func, args);
self.use_tracing.replace(true);
res?;
}
@@ -592,7 +644,7 @@ impl VirtualMachine {
pub fn invoke_with_locals(
&self,
function: PyObjectRef,
function: &PyObjectRef,
cells: PyDictRef,
locals: PyDictRef,
) -> PyResult {
@@ -605,7 +657,7 @@ impl VirtualMachine {
}
panic!(
"invoke_with_locals: expected python function, got: {:?}",
function
*function
);
}
@@ -831,7 +883,7 @@ impl VirtualMachine {
{
if let Some(method_or_err) = self.get_method(obj.clone(), method) {
let method = method_or_err?;
let result = self.invoke(method, vec![arg.clone()])?;
let result = self.invoke(&method, vec![arg.clone()])?;
if !result.is(&self.ctx.not_implemented()) {
return Ok(result);
}
@@ -864,6 +916,43 @@ impl VirtualMachine {
})
}
pub fn generic_getattribute(
&self,
obj: PyObjectRef,
name_str: PyStringRef,
) -> PyResult<Option<PyObjectRef>> {
let name = name_str.as_str();
let cls = obj.class();
if let Some(attr) = objtype::class_get_attr(&cls, &name) {
let attr_class = attr.class();
if objtype::class_has_attr(&attr_class, "__set__") {
if let Some(descriptor) = objtype::class_get_attr(&attr_class, "__get__") {
return self
.invoke(&descriptor, vec![attr, obj, cls.into_object()])
.map(Some);
}
}
}
let attr = if let Some(ref dict) = obj.dict {
dict.get_item_option(name_str.clone(), self)?
} else {
None
};
if let Some(obj_attr) = attr {
Ok(Some(obj_attr))
} else if let Some(attr) = objtype::class_get_attr(&cls, &name) {
self.call_get_descriptor(attr, obj).map(Some)
} else if let Some(getter) = objtype::class_get_attr(&cls, "__getattr__") {
self.invoke(&getter, vec![obj, name_str.into_object()])
.map(Some)
} else {
Ok(None)
}
}
pub fn is_callable(&self, obj: &PyObjectRef) -> bool {
match_class!(obj,
PyFunction => true,
@@ -1132,7 +1221,7 @@ impl VirtualMachine {
pub fn _membership(&self, haystack: PyObjectRef, needle: PyObjectRef) -> PyResult {
if let Some(method_or_err) = self.get_method(haystack.clone(), "__contains__") {
let method = method_or_err?;
self.invoke(method, vec![needle])
self.invoke(&method, vec![needle])
} else {
self._membership_iter_search(haystack, needle)
}

8
wasm/lib/src/browser_module.rs
View File

@@ -133,7 +133,7 @@ fn browser_request_animation_frame(func: PyCallable, vm: &VirtualMachine) -> PyR
let vm = &stored_vm.vm;
let func = func.clone();
let args = vec![vm.ctx.new_float(time)];
let _ = vm.invoke(func.into_object(), args);
let _ = vm.invoke(&func.into_object(), args);
let closure = f.borrow_mut().take();
drop(closure);
@@ -212,12 +212,12 @@ impl PyPromise {
} else {
vec![convert::js_to_py(vm, val)]
};
vm.invoke(on_fulfill.into_object(), PyFuncArgs::new(args, vec![]))
vm.invoke(&on_fulfill.into_object(), PyFuncArgs::new(args, vec![]))
}
Err(err) => {
if let OptionalArg::Present(on_reject) = on_reject {
let err = convert::js_to_py(vm, err);
vm.invoke(on_reject.into_object(), PyFuncArgs::new(vec![err], vec![]))
vm.invoke(&on_reject.into_object(), PyFuncArgs::new(vec![err], vec![]))
} else {
return Err(err);
}
@@ -240,7 +240,7 @@ impl PyPromise {
.expect("that the vm is valid when the promise resolves");
let vm = &stored_vm.vm;
let err = convert::js_to_py(vm, err);
let res = vm.invoke(on_reject.into_object(), PyFuncArgs::new(vec![err], vec![]));
let res = vm.invoke(&on_reject.into_object(), PyFuncArgs::new(vec![err], vec![]));
convert::pyresult_to_jsresult(vm, res)
})
});

2
wasm/lib/src/convert.rs
View File

@@ -94,7 +94,7 @@ pub fn py_to_js(vm: &VirtualMachine, py_obj: PyObjectRef) -> JsValue {
.insert(js_sys::JsString::from(key).into(), js_to_py(vm, val));
}
}
let result = vm.invoke(py_obj.clone(), py_func_args);
let result = vm.invoke(&py_obj, py_func_args);
pyresult_to_jsresult(vm, result)
};
let closure = Closure::wrap(Box::new(closure)

7
wasm/lib/src/js_module.rs
View File

@@ -1,9 +1,8 @@
use js_sys::{Array, Object, Reflect};
use rustpython_vm::function::Args;
use rustpython_vm::obj::{objfloat::PyFloatRef, objstr::PyStringRef, objtype::PyClassRef};
use rustpython_vm::pyobject::{
create_type, PyClassImpl, PyObjectRef, PyRef, PyResult, PyValue, TryFromObject,
};
use rustpython_vm::pyobject::{PyClassImpl, PyObjectRef, PyRef, PyResult, PyValue, TryFromObject};
use rustpython_vm::types::create_type;
use rustpython_vm::VirtualMachine;
use wasm_bindgen::{prelude::*, JsCast};
@@ -244,7 +243,7 @@ fn new_js_error(vm: &VirtualMachine, err: JsValue) -> PyObjectRef {
pub fn make_module(vm: &VirtualMachine) -> PyObjectRef {
let ctx = &vm.ctx;
py_module!(vm, "_js", {
"JsError" => create_type("JsError", &ctx.type_type, &ctx.exceptions.exception_type),
"JsError" => create_type("JsError", &ctx.type_type(), &ctx.exceptions.exception_type),
"JsValue" => PyJsValue::make_class(ctx),
})
}

2
wasm/lib/src/vm_class.rs
View File

@@ -256,7 +256,7 @@ impl WASMVirtualMachine {
let mod_name = name.clone();
let stdlib_init_fn = move |vm: &VirtualMachine| {
let module = vm.ctx.new_module(&name, vm.ctx.new_dict());
let module = vm.new_module(&name, vm.ctx.new_dict());
for (key, value) in module_items.clone() {
vm.set_attr(&module, key, value).unwrap();
}