Compare commits

...

53 Commits

Author SHA1 Message Date
copilot-swe-agent[bot]
9a11d44a5c Align test_str with CPython and move local coverage
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
2026-03-29 04:33:46 +09:00
copilot-swe-agent[bot]
514053415c Tighten str subclass conversion regression
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
2026-03-29 04:33:46 +09:00
copilot-swe-agent[bot]
2ff66e7ed7 Format str constructor fast path
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
2026-03-29 04:33:46 +09:00
copilot-swe-agent[bot]
ed2fd1e16f Preserve str subclass returned by __repr__
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
2026-03-29 04:33:46 +09:00
copilot-swe-agent[bot]
8a56d78bda Initial plan 2026-03-29 04:33:46 +09:00
Copilot
902985def7 Fix inspect.getsource returning truncated source for multi-line function definitions (#7519)
* Initial plan

* fix: restore def-line source range before entering function scope so co_firstlineno is correct

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/94701403-2011-4525-88f1-6e06891da6a4

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* fix: remove pre-existing expectedFailure decorators from test_gettext plural form tests

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/ce27bf53-569f-45a0-ad5a-08e8f322c717

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* remove extra_tests/snippets/inspect_getsource.py (covered by test_inspect)

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/2b64da1b-8aab-4fec-8b28-3a21d46ac2f9

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
2026-03-29 00:52:34 +09:00
dependabot[bot]
90c5464901 Bump serialize-javascript and terser-webpack-plugin in /wasm/demo (#7523)
Removes [serialize-javascript](https://github.com/yahoo/serialize-javascript). It's no longer used after updating ancestor dependency [terser-webpack-plugin](https://github.com/webpack/terser-webpack-plugin). These dependencies need to be updated together.


Removes `serialize-javascript`

Updates `terser-webpack-plugin` from 5.3.16 to 5.4.0
- [Release notes](https://github.com/webpack/terser-webpack-plugin/releases)
- [Changelog](https://github.com/webpack/terser-webpack-plugin/blob/main/CHANGELOG.md)
- [Commits](https://github.com/webpack/terser-webpack-plugin/compare/v5.3.16...v5.4.0)

---
updated-dependencies:
- dependency-name: serialize-javascript
  dependency-version: 
  dependency-type: indirect
- dependency-name: terser-webpack-plugin
  dependency-version: 5.4.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-29 00:52:00 +09:00
Huy VĹ© (Josh)
da440dbbbe type.__new__: preserve caller namespace when reading __qualname__ (#7524)
* type.__new__: preserve caller namespace when reading __qualname__

* type.__new__: preserve caller namespace when reading __qualname__
2026-03-29 00:51:49 +09:00
dependabot[bot]
1a9b10ece5 Bump winresource from 0.1.30 to 0.1.31 (#7522)
Bumps [winresource](https://github.com/BenjaminRi/winresource) from 0.1.30 to 0.1.31.
- [Commits](https://github.com/BenjaminRi/winresource/commits)

---
updated-dependencies:
- dependency-name: winresource
  dependency-version: 0.1.31
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-28 14:47:55 +09:00
dependabot[bot]
dd632363c8 Bump the wasmtime group with 3 updates (#7521)
Bumps the wasmtime group with 3 updates: [cranelift](https://github.com/bytecodealliance/wasmtime), [cranelift-jit](https://github.com/bytecodealliance/wasmtime) and [cranelift-module](https://github.com/bytecodealliance/wasmtime).


Updates `cranelift` from 0.129.1 to 0.130.0
- [Release notes](https://github.com/bytecodealliance/wasmtime/releases)
- [Changelog](https://github.com/bytecodealliance/wasmtime/blob/main/RELEASES.md)
- [Commits](https://github.com/bytecodealliance/wasmtime/commits)

Updates `cranelift-jit` from 0.129.1 to 0.130.0
- [Release notes](https://github.com/bytecodealliance/wasmtime/releases)
- [Changelog](https://github.com/bytecodealliance/wasmtime/blob/main/RELEASES.md)
- [Commits](https://github.com/bytecodealliance/wasmtime/commits)

Updates `cranelift-module` from 0.129.1 to 0.130.0
- [Release notes](https://github.com/bytecodealliance/wasmtime/releases)
- [Changelog](https://github.com/bytecodealliance/wasmtime/blob/main/RELEASES.md)
- [Commits](https://github.com/bytecodealliance/wasmtime/commits)

---
updated-dependencies:
- dependency-name: cranelift
  dependency-version: 0.130.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: wasmtime
- dependency-name: cranelift-jit
  dependency-version: 0.130.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: wasmtime
- dependency-name: cranelift-module
  dependency-version: 0.130.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: wasmtime
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-28 14:47:42 +09:00
Jeong, YunWon
f7556b00c1 Bytecode parity (#7514)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-28 09:19:12 +09:00
Jeong, YunWon
3dae07cd60 winapi functions (#7516) 2026-03-28 00:02:49 +09:00
Jeong, YunWon
fddd7cb690 fix cron-ci (#7483)
* fix cron-ci

* fix custom_text_test_runner
2026-03-27 23:55:30 +09:00
Shahar Naveh
410721740d Oparg resume depth (#7515)
* Base resume context

* Fixes for api change

* Align codegen

* Align `frame.rs` to the api changes

* fix jit

* Use new oparg

* Fix doc

* let `ir` to decide exception depth
2026-03-27 21:47:52 +09:00
dependabot[bot]
e3ac1bf8dc Bump node-forge from 1.3.2 to 1.4.0 in /wasm/demo (#7513)
Bumps [node-forge](https://github.com/digitalbazaar/forge) from 1.3.2 to 1.4.0.
- [Changelog](https://github.com/digitalbazaar/forge/blob/main/CHANGELOG.md)
- [Commits](https://github.com/digitalbazaar/forge/compare/v1.3.2...v1.4.0)

---
updated-dependencies:
- dependency-name: node-forge
  dependency-version: 1.4.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-27 12:44:55 +09:00
Jeong, YunWon
3a8fb76014 Bytecode parity (#7507)
* Bytecode parity phase 3

Compiler changes:
- Emit TO_BOOL in and/or short-circuit evaluation (COPY+TO_BOOL+JUMP)
- Add module-level __conditional_annotations__ cell (PEP 649)
- Only set conditional annotations for AnnAssign, not function params
- Skip __classdict__ cell when future annotations are active
- Convert list literals to tuples in for-loop iterables
- Fix cell variable ordering: parameters first, then alphabetical
- Fix RESUME DEPTH1 flag for yield-from/await
- Don't propagate __classdict__/__conditional_annotations__ freevar
  through regular functions — only annotation/type-param scopes
- Inline string compilation path

* Skip test_thread_safety in _test_multiprocessing

SIGSEGV in _finalizer_registry dict access under aggressive GC
and thread switching. Root cause is dict thread-safety in VM.

* Skip list→tuple optimization for async for; propagate future_annotations to nested scopes
2026-03-27 12:42:29 +09:00
Jeong, YunWon
a91127c91a Reorder PyNumberBinaryOp to match NB_* constants (#7512)
Align variant ordering with BinaryOperator enum and
CPython's NB_* constants from opcode.h. Divmod is placed
last as it has no corresponding NB_* constant.
2026-03-27 12:41:02 +09:00
Jeong, YunWon
af0c2526a7 Fix GC TOCTOU race in collect_inner referent traversal (#7511)
Pre-compute referent pointers once per object in step 3 and reuse
them in step 4 (BFS reachability). Previously, gc_get_referent_ptrs()
was called independently in both steps. If a dict's write lock state
changed between the two calls (e.g., held by another thread during
one traversal but not the other), the two traversals could return
different results. This caused live objects to be incorrectly
classified as unreachable and cleared by GC.
2026-03-27 12:39:45 +09:00
dependabot[bot]
f42ffd61a1 Bump strum from 0.27.2 to 0.28.0 (#7510)
Bumps [strum](https://github.com/Peternator7/strum) from 0.27.2 to 0.28.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/compare/v0.27.2...v0.28.0)

---
updated-dependencies:
- dependency-name: strum
  dependency-version: 0.28.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-27 11:27:19 +09:00
dependabot[bot]
3f92c3ad1c Bump aws-lc-rs from 1.16.0 to 1.16.2 (#7509)
Bumps [aws-lc-rs](https://github.com/aws/aws-lc-rs) from 1.16.0 to 1.16.2.
- [Release notes](https://github.com/aws/aws-lc-rs/releases)
- [Commits](https://github.com/aws/aws-lc-rs/compare/v1.16.0...v1.16.2)

---
updated-dependencies:
- dependency-name: aws-lc-rs
  dependency-version: 1.16.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-27 10:55:12 +09:00
Shahar Naveh
9282a870db Unify lint CI job (#7505)
* use `prek` for unified linting

* Fix actionlint error

* Generate metadata when specific files change

* `check_redundant_patches.py` to accept glob path

* Test

* revert defective changes

* use `rustfmt` over `cargo fmt` for individual files

* debug reviewdog

* rustfmt

* Move comment to correct location

* defevtive fmt test

* Fail with reviewdog

* fix reviewdog perms

* Try to use present token

* without checks oerms

* put normal perms

* fmt
2026-03-26 21:43:34 +09:00
Shahar Naveh
7a6dbd6624 Align concurrency CI groups names (#7508) 2026-03-26 14:33:14 +09:00
dependabot[bot]
6c3dd2885d Bump picomatch from 2.3.1 to 2.3.2 in /wasm/demo (#7506)
Bumps [picomatch](https://github.com/micromatch/picomatch) from 2.3.1 to 2.3.2.
- [Release notes](https://github.com/micromatch/picomatch/releases)
- [Changelog](https://github.com/micromatch/picomatch/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/picomatch/compare/2.3.1...2.3.2)

---
updated-dependencies:
- dependency-name: picomatch
  dependency-version: 2.3.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-26 12:51:49 +09:00
Jeong, YunWon
c9cfb3d606 Bytecode parity (#7504)
* Match CPython LOAD_SPECIAL stack semantics for with/async-with

LOAD_SPECIAL now pushes (callable, self_or_null) matching CPython's
CALL convention, instead of a single bound method:
- Function descriptors: push (func, self)
- Plain attributes: push (bound, NULL)

Updated all with-statement paths:
- Entry: add SWAP 3 after SWAP 2, remove PUSH_NULL before CALL 0
- Normal exit: remove PUSH_NULL before CALL 3
- Exception handler (WITH_EXCEPT_START): read exit_func at TOS-4
  and self_or_null at TOS-3
- Suppress block: 3 POP_TOPs after POP_EXCEPT (was 2)
- FBlock exit (preserve_tos): SWAP 3 + SWAP 2 rotation
- UnwindAction::With: remove PUSH_NULL

Stack effects updated: LoadSpecial (2,1), WithExceptStart (7,6)

* Normalize LOAD_FAST_CHECK and JUMP_BACKWARD_NO_INTERRUPT

Add LOAD_FAST_CHECK → LOAD_FAST and JUMP_BACKWARD_NO_INTERRUPT →
JUMP_BACKWARD to opname normalization in dis_dump.py. These are
optimization variants with identical semantics.

* Add EXTENDED_ARG to SKIP_OPS, normalize LOAD_FAST_CHECK and JUMP_BACKWARD_NO_INTERRUPT

* Remove duplicate return-None when block already has return

Skip duplicate_end_returns for blocks that already end with
LOAD_CONST + RETURN_VALUE. Run DCE + unreachable elimination
after duplication to remove the now-unreachable original return
block.

* Improve __static_attributes__ collection accuracy

- Support tuple/list unpacking targets: (self.x, self.y) = val
- Skip @staticmethod and @classmethod decorated methods
- Use scan_target_for_attrs helper for recursive target scanning

* Use method mode for function-local import attribute calls

Function-local imports (scope is Local+IMPORTED) should use method
mode LOAD_ATTR like regular names, not plain mode. Only module/class
scope imports use plain LOAD_ATTR + PUSH_NULL.

* Optimize constant iterable before GET_ITER to LOAD_CONST tuple

Convert BUILD_LIST/SET 0 + LOAD_CONST + LIST_EXTEND/SET_UPDATE + GET_ITER
to just LOAD_CONST (tuple) + GET_ITER, matching CPython's optimization
for constant list/set literals in for-loop iterables.

Also fix is_name_imported to use method mode for function-local imports,
and improve __static_attributes__ accuracy (skip @classmethod/@staticmethod,
handle tuple/list unpacking targets).

* Fix cell variable ordering: parameters first, then alphabetical

CPython orders cell variables with parameter cells first (in
parameter definition order), then non-parameter cells sorted
alphabetically. Previously all cells were sorted alphabetically.

Also add for-loop iterable optimization: constant BUILD_LIST/SET
before GET_ITER is folded to just LOAD_CONST tuple.

* Emit COPY_FREE_VARS before MAKE_CELL matching CPython order

CPython emits COPY_FREE_VARS first, then MAKE_CELL instructions.
Previously RustPython emitted them in reverse order.

* Fix RESUME AfterYield encoding to match CPython 3.14 (value 5)

CPython 3.14 uses RESUME arg=5 for after-yield, not 1.
Also reorder COPY_FREE_VARS before MAKE_CELL and fix cell
variable ordering (parameters first, then alphabetical).

* Address code review feedback from #7481

- Set is_generator flag for generator expressions in scan_comprehension
- Fix posonlyargs priority in collect_static_attributes first param
- Add match statement support to scan_store_attrs
- Fix stale decorator stack comment
- Reorder NOP removal after fold_unary_negative for better collection folding

* Fold constant list/set/tuple literals in compiler

When all elements of a list/set/tuple literal are constants and
there are 3+ elements, fold them into a single constant:
- list: BUILD_LIST 0 + LOAD_CONST (tuple) + LIST_EXTEND 1
- set:  BUILD_SET 0  + LOAD_CONST (tuple) + SET_UPDATE 1
- tuple: LOAD_CONST (tuple)

This matches CPython's compiler optimization and fixes the most
common bytecode difference (92/200 sampled files).

Also add bytecode comparison scripts (dis_dump.py, compare_bytecode.py)
for systematic parity tracking.

* Use BUILD_MAP 0 + MAP_ADD for large dicts (>= 16 pairs)

Match CPython's compiler behavior: dicts with 16+ key-value pairs
use BUILD_MAP 0 followed by MAP_ADD for each pair, instead of
pushing all keys/values on the stack and calling BUILD_MAP N.

* Fix clippy warnings and cargo fmt

* fix surrogate
2026-03-25 22:25:21 +09:00
lif
e1ecb87f32 fix: Flush stdout on shutdown matching CPython behavior (#7503)
* fix: flush stdout on interpreter shutdown matching CPython behavior

When stdout flush fails during shutdown, report the error via
run_unraisable and exit with code 120 (matching CPython's
Py_FinalizeEx). Skip flushing already-closed or None streams.
Stderr flush errors remain silently ignored per CPython behavior.

Fixes #5521

Signed-off-by: majiayu000 <1835304752@qq.com>

* refactor: replace magic number 120 with named constant EXITCODE_FLUSH_FAILURE

Address review feedback on PR #7503: improve readability by extracting
the CPython-compat exit code into a named constant.

Signed-off-by: majiayu000 <1835304752@qq.com>

---------

Signed-off-by: majiayu000 <1835304752@qq.com>
2026-03-25 19:59:16 +09:00
Jeong, YunWon
ea5a6cd9c0 Bytecode parity (#7481)
* Bytecode parity

 Compiler changes:
    - Remove PUSH_NULL from decorator cal
ls, use CALL 0
    - Collect __static_attributes__ from self.xxx = patterns
    - Sort __static_attributes__ alphabetically
    - Move __classdict__ init before __doc__ in class prologue
    - Fold unary negative constants
    - Fold constant list/set literals (3+ elements)
    - Use BUILD_MAP 0 + MAP_ADD for 16+ dict pairs
    - Always run peephole optimizer for s
uperinstructions
    - Emit RETURN_GENERATOR for generator
 functions
    - Add is_generator flag to SymbolTabl
e

* Fix formatting and collapsible_if clippy warnings in compile.rs

* Fix clippy, fold_unary_negative chaining, and generator line tracing

- Replace irrefutable if-let with let for ExceptHandler
- Remove folded UNARY_NEGATIVE instead of replacing with NOP,
  enabling chained negation folding
- Initialize prev_line to def line for generators/coroutines
  to suppress spurious LINE events from preamble instructions
- Remove expectedFailure markers for now-passing tests

* Fix JIT StoreFastStoreFast, format, and remove expectedFailure markers

- Add StoreFastStoreFast handling in JIT instructions
- Fix cargo fmt in frame.rs
- Remove 11 expectedFailure markers for async jump tests in
  test_sys_settrace that now pass

* Fix peephole optimizer: use NOP replacement instead of remove()

Using remove() shifts instruction indices and corrupts subsequent
references, causing "pop stackref but null found" panics at runtime.
Replace folded/combined instructions with NOP instead, which are
cleaned up by the existing remove_nops pass.

* Revert peephole_optimize to use remove() for chaining support

NOP replacement broke chaining of peephole optimizations (e.g.
LOAD_CONST+TO_BOOL then LOAD_CONST+UNARY_NOT for 'not True').
The remove() approach is used by upstream and works correctly here;
fold_unary_negative keeps NOP replacement since it doesn't need chaining.

* Fix StoreFastStoreFast to handle NULL from LoadFastAndClear

StoreFast uses pop_value_opt() to allow NULL values from
LoadFastAndClear in inlined comprehension cleanup paths.
StoreFastStoreFast must do the same, otherwise the peephole
optimizer's fusion of two StoreFast instructions panics when
restoring unbound locals after an inlined comprehension.
2026-03-25 16:10:19 +09:00
Jeong, YunWon
6b5c5a9e92 Handle EINTR retry in os.write() (PEP 475) (#7482)
* Handle EINTR retry in os.write() (PEP 475)

Add EINTR retry loop to os.write(), matching the existing
pattern in os.read() and os.readinto(). Remove the
expectedFailure marker from test_write in _test_eintr.py.

* Add atomic snapshot for dict/dict_keys in extract_elements

Add fast paths for dict and dict_keys types in
extract_elements_with, matching _list_extend() in CPython
Objects/listobject.c. Each branch takes an atomic snapshot
under a single read lock, preventing race conditions from
concurrent dict mutation without the GIL.

Remove expectedFailure from test_thread_safety.
2026-03-25 14:02:38 +09:00
Shahar Naveh
211649d148 Pin setup-node action to a commit hash (#7495)
* Pin `setup-node` action to a commit hash

* Don't use cache for release

* Revert changes of `release.yml`
2026-03-25 12:05:19 +09:00
Shahar Naveh
4ebc3112d9 Cleanup release.yml a bit (#7499)
* Cleanup matrix usage. enables jit on macos

* Pin some actions to commit hash

* Disable node cache

* Inline `CARGO_ARGS`

* Add `stdio` and `host_env` features

* Only upload to pages if not running on fork
2026-03-25 12:04:56 +09:00
Shahar Naveh
6db7910ca4 Pin rust-toolchain action to a commit hash (#7500) 2026-03-25 12:04:34 +09:00
Shahar Naveh
8d3bc4cb54 Add test_concurrent_futures to FLAKY_MP_TESTS (#7502) 2026-03-25 12:04:14 +09:00
Shahar Naveh
20c6505bb9 Resolve shellcheck warning on ci.yaml (#7501) 2026-03-25 12:03:25 +09:00
Shahar Naveh
372280ede4 Pin setup-python action to a commit hash (#7494) 2026-03-25 11:58:39 +09:00
dependabot[bot]
82432be962 Bump lz4_flex from 0.12.1 to 0.13.0 (#7497)
Bumps [lz4_flex](https://github.com/pseitz/lz4_flex) from 0.12.1 to 0.13.0.
- [Release notes](https://github.com/pseitz/lz4_flex/releases)
- [Changelog](https://github.com/PSeitz/lz4_flex/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pseitz/lz4_flex/compare/0.12.1...0.13.0)

---
updated-dependencies:
- dependency-name: lz4_flex
  dependency-version: 0.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 17:51:11 +09:00
psyche
40c84f51c8 Expose rustpython_pylib in rustpython (#7498)
Co-authored-by: EtherealPsyche <EtherealPsyche@users.noreply.github.com>
2026-03-24 17:51:01 +09:00
Shahar Naveh
5408627594 Pin setup-python action to a commit hash (#7492) 2026-03-24 12:53:21 +09:00
dependabot[bot]
fb6520e5cc Bump cargo-bins/cargo-binstall from 1.17.7 to 1.17.8 (#7488)
Bumps [cargo-bins/cargo-binstall](https://github.com/cargo-bins/cargo-binstall) from 1.17.7 to 1.17.8.
- [Release notes](https://github.com/cargo-bins/cargo-binstall/releases)
- [Changelog](https://github.com/cargo-bins/cargo-binstall/blob/main/release-plz.toml)
- [Commits](1800853f25...113a77a4ce)

---
updated-dependencies:
- dependency-name: cargo-bins/cargo-binstall
  dependency-version: 1.17.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 12:52:39 +09:00
dependabot[bot]
e9b45a1419 Bump actions/cache from 5.0.3 to 5.0.4 (#7487)
Bumps [actions/cache](https://github.com/actions/cache) from 5.0.3 to 5.0.4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](cdf6c1fa76...668228422a)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: 5.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 12:52:25 +09:00
dependabot[bot]
2acf76bbaf Bump github/gh-aw from 0.58.3 to 0.62.5 (#7486)
Bumps [github/gh-aw](https://github.com/github/gh-aw) from 0.58.3 to 0.62.5.
- [Release notes](https://github.com/github/gh-aw/releases)
- [Changelog](https://github.com/github/gh-aw/blob/main/CHANGELOG.md)
- [Commits](08a903b1fb...48d8fdfddc)

---
updated-dependencies:
- dependency-name: github/gh-aw
  dependency-version: 0.62.5
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 12:52:10 +09:00
dependabot[bot]
dc95db7ae3 Bump libsqlite3-sys from 0.36.0 to 0.37.0 (#7485)
Bumps [libsqlite3-sys](https://github.com/rusqlite/rusqlite) from 0.36.0 to 0.37.0.
- [Release notes](https://github.com/rusqlite/rusqlite/releases)
- [Changelog](https://github.com/rusqlite/rusqlite/blob/master/Changelog.md)
- [Commits](https://github.com/rusqlite/rusqlite/compare/v0.36.0...v0.37.0)

---
updated-dependencies:
- dependency-name: libsqlite3-sys
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 12:52:02 +09:00
Shahar Naveh
20ae3ccda2 Set dependabot cooldown (#7490)
* Set dependabot cooldown

* Increase default to 7 days
2026-03-24 12:45:30 +09:00
Shahar Naveh
f1d0fc31c5 Adjust permissions of update-doc-db job (#7496) 2026-03-24 12:45:06 +09:00
Shahar Naveh
56c3a37266 Pin setup-python action to a commit hash (#7491) 2026-03-24 12:44:42 +09:00
Jeong, YunWon
8c016157f4 marshal (#7467)
* CPython-compatible marshal format

Unify marshal to a single CPython-compatible format. No separate
"cpython_marshal" reader — one format for frozen modules, .pyc
files, and the Python-level marshal module.

- ComparisonOperator: `(cmp_index << 5) | mask` matching COMPARE_OP
- MakeFunctionFlag: bit-position matching SET_FUNCTION_ATTRIBUTE
- Exception table varint: big-endian (matching Python/assemble.c)
- Linetable varint: little-endian (unchanged)
- Integer: TYPE_INT (i32) / TYPE_LONG (base-2^15 digits)
- Code objects: CPython field order (argcount, posonlyargcount, ...,
  co_localsplusnames, co_localspluskinds, ..., co_exceptiontable)

- FLAG_REF / TYPE_REF for object deduplication (version >= 3)
- allow_code keyword argument on dumps/loads/dump/load
- Subclass rejection (int/float/complex/tuple/list/dict/set/frozenset)
- Slice serialization (version >= 5)
- Buffer protocol fallback for memoryview/array
- Recursion depth limit (2000) for both reads and writes
- Streaming load (reads one object, seeks file position)
- TYPE_INT64, TYPE_FLOAT (text), TYPE_COMPLEX (text) for compat

serialize_code writes co_localsplusnames/co_localspluskinds from
split varnames/cellvars/freevars. deserialize_code splits them back.
Cell variable DEREF indices are translated between flat (wire) and
cell-relative (internal) representations in both directions.

Replace bitwise trick with match for new ComparisonOperator values.

21 -> 3 expected failures. Remaining: test_bad_reader (IO layer),
test_deterministic_sets (PYTHONHASHSEED), testIntern (string interning).

* Address code review: preserve CO_FAST_HIDDEN, fix varint overflow

- Use original localspluskinds from marshal data instead of
  rebuilding, preserving CO_FAST_HIDDEN and other flags
- Fix write_varint_be to handle values >= 2^30 (add 6th chunk)
- Remove unused build_localspluskinds_from_split

* Add depth guard to deserialize_value_typed

Prevents usize underflow when dict key deserialization path calls
deserialize_value_typed with depth=0 on composite types.
2026-03-23 13:10:51 +09:00
Jeong, YunWon
907ce4d895 Bytecode parity (#7475)
* Emit TO_BOOL before conditional jumps, fix class/module prologue

- Emit TO_BOOL before POP_JUMP_IF_TRUE/FALSE in the general case
  of compile_jump_if (Compare expressions excluded since they
  already produce a bool)
- Module-level __doc__: use STORE_NAME instead of STORE_GLOBAL
- Class body __module__: use LOAD_NAME instead of LOAD_GLOBAL
- Class body: store __firstlineno__ before __doc__

* Emit MAKE_CELL and COPY_FREE_VARS before RESUME

Emit MAKE_CELL for each cell variable and COPY_FREE_VARS N for
free variables at the start of each code object, before RESUME.
These instructions are no-ops in the VM but align the bytecode
with CPython 3.14's output.

* Emit __static_attributes__ at end of class bodies

Store a tuple of attribute names (currently always empty) as
__static_attributes__ in the class namespace, matching CPython
3.14's class body epilogue. Attribute name collection from
self.xxx accesses is a follow-up task.

* Remove expectedFailure from DictProxyTests iter tests

test_iter_keys, test_iter_values, test_iter_items now pass
because class bodies emit __static_attributes__ and
__firstlineno__, matching the expected dict key set.

* Use 1-based stack indexing for LIST_EXTEND, SET_UPDATE, etc.

Switch LIST_APPEND, LIST_EXTEND, SET_ADD, SET_UPDATE, MAP_ADD
from 0-based to 1-based stack depth argument, matching CPython's
PEEK(oparg) convention. Adjust the VM to subtract 1 before
calling nth_value.

* Use plain LOAD_ATTR + PUSH_NULL for calls on imported names

When the call target is an attribute of an imported name (e.g.,
logging.getLogger()), use plain LOAD_ATTR (method_flag=0) with
a separate PUSH_NULL instead of method-mode LOAD_ATTR. This
matches CPython 3.14's behavior which avoids the method call
optimization for module attribute access.

* Duplicate return-None epilogue for fall-through blocks

When the last block in a code object is exactly LOAD_CONST None +
RETURN_VALUE (the implicit return), duplicate these instructions
into blocks that would otherwise fall through to it. This matches
CPython 3.14's behavior of giving each code path its own explicit
return instruction.

* Run cargo fmt on ir.rs

* Remove expectedFailure from test_intrinsic_1 in test_dis

* Emit TO_BOOL before conditional jumps for all expressions including Compare

* Add __classdict__ cell for classes with function definitions

Set needs_classdict=true for class scopes that contain function
definitions (def/async def), matching CPython 3.14's behavior for
PEP 649 deferred annotation support. Also restore the Compare
expression check in compile_jump_if to skip TO_BOOL for comparison
operations.

* Emit __classdictcell__ store in class body epilogue

Store the __classdict__ cell reference as __classdictcell__ in
the class namespace when the class has __classdict__ as a cell
variable. Uses LOAD_DEREF (RustPython separates cell vars from
fast locals unlike CPython's unified array).

* Always run DCE to remove dead code after terminal instructions

Run basic dead code elimination (truncating instructions after
RETURN_VALUE/RAISE/JUMP within blocks) at all optimization
levels, not just optimize > 0. CPython always removes this dead
code during assembly.

* Restrict LOAD_ATTR plain mode to module/class scope imports

Only use plain LOAD_ATTR + PUSH_NULL for imports at module or
class scope. Function-local imports use method call mode LOAD_ATTR,
matching CPython 3.14's behavior.

* Eliminate unreachable blocks after jump normalization

Split DCE into two phases: (1) within-block truncation after
terminal instructions (always runs), (2) whole-block elimination
for blocks only reachable via fall-through from terminal blocks
(runs after normalize_jumps when dead jump instructions exist).

* Fold BUILD_TUPLE 0 into LOAD_CONST empty tuple

Convert BUILD_TUPLE with size 0 to LOAD_CONST () during constant
folding, matching CPython's optimization for empty tuple literals.

* Handle __classcell__ and __classdictcell__ in type.__new__

- Remove __classcell__ from class dict after setting the cell value
- Add __classdictcell__ handling: set cell to class namespace dict,
  then remove from class dict
- Register __classdictcell__ identifier
- Use LoadClosure instead of LoadDeref for __classdictcell__ emission
- Reorder MakeFunctionFlag bits to match CPython
- Run ruff format on scripts

* Revert __classdict__ cell and __classdictcell__ changes

The __classdict__ cell addition (for classes with function defs)
and __classdictcell__ store caused cell initialization failures
in importlib. These require deeper VM changes to properly support
the cell variable lifecycle. Reverted for stability.

* Fix unreachable block elimination with fixpoint reachability

Use fixpoint iteration to properly determine block reachability:
only mark jump targets of already-reachable blocks, preventing
orphaned blocks from falsely marking their targets as reachable.
Also add a final DCE pass after assembly NOP removal to catch
dead code created by normalize_jumps.

* Check enclosing scopes for IMPORTED flag in LOAD_ATTR mode

When deciding whether to use plain LOAD_ATTR for attribute calls,
check if the name is imported in any enclosing scope (not just
the current scope). This handles the common pattern where a module
is imported at module level but used inside functions.

* Add __classdict__ cell for classes with function definitions

Set needs_classdict=true when a class scope contains function
definitions (def/async def), matching CPython 3.14 which always
creates a __classdict__ cell for PEP 649 support in such classes.

* Store __classdictcell__ in class body epilogue

Store the __classdict__ cell reference as __classdictcell__ in
the class namespace using LoadClosure (which loads the cell
object itself, not the value inside). This matches CPython 3.14's
class body epilogue.

* Fix clippy collapsible_if warnings and cargo fmt

* Revert __classdict__ and __classdictcell__ changes (cause import failures)

* Revert type.__new__ __classcell__ removal and __classdictcell__ handling

Revert the class cell cleanup changes from e6975f973 that cause
import failures when frozen module bytecode is stale. The original
behavior (not removing __classcell__ from class dict) is restored.

* Re-add __classdict__ cell and __classdictcell__ store

Restore the __classdict__ cell for classes with function
definitions and __classdictcell__ store in class body epilogue.
Previous failure was caused by stale .pyc cache files containing
bytecode from an intermediate MakeFunctionFlag reorder attempt,
not by these changes themselves.

* Reorder MakeFunctionFlag to match CPython's SET_FUNCTION_ATTRIBUTE

Reorder discriminants: Defaults=0, KwOnlyDefaults=1, Annotations=2,
Closure=3, Annotate=4, TypeParams=5. This aligns the oparg values
with CPython 3.14's convention.

Note: after this change, stale .pyc cache files must be deleted
(find . -name '*.pyc' -delete) to avoid bytecode mismatch errors.

* Use CPython-compatible power-of-two encoding for SET_FUNCTION_ATTRIBUTE

Override From/TryFrom for MakeFunctionFlag to use power-of-two
values (1,2,4,8,16,32) matching CPython's SET_FUNCTION_ATTRIBUTE
oparg encoding, instead of sequential discriminants (0,1,2,3,4,5).

* Remove expectedFailure from test_elim_jump_after_return1 and test_no_jump_over_return_out_of_finally_block

* Remove __classcell__ and __classdictcell__ from class dict in type.__new__

* Remove expectedFailure from test___classcell___expected_behaviour, cargo fmt

* Handle MakeCell and CopyFreeVars as no-ops in JIT

These prologue instructions are handled at frame creation time
by the VM. The JIT operates on already-initialized frames, so
these can be safely skipped during compilation.

* Remove expectedFailure from test_load_fast_known_simple

* Restore expectedFailure for test_load_fast_known_simple

The test expects LOAD_FAST_BORROW_LOAD_FAST_BORROW superinstruction
which RustPython does not emit yet.
2026-03-23 11:31:30 +09:00
Christian Legnitto
2180f535d8 Fix sub_table ordering for nested inlined comprehensions (PEP 709) (#7480)
When an inlined comprehension's first iterator expression contains
nested scopes (such as a lambda), those scopes' sub_tables appear at the
current position in the parent's sub_table list. The previous code
spliced the comprehension's own child sub_tables (e.g. inner inlined
comprehensions) into that same position before compiling the iterator,
which shifted the iterator's sub_tables to wrong indices.

Move the splice after the first iterator is compiled so its sub_tables
are consumed at their original positions.

Fixes nested list comprehensions like:
```python
    [[x for _, x in g] for _, g in itertools.groupby(..., lambda x: ...)]
```

Disclosure: I used AI to develop the patch though I was heavily
involved.
2026-03-22 17:23:03 +09:00
Lee Dogeon
3c62b5679f Implement bytearray.__str__ && bytes.__str__ (#7477)
* Implement bytearray.__str__

* Implement bytes.__str__

* Turn __str__ method into slot
2026-03-22 12:50:31 +09:00
dependabot[bot]
dfcb07cd93 Bump rustls-webpki from 0.103.9 to 0.103.10 (#7479)
Bumps [rustls-webpki](https://github.com/rustls/webpki) from 0.103.9 to 0.103.10.
- [Release notes](https://github.com/rustls/webpki/releases)
- [Commits](https://github.com/rustls/webpki/compare/v/0.103.9...v/0.103.10)

---
updated-dependencies:
- dependency-name: rustls-webpki
  dependency-version: 0.103.10
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-21 22:49:25 +09:00
Jeong, YunWon
2d676e7f4d A few windows fix (#7458)
* Disallow instantiation of sys.getwindowsversion type

Add slot_new to PyWindowsVersion that raises TypeError,
matching sys.flags behavior.

* Remove incorrect WSAHOS errno constant

WSAHOS was hardcoded as an alias for WSAHOST_NOT_FOUND, but
CPython guards it with #ifdef WSAHOS which doesn't exist in
modern Windows SDK headers.

* Fix mmap resize to raise OSError instead of SystemError

* Fix CreateProcess with empty environment on Windows

Empty env dict produced a single null terminator, but
CreateProcessW requires a double null for a valid empty
environment block.

* Revert mmap resize error to SystemError and fix errno.rs formatting

mmap resize raises SystemError (not OSError) when mremap is unavailable,
matching CPython behavior. test_mmap catches SystemError to skip unsupported
resize operations.

* Fix named mmap resize to raise OSError and unmark test_sleep expectedFailure

Named mmap resize on Windows should raise OSError (not SystemError).
Remove expectedFailure mark from TimeEINTRTest.test_sleep as it now passes.

* Use expectedFailureIf for TimeEINTRTest.test_sleep on Linux

test_sleep passes on macOS but fails on Linux due to timing.

* Remove expectedFailure for TimeEINTRTest.test_sleep

test_sleep now passes on all platforms.
2026-03-21 22:49:09 +09:00
Jeong, YunWon
3e9f825e1d Enable PEP 709 inlined comprehensions (#7412)
* Enable PEP 709 inlined comprehensions for function-like scopes

Activate the existing compile_inlined_comprehension() implementation
by fixing 6 bugs that prevented it from working:

- LoadFastAndClear: push NULL (not None) when slot is empty so
  StoreFast can restore empty state after comprehension
- StoreFast: accept NULL from stack for the restore path
- sub_tables.remove(0) replaced with next_sub_table cursor to
  match the pattern used elsewhere in the compiler
- in_inlined_comp flag moved from non-inlined to inlined path
- is_inlined_comprehension_context() now checks comp_inlined flag
  and restricts inlining to function-like scopes
- comp_inlined set only when parent scope uses fastlocals

Symbol table analysis handles conflict detection:
- Nested scopes in comprehension → skip inlining
- Bound name conflicts with parent symbol → skip inlining
- Cross-comprehension reference conflicts → skip inlining
- Splice comprehension sub_tables into parent for nested scope tracking

* Add localspluskinds, unify DEREF to localsplus index

- Add CO_FAST_LOCAL/CELL/FREE/HIDDEN constants and
  localspluskinds field to CodeObject for per-slot metadata
- Change DEREF instruction opargs from cell-relative indices
  (NameIdx) to localsplus absolute indices (oparg::VarNum)
- Add fixup_deref_opargs pass in ir.rs to convert cell-relative
  indices to localsplus indices after finalization
- Replace get_cell_name with get_localsplus_name in
  InstrDisplayContext trait
- Update VM cell_ref/get_cell_contents/set_cell_contents to use
  localsplus indices directly (no nlocals offset)
- Update function.rs cell2arg, super.rs __class__ lookup with
  explicit nlocals offsets

* Fix clippy warnings, formatting, restore _opcode_metadata.py

Fix cast_possible_truncation, nonminimal_bool, collapsible_if,
manual_contains clippy lints. Restore _opcode_metadata.py to
upstream/main version (3.14 aligned).

Pre-copy closure cells in Frame::new for coroutine locals().
Handle raw values in merged cell slots during inlined comps.
Exclude async comprehensions from inlining path.

* Exclude async/await comprehensions from PEP 709 inlining in symboltable

Async comprehensions and comprehensions with await in the element
expression need their own coroutine scope and cannot be inlined.
The symboltable builder was not checking these conditions, causing
incorrect symbol scope resolution when an async comprehension was
nested inside an inlined comprehension (e.g. [[x async for x in g]
for j in items]).
2026-03-21 22:48:35 +09:00
dependabot[bot]
4abe4c5bf0 Bump aws-lc-fips-sys from 0.13.12 to 0.13.13 (#7478)
Bumps [aws-lc-fips-sys](https://github.com/aws/aws-lc-rs) from 0.13.12 to 0.13.13.
- [Release notes](https://github.com/aws/aws-lc-rs/releases)
- [Commits](https://github.com/aws/aws-lc-rs/compare/aws-lc-fips-sys/v0.13.12...aws-lc-fips-sys/v0.13.13)

---
updated-dependencies:
- dependency-name: aws-lc-fips-sys
  dependency-version: 0.13.13
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-21 09:46:15 +09:00
Lee Dogeon
a1203ae207 Improve CPython compatibility related with PyBoundMethod (#7476)
* Add GetDescriptor for PyBoundMethod (return self)

CPython's method_descr_get always returns the bound method unchanged.
This preserves the original binding when __get__ is called on an
already-bound method (e.g. a.meth.__get__(b, B) still returns a).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add constructor validation for PyBoundMethod

Reject non-callable functions and None instances, matching CPython's
method_new which checks PyCallable_Check(func) and instance != Py_None.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix PyBoundMethod __reduce__ to propagate errors

Previously swallowed errors from get_attr with .ok(), silently
returning None. Now propagates errors matching CPython's method_reduce.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 00:33:40 +09:00
dependabot[bot]
5b6a479a1d Bump lexopt from 0.3.1 to 0.3.2 (#7470)
Bumps [lexopt](https://github.com/blyxxyz/lexopt) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/blyxxyz/lexopt/releases)
- [Changelog](https://github.com/blyxxyz/lexopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/blyxxyz/lexopt/compare/v0.3.1...v0.3.2)

---
updated-dependencies:
- dependency-name: lexopt
  dependency-version: 0.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-20 22:49:41 +09:00
78 changed files with 4290 additions and 1705 deletions

View File

@@ -109,6 +109,7 @@ lineiterator
linetable
loadfast
localsplus
localspluskinds
Lshift
lsprof
MAXBLOCKS

View File

@@ -70,6 +70,7 @@
"lossily",
"mcache",
"oparg",
"opargs",
"pyc",
"significand",
"summands",

View File

@@ -5,6 +5,11 @@ updates:
directory: /
schedule:
interval: weekly
cooldown:
default-days: 7
semver-major-days: 30
semver-minor-days: 7
semver-patch-days: 3
groups:
criterion:
patterns:
@@ -143,7 +148,20 @@ updates:
directory: /
schedule:
interval: weekly
cooldown:
default-days: 7
- package-ecosystem: npm
directory: /
schedule:
interval: weekly
cooldown:
default-days: 7
semver-major-days: 30
semver-minor-days: 7
semver-patch-days: 3
- package-ecosystem: pre-commit
directory: /
schedule:
interval: weekly
cooldown:
default-days: 7

View File

@@ -12,7 +12,7 @@ name: CI
# with the same event (push/pull_request) even they are in progress.
# This setting will help reduce the number of duplicated workflows.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name }}
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true
env:
@@ -45,9 +45,10 @@ jobs:
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@efa25f7f19611383d5b0ccf2d1c8914531636bf9
with:
components: clippy
toolchain: stable
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
with:
@@ -65,6 +66,9 @@ jobs:
- name: check compilation without threading
run: cargo check ${{ env.CARGO_ARGS }}
- run: cargo doc --locked
if: runner.os == 'Linux'
- name: check compilation without host_env (sandbox mode)
run: |
cargo check -p rustpython-vm --no-default-features --features compiler
@@ -148,9 +152,10 @@ jobs:
musl-tools: ${{ matrix.dependencies.musl-tools || false }}
gcc-aarch64-linux-gnu: ${{ matrix.dependencies.gcc-aarch64-linux-gnu || false }}
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@efa25f7f19611383d5b0ccf2d1c8914531636bf9
with:
targets: ${{ join(matrix.targets, ',') }}
toolchain: stable
- name: Setup Android NDK
if: ${{ contains(matrix.targets, 'aarch64-linux-android') }}
@@ -188,6 +193,7 @@ jobs:
# Tests that can be flaky when running with multiple processes `-j 2`. We will use `-j 1` for these.
FLAKY_MP_TESTS: >-
test_class
test_concurrent_futures
test_eintr
test_multiprocessing_fork
test_multiprocessing_forkserver
@@ -224,13 +230,15 @@ jobs:
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@efa25f7f19611383d5b0ccf2d1c8914531636bf9
with:
toolchain: stable
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
with:
save-if: ${{ github.ref == 'refs/heads/main' }}
- uses: actions/setup-python@v6.2.0
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -251,7 +259,7 @@ jobs:
shell: bash
run: |
cores=$(python -c 'print(__import__("os").process_cpu_count())')
echo "cores=${cores}" >> $GITHUB_OUTPUT
echo "cores=${cores}" >> "$GITHUB_OUTPUT"
- name: Run CPython tests
run: |
@@ -270,28 +278,32 @@ jobs:
- name: run cpython tests to check if env polluters have stopped polluting
shell: bash
run: |
for thing in ${{ join(matrix.env_polluting_tests, ' ') }}; do
IFS=' ' read -r -a target_array <<< "$TARGETS"
for thing in "${target_array[@]}"; do
for i in $(seq 1 10); do
set +e
target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v ${thing}
target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v "${thing}"
exit_code=$?
set -e
if [ ${exit_code} -eq 3 ]; then
if [ "${exit_code}" -eq 3 ]; then
echo "Test ${thing} polluted the environment on attempt ${i}."
break
fi
done
if [ ${exit_code} -ne 3 ]; then
if [ "${exit_code}" -ne 3 ]; then
echo "Test ${thing} is no longer polluting the environment after ${i} attempts!"
echo "Please remove ${thing} from matrix.env_polluting_tests in '.github/workflows/ci.yaml'."
echo "Please also remove the skip decorators that include the word 'POLLUTERS' in ${thing}."
if [ ${exit_code} -ne 0 ]; then
if [ "${exit_code}" -ne 0 ]; then
echo "Test ${thing} failed with exit code ${exit_code}."
echo "Please investigate which test item in ${thing} is failing and either mark it as an expected failure or a skip."
fi
exit 1
fi
done
env:
TARGETS: ${{ join(matrix.env_polluting_tests, ' ') }}
timeout-minutes: 15
- if: runner.os != 'Windows'
@@ -317,63 +329,68 @@ jobs:
run: python -I scripts/whats_left.py ${{ env.CARGO_ARGS }} --features jit
lint:
name: Lint Rust & Python code
name: Lint
runs-on: ubuntu-latest
permissions:
contents: read
checks: write
pull-requests: write
security-events: write # for zizmor
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@v6.2.0
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Check for redundant test patches
run: python scripts/check_redundant_patches.py
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@efa25f7f19611383d5b0ccf2d1c8914531636bf9
with:
components: clippy
toolchain: stable
components: rustfmt
- name: run clippy on wasm
run: cargo clippy --manifest-path=crates/wasm/Cargo.toml -- -Dwarnings
- uses: cargo-bins/cargo-binstall@113a77a4ce971c41332f2129c3d995df993cf746 # v1.17.8
- name: Ensure docs generate no warnings
run: cargo doc --locked
- name: Ensure Lib/_opcode_metadata is updated
- name: cargo shear
run: |
python scripts/generate_opcode_metadata.py
if [ -n "$(git status --porcelain)" ]; then
exit 1
fi
cargo binstall --no-confirm cargo-shear
cargo shear
- name: Install ruff
uses: astral-sh/ruff-action@4919ec5cf1f49eff0871dbcea0da843445b837e6 # v3.6.1
- name: actionlint
uses: reviewdog/action-actionlint@0d952c597ef8459f634d7145b0b044a9699e5e43 # v1.71.0
- name: zizmor
uses: zizmorcore/zizmor-action@71321a20a9ded102f6e9ce5718a2fcec2c4f70d8 # v0.5.2
- name: restore prek cache
if: ${{ github.ref != 'refs/heads/main' }} # never restore on main
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with:
version: "0.15.5"
args: "--version"
key: prek-${{ hashFiles('.pre-commit-config.yaml') }}
path: ~/.cache/prek
- run: ruff check --diff
- run: ruff format --check
- name: install prettier
run: |
yarn global add prettier
yarn global bin >> "$GITHUB_PATH"
- name: check wasm code with prettier
# prettier doesn't handle ignore files very well: https://github.com/prettier/prettier/issues/8506
run: cd wasm && git ls-files -z | xargs -0 prettier --check -u
# Keep cspell check as the last step. This is optional test.
- name: install extra dictionaries
run: npm install @cspell/dict-en_us @cspell/dict-cpp @cspell/dict-python @cspell/dict-rust @cspell/dict-win32 @cspell/dict-shell
- name: spell checker
uses: streetsidesoftware/cspell-action@v8
- name: prek
id: prek
uses: j178/prek-action@79f765515bd648eb4d6bb1b17277b7cb22cb6468 # v2.0.0
with:
files: "**/*.rs"
incremental_files_only: true
cache: false
show-verbose-logs: false
continue-on-error: true
- name: save prek cache
if: ${{ github.ref == 'refs/heads/main' }} # only save on main
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with:
key: prek-${{ hashFiles('.pre-commit-config.yaml') }}
path: ~/.cache/prek
- name: reviewdog
uses: reviewdog/action-suggester@aa38384ceb608d00f84b4690cacc83a5aba307ff # 1.24.0
with:
level: warning
fail_level: error
cleanup: false
miri:
if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip:ci') }}
@@ -387,7 +404,7 @@ jobs:
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@master
- uses: dtolnay/rust-toolchain@efa25f7f19611383d5b0ccf2d1c8914531636bf9
with:
toolchain: ${{ env.NIGHTLY_CHANNEL }}
components: miri
@@ -413,12 +430,18 @@ jobs:
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@efa25f7f19611383d5b0ccf2d1c8914531636bf9
with:
components: clippy
toolchain: stable
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
with:
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: cargo clippy
run: cargo clippy --manifest-path=crates/wasm/Cargo.toml -- -Dwarnings
- name: install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
- name: install geckodriver
@@ -426,12 +449,14 @@ jobs:
wget https://github.com/mozilla/geckodriver/releases/download/v0.36.0/geckodriver-v0.36.0-linux64.tar.gz
mkdir geckodriver
tar -xzf geckodriver-v0.36.0-linux64.tar.gz -C geckodriver
- uses: actions/setup-python@v6.2.0
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- run: python -m pip install -r requirements.txt
working-directory: ./wasm/tests
- uses: actions/setup-node@v6
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
cache: "npm"
cache-dependency-path: "wasm/demo/package-lock.json"
@@ -483,9 +508,10 @@ jobs:
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@efa25f7f19611383d5b0ccf2d1c8914531636bf9
with:
target: wasm32-wasip1
toolchain: stable
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
with:
@@ -502,32 +528,6 @@ jobs:
- name: build rustpython
run: cargo build --release --target wasm32-wasip1 --features freeze-stdlib,stdlib --verbose
- name: run snippets
run: wasmer run --dir $(pwd) target/wasm32-wasip1/release/rustpython.wasm -- "$(pwd)/extra_tests/snippets/stdlib_random.py"
run: wasmer run --dir "$(pwd)" target/wasm32-wasip1/release/rustpython.wasm -- "$(pwd)/extra_tests/snippets/stdlib_random.py"
- name: run cpython unittest
run: wasmer run --dir $(pwd) target/wasm32-wasip1/release/rustpython.wasm -- "$(pwd)/Lib/test/test_int.py"
cargo-shear:
name: cargo shear
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: cargo-bins/cargo-binstall@1800853f2578f8c34492ec76154caef8e163fbca # v1.17.7
- run: cargo binstall --no-confirm cargo-shear
- run: cargo shear
security-lint:
runs-on: ubuntu-latest
permissions:
security-events: write
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Run zizmor
uses: zizmorcore/zizmor-action@71321a20a9ded102f6e9ce5718a2fcec2c4f70d8 # v0.5.2
run: wasmer run --dir "$(pwd)" target/wasm32-wasip1/release/rustpython.wasm -- "$(pwd)/Lib/test/test_int.py"

View File

@@ -12,7 +12,7 @@ on:
name: Periodic checks/tasks
env:
CARGO_ARGS: --no-default-features --features stdlib,importlib,encodings,ssl-rustls,jit
CARGO_ARGS: --no-default-features --features stdlib,importlib,stdio,encodings,ssl-rustls,jit,host_env
PYTHON_VERSION: "3.14.3"
jobs:
@@ -35,7 +35,7 @@ jobs:
python-version: ${{ env.PYTHON_VERSION }}
- run: sudo apt-get update && sudo apt-get -y install lcov
- name: Run cargo-llvm-cov with Rust tests.
run: cargo llvm-cov --no-report --workspace --exclude rustpython_wasm --exclude rustpython-compiler-source --exclude rustpython-venvlauncher --verbose --no-default-features --features stdlib,importlib,encodings,ssl-rustls,jit
run: cargo llvm-cov --no-report --workspace --exclude rustpython_wasm --exclude rustpython-compiler-source --exclude rustpython-venvlauncher --verbose --no-default-features --features stdlib,importlib,stdio,encodings,ssl-rustls,jit,host_env
- name: Run cargo-llvm-cov with Python snippets.
run: python scripts/cargo-llvm-cov.py
continue-on-error: true
@@ -48,7 +48,7 @@ jobs:
if: ${{ github.event_name != 'pull_request' }}
uses: codecov/codecov-action@v5
with:
file: ./codecov.lcov
files: ./codecov.lcov
testdata:
name: Collect regression test data
@@ -170,12 +170,12 @@ jobs:
- name: restructure generated files
run: |
cd ./target/criterion/reports
find -type d -name cpython | xargs rm -rf
find -type d -name rustpython | xargs rm -rf
find -mindepth 2 -maxdepth 2 -name violin.svg | xargs rm -rf
find -type f -not -name violin.svg | xargs rm -rf
for file in $(find -type f -name violin.svg); do mv $file $(echo $file | sed -E "s_\./([^/]+)/([^/]+)/violin\.svg_./\1/\2.svg_"); done
find -mindepth 2 -maxdepth 2 -type d | xargs rm -rf
find . -type d -name cpython -print0 | xargs -0 rm -rf
find . -type d -name rustpython -print0 | xargs -0 rm -rf
find . -mindepth 2 -maxdepth 2 -name violin.svg -print0 | xargs -0 rm -rf
find . -type f -not -name violin.svg -print0 | xargs -0 rm -rf
find . -type f -name violin.svg -exec sh -c 'for file; do mv "$file" "$(echo "$file" | sed -E "s_\./([^/]+)/([^/]+)/violin\.svg_./\1/\2.svg_")"; done' _ {} +
find . -mindepth 2 -maxdepth 2 -type d -print0 | xargs -0 rm -rf
cd ..
mv reports/* .
rmdir reports

View File

@@ -7,7 +7,7 @@ on:
- "Lib/**"
concurrency:
group: lib-deps-${{ github.event.pull_request.number }}
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
env:
@@ -74,7 +74,7 @@ jobs:
- name: Setup Python
if: steps.changed-files.outputs.modules != ''
uses: actions/setup-python@v6.2.0
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "${{ env.PYTHON_VERSION }}"
@@ -83,22 +83,15 @@ jobs:
id: deps-check
run: |
# Run deps for all modules at once
python scripts/update_lib deps ${{ steps.changed-files.outputs.modules }} --depth 2 > /tmp/deps_output.txt 2>&1 || true
# Read output for GitHub Actions
echo "deps_output<<EOF" >> $GITHUB_OUTPUT
cat /tmp/deps_output.txt >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
# Check if there's any meaningful output
if [ -s /tmp/deps_output.txt ]; then
echo "has_output=true" >> $GITHUB_OUTPUT
else
echo "has_output=false" >> $GITHUB_OUTPUT
fi
echo "deps_output<<EOF" >> "$GITHUB_OUTPUT"
output=$(python scripts/update_lib deps "${MODULES}" --depth 2 2>&1 || true)
echo "$output" >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
env:
MODULES: ${{ steps.changed-files.outputs.modules }}
- name: Post comment
if: steps.deps-check.outputs.has_output == 'true'
if: steps.deps-check.outputs.deps_output != ''
uses: marocchino/sticky-pull-request-comment@v3
with:
header: lib-deps-check

View File

@@ -1,74 +0,0 @@
name: Format Check
# This workflow triggers when a PR is opened/updated
# Posts inline suggestion comments instead of auto-committing
on:
pull_request:
types: [opened, synchronize, reopened]
branches:
- main
- release
concurrency:
group: format-check-${{ github.event.pull_request.number }}
cancel-in-progress: true
env:
PYTHON_VERSION: "3.14.3"
jobs:
format_check:
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: reviewdog/action-actionlint@0d952c597ef8459f634d7145b0b044a9699e5e43 # v1.71.0
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt
- name: Run cargo fmt
run: cargo fmt --all
- name: Install ruff
uses: astral-sh/ruff-action@4919ec5cf1f49eff0871dbcea0da843445b837e6 # v3.6.1
with:
version: "0.15.4"
args: "--version"
- name: Run ruff format
run: ruff format
- name: Run ruff check import sorting
run: ruff check --select I --fix
- uses: actions/setup-python@v6.2.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Run generate_opcode_metadata.py
run: python scripts/generate_opcode_metadata.py
- name: Check for formatting changes
run: |
if ! git diff --exit-code; then
echo "::error::Formatting changes detected. Please run 'cargo fmt --all', 'ruff format', and 'ruff check --select I --fix' locally."
exit 1
fi
- name: Post formatting suggestions
if: failure()
uses: reviewdog/action-suggester@v1
with:
tool_name: auto-format
github_token: ${{ secrets.GITHUB_TOKEN }}
level: warning
filter_mode: diff_context

View File

@@ -16,40 +16,39 @@ permissions:
contents: write
env:
CARGO_ARGS: --no-default-features --features stdlib,importlib,encodings,sqlite,ssl
X86_64_PC_WINDOWS_MSVC_OPENSSL_LIB_DIR: C:\Program Files\OpenSSL\lib\VC\x64\MD
X86_64_PC_WINDOWS_MSVC_OPENSSL_INCLUDE_DIR: C:\Program Files\OpenSSL\include
jobs:
build:
runs-on: ${{ matrix.platform.runner }}
runs-on: ${{ matrix.os }}
# Disable this scheduled job when running on a fork.
if: ${{ github.repository == 'RustPython/RustPython' || github.event_name != 'schedule' }}
strategy:
matrix:
platform:
- runner: ubuntu-latest
include:
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
# - runner: ubuntu-latest
# target: i686-unknown-linux-gnu
# - runner: ubuntu-latest
# target: aarch64-unknown-linux-gnu
# - runner: ubuntu-latest
# target: armv7-unknown-linux-gnueabi
# - runner: ubuntu-latest
# target: s390x-unknown-linux-gnu
# - runner: ubuntu-latest
# target: powerpc64le-unknown-linux-gnu
- runner: macos-latest
- os: macos-latest
target: aarch64-apple-darwin
# - runner: macos-latest
# target: x86_64-apple-darwin
- runner: windows-2025
- os: windows-2025
target: x86_64-pc-windows-msvc
# - runner: windows-2025
# target: i686-pc-windows-msvc
# - runner: windows-2025
# target: aarch64-pc-windows-msvc
# - os: ubuntu-latest
# target: i686-unknown-linux-gnu
# - os: ubuntu-latest
# target: aarch64-unknown-linux-gnu
# - os: ubuntu-latest
# target: armv7-unknown-linux-gnueabi
# - os: ubuntu-latest
# target: s390x-unknown-linux-gnu
# - os: ubuntu-latest
# target: powerpc64le-unknown-linux-gnu
# - os: macos-latest
# target: x86_64-apple-darwin
# - os: windows-2025
# target: i686-pc-windows-msvc
# - os: windows-2025
# target: aarch64-pc-windows-msvc
fail-fast: false
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -57,34 +56,32 @@ jobs:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- uses: cargo-bins/cargo-binstall@main
with:
target: ${{ matrix.target }}
- name: Set up Environment
shell: bash
run: rustup target add ${{ matrix.platform.target }}
- name: Set up MacOS Environment
run: brew install autoconf automake libtool
if: runner.os == 'macOS'
- name: Install macOS dependencies
uses: ./.github/actions/install-macos-deps
with:
autoconf: true
automake: true
libtool: true
- name: Build RustPython
run: cargo build --release --target=${{ matrix.platform.target }} --verbose --features=threading ${{ env.CARGO_ARGS }}
if: runner.os == 'macOS'
- name: Build RustPython
run: cargo build --release --target=${{ matrix.platform.target }} --verbose --features=threading ${{ env.CARGO_ARGS }},jit
if: runner.os != 'macOS'
run: cargo build --release --target=${{ matrix.target }} --verbose --no-default-features --features stdlib,stdio,importlib,encodings,sqlite,host_env,ssl-rustls,threading,jit
- name: Rename Binary
run: cp target/${{ matrix.platform.target }}/release/rustpython target/rustpython-release-${{ runner.os }}-${{ matrix.platform.target }}
run: cp target/${{ matrix.target }}/release/rustpython target/rustpython-release-${{ runner.os }}-${{ matrix.target }}
if: runner.os != 'Windows'
- name: Rename Binary
run: cp target/${{ matrix.platform.target }}/release/rustpython.exe target/rustpython-release-${{ runner.os }}-${{ matrix.platform.target }}.exe
run: cp target/${{ matrix.target }}/release/rustpython.exe target/rustpython-release-${{ runner.os }}-${{ matrix.target }}.exe
if: runner.os == 'Windows'
- name: Upload Binary Artifacts
uses: actions/upload-artifact@v7.0.0
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: rustpython-release-${{ runner.os }}-${{ matrix.platform.target }}
path: target/rustpython-release-${{ runner.os }}-${{ matrix.platform.target }}*
name: rustpython-release-${{ runner.os }}-${{ matrix.target }}
path: target/rustpython-release-${{ runner.os }}-${{ matrix.target }}*
build-wasm:
runs-on: ubuntu-latest
@@ -106,16 +103,21 @@ jobs:
run: cp target/wasm32-wasip1/release/rustpython.wasm target/rustpython-release-wasm32-wasip1.wasm
- name: Upload Binary Artifacts
uses: actions/upload-artifact@v7.0.0
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: rustpython-release-wasm32-wasip1
path: target/rustpython-release-wasm32-wasip1.wasm
- name: install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
- uses: actions/setup-node@v6
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
package-manager-cache: false
- uses: mwilliamson/setup-wabt-action@v3
with: { wabt-version: "1.0.30" }
- name: build demo
run: |
npm install
@@ -123,6 +125,7 @@ jobs:
env:
NODE_OPTIONS: "--openssl-legacy-provider"
working-directory: ./wasm/demo
- name: build notebook demo
run: |
npm install
@@ -131,7 +134,9 @@ jobs:
env:
NODE_OPTIONS: "--openssl-legacy-provider"
working-directory: ./wasm/notebook
- name: Deploy demo to Github Pages
if: ${{ github.repository == 'RustPython/RustPython' }}
uses: peaceiris/actions-gh-pages@v4
with:
deploy_key: ${{ secrets.ACTIONS_DEMO_DEPLOY_KEY }}
@@ -150,26 +155,21 @@ jobs:
persist-credentials: false
- name: Download Binary Artifacts
uses: actions/download-artifact@v8.0.1
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: bin
pattern: rustpython-*
merge-multiple: true
- name: Create Lib Archive
run: |
zip -r bin/rustpython-lib.zip Lib/
run: zip -r bin/rustpython-lib.zip Lib/
- name: List Binaries
run: |
ls -lah bin/
file bin/*
- name: Create Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
tag: ${{ github.ref_name }}
run: ${{ github.run_number }}
PRE_RELEASE_INPUT: ${{ github.event.inputs.pre-release }}
run: |
if [[ "${PRE_RELEASE_INPUT}" == "false" ]]; then
RELEASE_TYPE_NAME=Release
@@ -188,3 +188,8 @@ jobs:
--generate-notes \
$PRERELEASE_ARG \
bin/rustpython-release-*
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
tag: ${{ github.ref_name }}
run: ${{ github.run_number }}
PRE_RELEASE_INPUT: ${{ github.event.inputs.pre-release }}

View File

@@ -1,8 +1,6 @@
name: Update doc DB
permissions:
contents: write
pull-requests: write
permissions: {}
on:
workflow_dispatch:
@@ -22,6 +20,8 @@ defaults:
jobs:
generate:
permissions:
contents: read
runs-on: ${{ matrix.os }}
strategy:
matrix:
@@ -54,17 +54,19 @@ jobs:
merge:
runs-on: ubuntu-latest
needs: generate
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: true
ref: ${{ inputs.base-ref }}
token: ${{ secrets.AUTO_COMMIT_PAT }}
- name: Create update branch
run: git switch -c "update-doc-${PYTHON_VERSION}"
env:
PYTHON_VERSION: ${{ inputs.python-version }}
run: git switch -c "update-doc-${PYTHON_VERSION}"
- name: Download generated doc DBs
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
@@ -107,7 +109,7 @@ jobs:
- name: Commit, push and create PR
env:
GH_TOKEN: ${{ secrets.AUTO_COMMIT_PAT }}
GH_TOKEN: ${{ github.token }}
PYTHON_VERSION: ${{ inputs.python-version }}
BASE_REF: ${{ inputs.base-ref }}
run: |

View File

@@ -58,7 +58,7 @@ jobs:
comment_repo: ""
steps:
- name: Setup Scripts
uses: github/gh-aw/actions/setup@08a903b1fb2e493a84a57577778fe5dd711f9468 # v0.58.3
uses: github/gh-aw/actions/setup@48d8fdfddc8cad854ac0c70ceb573f09fb8f9c9b # v0.62.5
with:
destination: /opt/gh-aw/actions
- name: Check workflow file timestamps
@@ -99,7 +99,7 @@ jobs:
secret_verification_result: ${{ steps.validate-secret.outputs.verification_result }}
steps:
- name: Setup Scripts
uses: github/gh-aw/actions/setup@08a903b1fb2e493a84a57577778fe5dd711f9468 # v0.58.3
uses: github/gh-aw/actions/setup@48d8fdfddc8cad854ac0c70ceb573f09fb8f9c9b # v0.62.5
with:
destination: /opt/gh-aw/actions
- name: Checkout repository
@@ -114,7 +114,7 @@ jobs:
run: bash /opt/gh-aw/actions/create_gh_aw_tmp_dir.sh
# Cache configuration from frontmatter processed below
- name: Cache (cpython-lib-${{ env.PYTHON_VERSION }})
uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with:
key: cpython-lib-${{ env.PYTHON_VERSION }}
path: cpython
@@ -804,7 +804,7 @@ jobs:
total_count: ${{ steps.missing_tool.outputs.total_count }}
steps:
- name: Setup Scripts
uses: github/gh-aw/actions/setup@08a903b1fb2e493a84a57577778fe5dd711f9468 # v0.58.3
uses: github/gh-aw/actions/setup@48d8fdfddc8cad854ac0c70ceb573f09fb8f9c9b # v0.62.5
with:
destination: /opt/gh-aw/actions
- name: Download agent output artifact
@@ -925,7 +925,7 @@ jobs:
success: ${{ steps.parse_results.outputs.success }}
steps:
- name: Setup Scripts
uses: github/gh-aw/actions/setup@08a903b1fb2e493a84a57577778fe5dd711f9468 # v0.58.3
uses: github/gh-aw/actions/setup@48d8fdfddc8cad854ac0c70ceb573f09fb8f9c9b # v0.62.5
with:
destination: /opt/gh-aw/actions
- name: Download agent artifacts
@@ -1037,7 +1037,7 @@ jobs:
process_safe_outputs_temporary_id_map: ${{ steps.process_safe_outputs.outputs.temporary_id_map }}
steps:
- name: Setup Scripts
uses: github/gh-aw/actions/setup@08a903b1fb2e493a84a57577778fe5dd711f9468 # v0.58.3
uses: github/gh-aw/actions/setup@48d8fdfddc8cad854ac0c70ceb573f09fb8f9c9b # v0.62.5
with:
destination: /opt/gh-aw/actions
- name: Download agent output artifact

71
.pre-commit-config.yaml Normal file
View File

@@ -0,0 +1,71 @@
# NOTE: Reason for not using `prek.toml` is dependabot supports `pre-commit` as an ecosystem
# See: https://github.blog/changelog/2026-03-10-dependabot-now-supports-pre-commit-hooks/
fail_fast: false
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-merge-conflict
priority: 0
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.7
hooks:
- id: ruff-format
priority: 0
- id: ruff-check
args: [--select, I, --fix, --exit-non-zero-on-fix]
types_or: [python]
require_serial: true
priority: 1
- repo: local
hooks:
- id: redundant-test-patches
name: check redundant test patches
entry: scripts/check_redundant_patches.py
files: '^Lib/test/.*\.py$'
language: script
types: [python]
priority: 0
- repo: local
hooks:
- id: rustfmt
name: rustfmt
entry: rustfmt
language: system
types: [rust]
priority: 0
- id: generate-opcode-metadata
name: generate opcode metadata
entry: python scripts/generate_opcode_metadata.py
files: '^(crates/compiler-core/src/bytecode/instruction\.rs|scripts/generate_opcode_metadata\.py)$'
pass_filenames: false
language: system
require_serial: true
priority: 1 # so rustfmt runs first
- repo: https://github.com/streetsidesoftware/cspell-cli
rev: v9.7.0
hooks:
- id: cspell
types: [rust]
additional_dependencies:
- '@cspell/dict-en_us'
- '@cspell/dict-cpp'
- '@cspell/dict-python'
- '@cspell/dict-rust'
- '@cspell/dict-win32'
- '@cspell/dict-shell'
priority: 0
- repo: https://github.com/rbubley/mirrors-prettier
rev: v3.8.1
hooks:
- id: prettier
files: '^wasm/.*$'
priority: 0

155
Cargo.lock generated
View File

@@ -249,9 +249,9 @@ checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "aws-lc-fips-sys"
version = "0.13.12"
version = "0.13.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ed8cd42adddefbdb8507fb7443fa9b666631078616b78f70ed22117b5c27d90"
checksum = "f8bce4948d2520386c6d92a6ea2d472300257702242e5a1d01d6add52bd2e7c1"
dependencies = [
"bindgen 0.72.1",
"cc",
@@ -263,9 +263,9 @@ dependencies = [
[[package]]
name = "aws-lc-rs"
version = "1.16.0"
version = "1.16.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9a7b350e3bb1767102698302bc37256cbd48422809984b98d292c40e2579aa9"
checksum = "a054912289d18629dc78375ba2c3726a3afe3ff71b4edba9dedfca0e3446d1fc"
dependencies = [
"aws-lc-fips-sys",
"aws-lc-sys",
@@ -275,9 +275,9 @@ dependencies = [
[[package]]
name = "aws-lc-sys"
version = "0.37.0"
version = "0.39.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c34dda4df7017c8db52132f0f8a2e0f8161649d15723ed63fc00c82d0f2081a"
checksum = "1fa7e52a4c5c547c741610a2c6f123f3881e409b714cd27e6798ef020c514f0a"
dependencies = [
"cc",
"cmake",
@@ -401,9 +401,9 @@ dependencies = [
[[package]]
name = "bumpalo"
version = "3.19.1"
version = "3.20.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5dd9dc738b7a8311c7ade152424974d8115f2cdad61e8dab8dac9f2362298510"
checksum = "5d20789868f4b01b2f2caec9f5c4e0213b41e3e5702a50157d699ae31ced2fcb"
dependencies = [
"allocator-api2",
]
@@ -712,9 +712,9 @@ dependencies = [
[[package]]
name = "cranelift"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1cc3f22b5e916179fc510801c078f0610a467939cc2809bf5eb8a06fde4aea69"
checksum = "f13b593d4c3fe30bdf7e7bf4cbe78637849515822d305da6080c7ddda554d251"
dependencies = [
"cranelift-codegen",
"cranelift-frontend",
@@ -723,45 +723,46 @@ dependencies = [
[[package]]
name = "cranelift-assembler-x64"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40630d663279bc855bff805d6f5e8a0b6a1867f9df95b010511ac6dc894e9395"
checksum = "4f248321c6a7d4de5dcf2939368e96a397ad3f53b6a076e38d0104d1da326d37"
dependencies = [
"cranelift-assembler-x64-meta",
]
[[package]]
name = "cranelift-assembler-x64-meta"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ee6aec5ceb55e5fdbcf7ef677d7c7195531360ff181ce39b2b31df11d57305f"
checksum = "ab6d78ff1f7d9bf8b7e1afbedbf78ba49e38e9da479d4c8a2db094e22f64e2bc"
dependencies = [
"cranelift-srcgen",
]
[[package]]
name = "cranelift-bforest"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a92d78cc3f087d7e7073828f08d98c7074a3a062b6b29a1b7783ce74305685e"
checksum = "6b6005ba640213a5b95382aeaf6b82bf028309581c8d7349778d66f27dc1180b"
dependencies = [
"cranelift-entity",
"wasmtime-internal-core",
]
[[package]]
name = "cranelift-bitset"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "edcc73d756f2e0d7eda6144fe64a2bc69c624de893cb1be51f1442aed77881d2"
checksum = "81fb5b134a12b559ff0c0f5af0fcd755ad380723b5016c4e0d36f74d39485340"
dependencies = [
"wasmtime-internal-core",
]
[[package]]
name = "cranelift-codegen"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d94c2cd0d73b41369b88da1129589bc3a2d99cf49979af1d14751f35b7a1b"
checksum = "85837de8be7f17a4034a6b08816f05a3144345d2091937b39d415990daca28f4"
dependencies = [
"bumpalo",
"cranelift-assembler-x64",
@@ -773,7 +774,7 @@ dependencies = [
"cranelift-entity",
"cranelift-isle",
"gimli",
"hashbrown 0.15.5",
"hashbrown 0.16.1",
"libm",
"log",
"regalloc2",
@@ -786,9 +787,9 @@ dependencies = [
[[package]]
name = "cranelift-codegen-meta"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "235da0e52ee3a0052d0e944c3470ff025b1f4234f6ec4089d3109f2d2ffa6cbd"
checksum = "e433faa87d38e5b8ff469e44a26fea4f93e58abd7a7c10bad9810056139700c9"
dependencies = [
"cranelift-assembler-x64-meta",
"cranelift-codegen-shared",
@@ -798,24 +799,24 @@ dependencies = [
[[package]]
name = "cranelift-codegen-shared"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20c07c6c440bd1bf920ff7597a1e743ede1f68dcd400730bd6d389effa7662af"
checksum = "5397ba61976e13944ca71230775db13ee1cb62849701ed35b753f4761ed0a9b7"
[[package]]
name = "cranelift-control"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8797c022e02521901e1aee483dea3ed3c67f2bf0a26405c9dd48e8ee7a70944b"
checksum = "cc81c88765580720eb30f4fc2c1bfdb75fcbf3094f87b3cd69cecca79d77a245"
dependencies = [
"arbitrary",
]
[[package]]
name = "cranelift-entity"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59d8e72637246edd2cba337939850caa8b201f6315925ec4c156fdd089999699"
checksum = "463feed5d46cf8763f3ba3045284cf706dd161496e20ec9c14afbb4ba09b9e66"
dependencies = [
"cranelift-bitset",
"wasmtime-internal-core",
@@ -823,9 +824,9 @@ dependencies = [
[[package]]
name = "cranelift-frontend"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4c31db0085c3dfa131e739c3b26f9f9c84d69a9459627aac1ac4ef8355e3411b"
checksum = "a4c5eca7696c1c04ab4c7ed8d18eadbb47d6cc9f14ec86fe0881bf1d7e97e261"
dependencies = [
"cranelift-codegen",
"log",
@@ -835,15 +836,15 @@ dependencies = [
[[package]]
name = "cranelift-isle"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "524d804c1ebd8c542e6f64e71aa36934cec17c5da4a9ae3799796220317f5d23"
checksum = "f1153844610cc9c6da8cf10ce205e45da1a585b7688ed558aa808bbe2e4e6d77"
[[package]]
name = "cranelift-jit"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02ca12808d5c1ccf40cb02493a8f1790358f230867fe37735e9af8b76a2262cb"
checksum = "41836de8321b303d3d4188e58cc09c30c7645337342acfcfb363732695cae098"
dependencies = [
"anyhow",
"cranelift-codegen",
@@ -861,9 +862,9 @@ dependencies = [
[[package]]
name = "cranelift-module"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4d92fca47132ffc3de8783e82a577a2c8aedf85d1e12b92d08863d9af8a76bd4"
checksum = "b731f66cb1b69b60a74216e632968ebdbb95c488d26aa1448ec226ae0ffec33e"
dependencies = [
"anyhow",
"cranelift-codegen",
@@ -872,9 +873,9 @@ dependencies = [
[[package]]
name = "cranelift-native"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc9598f02540e382e1772416eba18e93c5275b746adbbf06ac1f3cf149415270"
checksum = "a97b583fe9a60f06b0464cee6be5a17f623fd91b217aaac99b51b339d19911af"
dependencies = [
"cranelift-codegen",
"libc",
@@ -883,9 +884,9 @@ dependencies = [
[[package]]
name = "cranelift-srcgen"
version = "0.129.1"
version = "0.130.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d953932541249c91e3fa70a75ff1e52adc62979a2a8132145d4b9b3e6d1a9b6a"
checksum = "8594dc6bb4860fa8292f1814c76459dbfb933e1978d8222de6380efce45c7cee"
[[package]]
name = "crc32fast"
@@ -1262,12 +1263,6 @@ version = "1.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
[[package]]
name = "foldhash"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "foldhash"
version = "0.2.0"
@@ -1432,9 +1427,6 @@ name = "hashbrown"
version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"foldhash 0.1.5",
]
[[package]]
name = "hashbrown"
@@ -1442,7 +1434,7 @@ version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
dependencies = [
"foldhash 0.2.0",
"foldhash",
]
[[package]]
@@ -1715,9 +1707,9 @@ checksum = "2604dd126bb14f13fb5d1bd6a66155079cb9fa655b37f875b3a742c705dbed17"
[[package]]
name = "lexopt"
version = "0.3.1"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9fa0e2a1fcbe2f6be6c42e342259976206b383122fc152e872795338b5a3f3a7"
checksum = "803ec87c9cfb29b9d2633f20cba1f488db3fd53f2158b1024cbefb47ba05d413"
[[package]]
name = "libbz2-rs-sys"
@@ -1808,9 +1800,9 @@ dependencies = [
[[package]]
name = "libsqlite3-sys"
version = "0.36.0"
version = "0.37.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "95b4103cffefa72eb8428cb6b47d6627161e51c2739fc5e3b734584157bc642a"
checksum = "b1f111c8c41e7c61a49cd34e44c7619462967221a6443b0ec299e0ac30cfb9b1"
dependencies = [
"cc",
"pkg-config",
@@ -1849,9 +1841,9 @@ checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "lz4_flex"
version = "0.12.1"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "98c23545df7ecf1b16c303910a69b079e8e251d60f7dd2cc9b4177f2afaf1746"
checksum = "db9a0d582c2874f68138a16ce1867e0ffde6c0bb0a0df85e1f36d04146db488a"
dependencies = [
"twox-hash",
]
@@ -2797,9 +2789,9 @@ dependencies = [
[[package]]
name = "regalloc2"
version = "0.13.5"
version = "0.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08effbc1fa53aaebff69521a5c05640523fab037b34a4a2c109506bc938246fa"
checksum = "952ddbfc6f9f64d006c3efd8c9851a6ba2f2b944ba94730db255d55006e0ffda"
dependencies = [
"allocator-api2",
"bumpalo",
@@ -3052,9 +3044,9 @@ checksum = "f87165f0995f63a9fbeea62b64d10b4d9d8e78ec6d7d51fb2125fda7bb36788f"
[[package]]
name = "rustls-webpki"
version = "0.103.9"
version = "0.103.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7df23109aa6c1567d1c575b9952556388da57401e4ace1d15f79eedad0d8f53"
checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef"
dependencies = [
"aws-lc-rs",
"ring",
@@ -3623,9 +3615,9 @@ dependencies = [
[[package]]
name = "serde_spanned"
version = "1.0.4"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8bbf91e5a4d6315eee45e704372590b30e260ee83af6639d64557f51b067776"
checksum = "876ac351060d4f882bb1032b6369eb0aef79ad9df1ea8bc404874d8cc3d0cd98"
dependencies = [
"serde_core",
]
@@ -3760,9 +3752,9 @@ checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f"
[[package]]
name = "strum"
version = "0.27.2"
version = "0.28.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "af23d6f6c1a224baef9d3f61e287d2761385a5b88fdab4eb4c6f11aeb54c4bcf"
checksum = "9628de9b8791db39ceda2b119bbe13134770b56c138ec1d3af810d045c04f9bd"
[[package]]
name = "strum_macros"
@@ -4024,9 +4016,9 @@ dependencies = [
[[package]]
name = "toml"
version = "0.9.11+spec-1.1.0"
version = "1.1.0+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3afc9a848309fe1aaffaed6e1546a7a14de1f935dc9d89d32afd9a44bab7c46"
checksum = "f8195ca05e4eb728f4ba94f3e3291661320af739c4e43779cbdfae82ab239fcc"
dependencies = [
"indexmap",
"serde_core",
@@ -4039,27 +4031,27 @@ dependencies = [
[[package]]
name = "toml_datetime"
version = "0.7.5+spec-1.1.0"
version = "1.1.0+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92e1cfed4a3038bc5a127e35a2d360f145e1f4b971b551a2ba5fd7aedf7e1347"
checksum = "97251a7c317e03ad83774a8752a7e81fb6067740609f75ea2b585b569a59198f"
dependencies = [
"serde_core",
]
[[package]]
name = "toml_parser"
version = "1.0.6+spec-1.1.0"
version = "1.1.0+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3198b4b0a8e11f09dd03e133c0280504d0801269e9afa46362ffde1cbeebf44"
checksum = "2334f11ee363607eb04df9b8fc8a13ca1715a72ba8662a26ac285c98aabb4011"
dependencies = [
"winnow",
]
[[package]]
name = "toml_writer"
version = "1.0.6+spec-1.1.0"
version = "1.1.0+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab16f14aed21ee8bfd8ec22513f7287cd4a91aa92e44edfe2c17ddd004e92607"
checksum = "d282ade6016312faf3e41e57ebbba0c073e4056dab1232ab1cb624199648f8ed"
[[package]]
name = "twox-hash"
@@ -4401,18 +4393,19 @@ dependencies = [
[[package]]
name = "wasmtime-internal-core"
version = "42.0.1"
version = "43.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03a4a3f055a804a2f3d86e816a9df78a8fa57762212a8506164959224a40cd48"
checksum = "e671917bb6856ae360cb59d7aaf26f1cfd042c7b924319dd06fd380739fc0b2e"
dependencies = [
"hashbrown 0.16.1",
"libm",
]
[[package]]
name = "wasmtime-internal-jit-icache-coherence"
version = "42.0.1"
version = "43.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c57d24e8d1334a0e5a8b600286ffefa1fc4c3e8176b110dff6fbc1f43c4a599b"
checksum = "9b3112806515fac8495883885eb8dbdde849988ae91fe6beb544c0d7c0f4c9aa"
dependencies = [
"cfg-if",
"libc",
@@ -4796,15 +4789,15 @@ checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650"
[[package]]
name = "winnow"
version = "0.7.14"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a5364e9d77fcdeeaa6062ced926ee3381faa2ee02d3eb83a5c27a8825540829"
checksum = "a90e88e4667264a994d34e6d1ab2d26d398dcdca8b7f52bec8668957517fc7d8"
[[package]]
name = "winresource"
version = "0.1.30"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e287ced0f21cd11f4035fe946fd3af145f068d1acb708afd248100f89ec7432d"
checksum = "0986a8b1d586b7d3e4fe3d9ea39fb451ae22869dcea4aa109d287a374d866087"
dependencies = [
"toml",
"version_check",

View File

@@ -211,7 +211,7 @@ schannel = "0.1.28"
scoped-tls = "1"
scopeguard = "1"
static_assertions = "1.1"
strum = "0.27"
strum = "0.28"
strum_macros = "0.28"
syn = "2"
thiserror = "2.0"

View File

@@ -163,7 +163,6 @@ class OSEINTRTest(EINTRBaseTest):
self.assertEqual(os.readinto(fd, buffer), len(expected))
self.assertEqual(buffer, expected)
@unittest.expectedFailure # TODO: RUSTPYTHON; InterruptedError: [Errno 4] Interrupted system call
def test_write(self):
rd, wr = os.pipe()
self.addCleanup(os.close, wr)

View File

@@ -4813,9 +4813,9 @@ class _TestFinalize(BaseTestCase):
result = [obj for obj in iter(conn.recv, 'STOP')]
self.assertEqual(result, ['a', 'b', 'd10', 'd03', 'd02', 'd01', 'e'])
# TODO: RUSTPYTHON; SIGSEGV due to dict thread-safety issue under aggressive GC
@unittest.skip("TODO: RUSTPYTHON")
@support.requires_resource('cpu')
# TODO: RUSTPYTHON; dict iteration races with concurrent GC mutations
@unittest.expectedFailure
def test_thread_safety(self):
# bpo-24484: _run_finalizers() should be thread-safe
def cb():

View File

@@ -475,8 +475,6 @@ class CmdLineTest(unittest.TestCase):
self.assertRegex(err.decode('ascii', 'ignore'), 'SyntaxError')
self.assertEqual(b'', out)
# TODO: RUSTPYTHON
@unittest.expectedFailure
def test_stdout_flush_at_shutdown(self):
# Issue #5319: if stdout.flush() fails at shutdown, an error should
# be printed out.

View File

@@ -2486,7 +2486,6 @@ class TestSourcePositions(unittest.TestCase):
class TestStaticAttributes(unittest.TestCase):
@unittest.expectedFailure # TODO: RUSTPYTHON; AttributeError: type object 'C' has no attribute '__static_attributes__'
def test_basic(self):
class C:
def f(self):
@@ -2518,7 +2517,6 @@ class TestStaticAttributes(unittest.TestCase):
self.assertEqual(sorted(C.__static_attributes__), ['u', 'v', 'x', 'y', 'z'])
@unittest.expectedFailure # TODO: RUSTPYTHON; AttributeError: type object 'C' has no attribute '__static_attributes__'
def test_nested_class(self):
class C:
def f(self):
@@ -2533,7 +2531,6 @@ class TestStaticAttributes(unittest.TestCase):
self.assertEqual(sorted(C.__static_attributes__), ['x', 'y'])
self.assertEqual(sorted(C.D.__static_attributes__), ['y', 'z'])
@unittest.expectedFailure # TODO: RUSTPYTHON; AttributeError: type object 'C' has no attribute '__static_attributes__'
def test_subclass(self):
class C:
def f(self):
@@ -2593,7 +2590,6 @@ class TestExpressionStackSize(unittest.TestCase):
def test_set(self):
self.check_stack_size("{" + "x, " * self.N + "x}")
@unittest.expectedFailure # TODO: RUSTPYTHON; AssertionError: 202 not less than or equal to 7
def test_dict(self):
self.check_stack_size("{" + "x:x, " * self.N + "x:x}")

View File

@@ -4987,7 +4987,6 @@ class ClassPropertiesAndMethods(unittest.TestCase):
self.assertEqual(Y.__qualname__, 'Y')
self.assertEqual(Y.Inside.__qualname__, 'Y.Inside')
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_qualname_dict(self):
ns = {'__qualname__': 'some.name'}
tp = type('Foo', (), ns)
@@ -5130,7 +5129,6 @@ class ClassPropertiesAndMethods(unittest.TestCase):
gc.collect()
self.assertEqual(Parent.__subclasses__(), [])
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_instance_method_get_behavior(self):
# test case for gh-113157
@@ -5180,7 +5178,6 @@ class DictProxyTests(unittest.TestCase):
pass
self.C = C
@unittest.expectedFailure # TODO: RUSTPYTHON
@unittest.skipIf(hasattr(sys, 'gettrace') and sys.gettrace(),
'trace function introduces __local__')
def test_iter_keys(self):
@@ -5194,7 +5191,6 @@ class DictProxyTests(unittest.TestCase):
'__static_attributes__', '__weakref__',
'meth'])
@unittest.expectedFailure # TODO: RUSTPYTHON; AssertionError: 5 != 7
@unittest.skipIf(hasattr(sys, 'gettrace') and sys.gettrace(),
'trace function introduces __local__')
def test_iter_values(self):
@@ -5204,7 +5200,6 @@ class DictProxyTests(unittest.TestCase):
values = list(it)
self.assertEqual(len(values), 7)
@unittest.expectedFailure # TODO: RUSTPYTHON
@unittest.skipIf(hasattr(sys, 'gettrace') and sys.gettrace(),
'trace function introduces __local__')
def test_iter_items(self):

View File

@@ -1134,7 +1134,6 @@ class DisTests(DisTestBase):
# Test that value is displayed for keyword argument names:
self.do_disassembly_test(wrap_func_w_kwargs, dis_kw_names)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_intrinsic_1(self):
# Test that argrepr is displayed for CALL_INTRINSIC_1
self.do_disassembly_test("from math import *", dis_intrinsic_1_2)

View File

@@ -385,7 +385,6 @@ class PluralFormsTests:
x = ngettext(singular, plural, None)
self.assertEqual(x, tplural)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_plural_forms(self):
self._test_plural_forms(
self.ngettext, self.gettext,
@@ -396,7 +395,6 @@ class PluralFormsTests:
'%d file deleted', '%d files deleted',
'%d file deleted', '%d files deleted')
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_plural_context_forms(self):
ngettext = partial(self.npgettext, 'With context')
gettext = partial(self.pgettext, 'With context')
@@ -409,7 +407,6 @@ class PluralFormsTests:
'%d file deleted', '%d files deleted',
'%d file deleted', '%d files deleted')
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_plural_wrong_context_forms(self):
self._test_plural_forms(
partial(self.npgettext, 'Unknown context'),
@@ -442,7 +439,6 @@ class GNUTranslationsWithDomainPluralFormsTestCase(PluralFormsTests, GettextBase
self.pgettext = partial(gettext.dpgettext, 'gettext')
self.npgettext = partial(gettext.dnpgettext, 'gettext')
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_plural_forms_wrong_domain(self):
self._test_plural_forms(
partial(gettext.dngettext, 'unknown'),
@@ -451,7 +447,6 @@ class GNUTranslationsWithDomainPluralFormsTestCase(PluralFormsTests, GettextBase
'There is %s file', 'There are %s files',
numbers_only=False)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_plural_context_forms_wrong_domain(self):
self._test_plural_forms(
partial(gettext.dnpgettext, 'unknown', 'With context'),
@@ -472,7 +467,6 @@ class GNUTranslationsClassPluralFormsTestCase(PluralFormsTests, GettextBaseTest)
self.pgettext = t.pgettext
self.npgettext = t.npgettext
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_plural_forms_null_translations(self):
t = gettext.NullTranslations()
self._test_plural_forms(
@@ -481,7 +475,6 @@ class GNUTranslationsClassPluralFormsTestCase(PluralFormsTests, GettextBaseTest)
'There is %s file', 'There are %s files',
numbers_only=False)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_plural_context_forms_null_translations(self):
t = gettext.NullTranslations()
self._test_plural_forms(

View File

@@ -961,7 +961,6 @@ class TestGettingSourceOfToplevelFrames(GetSourceBase):
class TestDecorators(GetSourceBase):
fodderModule = mod2
@unittest.expectedFailure # TODO: RUSTPYTHON; pass
def test_wrapped_decorator(self):
self.assertSourceEqual(mod2.wrapped, 14, 17)
@@ -1259,7 +1258,6 @@ class TestNoEOL(GetSourceBase):
class TestComplexDecorator(GetSourceBase):
fodderModule = mod2
@unittest.expectedFailure # TODO: RUSTPYTHON; return foo + bar()
def test_parens_in_decorator(self):
self.assertSourceEqual(self.fodderModule.complex_decorated, 273, 275)

View File

@@ -49,7 +49,6 @@ class IntTestCase(unittest.TestCase, HelperMixin):
self.helper(expected)
n = n >> 1
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_int64(self):
# Simulate int marshaling with TYPE_INT64.
maxint64 = (1 << 63) - 1
@@ -141,7 +140,6 @@ class CodeTestCase(unittest.TestCase):
self.assertEqual(co1.co_filename, "f1")
self.assertEqual(co2.co_filename, "f2")
@unittest.expectedFailure # TODO: RUSTPYTHON; TypeError: Unexpected keyword argument allow_code
def test_no_allow_code(self):
data = {'a': [({0},)]}
dump = marshal.dumps(data, allow_code=False)
@@ -234,14 +232,12 @@ class BufferTestCase(unittest.TestCase, HelperMixin):
new = marshal.loads(marshal.dumps(b))
self.assertEqual(type(new), bytes)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_memoryview(self):
b = memoryview(b"abc")
self.helper(b)
new = marshal.loads(marshal.dumps(b))
self.assertEqual(type(new), bytes)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_array(self):
a = array.array('B', b"abc")
new = marshal.loads(marshal.dumps(a))
@@ -274,7 +270,6 @@ class BugsTestCase(unittest.TestCase):
except Exception:
pass
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_loads_recursion(self):
def run_tests(N, check):
# (((...None...),),)
@@ -295,7 +290,7 @@ class BugsTestCase(unittest.TestCase):
run_tests(2**20, check)
@unittest.skipIf(support.is_android, "TODO: RUSTPYTHON; segfault")
@unittest.expectedFailure # TODO: RUSTPYTHON; segfault
@unittest.skipIf(os.name == 'nt', "TODO: RUSTPYTHON; write depth limit is 2000 not 1000")
def test_recursion_limit(self):
# Create a deeply nested structure.
head = last = []
@@ -324,7 +319,6 @@ class BugsTestCase(unittest.TestCase):
last.append([0])
self.assertRaises(ValueError, marshal.dumps, head)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_exact_type_match(self):
# Former bug:
# >>> class Int(int): pass
@@ -348,7 +342,6 @@ class BugsTestCase(unittest.TestCase):
invalid_string = b'l\x02\x00\x00\x00\x00\x00\x00\x00'
self.assertRaises(ValueError, marshal.loads, invalid_string)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_multiple_dumps_and_loads(self):
# Issue 12291: marshal.load() should be callable multiple times
# with interleaved data written by non-marshal code
@@ -532,66 +525,56 @@ class InstancingTestCase(unittest.TestCase, HelperMixin):
else:
self.assertGreaterEqual(len(s2), len(s3))
@unittest.expectedFailure # TODO: RUSTPYTHON
def testInt(self):
intobj = 123321
self.helper(intobj)
self.helper3(intobj, simple=True)
@unittest.expectedFailure # TODO: RUSTPYTHON
def testFloat(self):
floatobj = 1.2345
self.helper(floatobj)
self.helper3(floatobj)
@unittest.expectedFailure # TODO: RUSTPYTHON
def testStr(self):
strobj = "abcde"*3
self.helper(strobj)
self.helper3(strobj)
@unittest.expectedFailure # TODO: RUSTPYTHON
def testBytes(self):
bytesobj = b"abcde"*3
self.helper(bytesobj)
self.helper3(bytesobj)
@unittest.expectedFailure # TODO: RUSTPYTHON
def testList(self):
for obj in self.keys:
listobj = [obj, obj]
self.helper(listobj)
self.helper3(listobj)
@unittest.expectedFailure # TODO: RUSTPYTHON
def testTuple(self):
for obj in self.keys:
tupleobj = (obj, obj)
self.helper(tupleobj)
self.helper3(tupleobj)
@unittest.expectedFailure # TODO: RUSTPYTHON
def testSet(self):
for obj in self.keys:
setobj = {(obj, 1), (obj, 2)}
self.helper(setobj)
self.helper3(setobj)
@unittest.expectedFailure # TODO: RUSTPYTHON
def testFrozenSet(self):
for obj in self.keys:
frozensetobj = frozenset({(obj, 1), (obj, 2)})
self.helper(frozensetobj)
self.helper3(frozensetobj)
@unittest.expectedFailure # TODO: RUSTPYTHON
def testDict(self):
for obj in self.keys:
dictobj = {"hello": obj, "goodbye": obj, obj: "hello"}
self.helper(dictobj)
self.helper3(dictobj)
@unittest.expectedFailure # TODO: RUSTPYTHON
def testModule(self):
with open(__file__, "rb") as f:
code = f.read()
@@ -651,7 +634,6 @@ class InterningTestCase(unittest.TestCase, HelperMixin):
self.assertNotEqual(id(s2), id(s))
class SliceTestCase(unittest.TestCase, HelperMixin):
@unittest.expectedFailure # TODO: RUSTPYTHON; NotImplementedError: TODO: not implemented yet or marshal unsupported type
def test_slice(self):
for obj in (
slice(None), slice(1), slice(1, 2), slice(1, 2, 3),

View File

@@ -867,7 +867,6 @@ class MmapTests(unittest.TestCase):
finally:
f.close()
@unittest.expectedFailure # TODO: RUSTPYTHON
@unittest.skipUnless(os.name == 'nt', 'requires Windows')
def test_resize_succeeds_with_error_for_second_named_mapping(self):
"""If a more than one mapping exists of the same name, none of them can

View File

@@ -612,7 +612,6 @@ class TestTranforms(BytecodeTestCase):
print(i)
self.check_jump_targets(f)
@unittest.expectedFailure # TODO: RUSTPYTHON; 611 JUMP_BACKWARD 16
def test_elim_jump_after_return1(self):
# Eliminate dead code: jumps immediately after returns can't be reached
def f(cond1, cond2):
@@ -646,7 +645,6 @@ class TestTranforms(BytecodeTestCase):
self.assertEqual(count_instr_recursively(containtest, 'BUILD_LIST'), 0)
self.check_lnotab(containtest)
@unittest.expectedFailure # TODO: RUSTPYTHON; no BUILD_LIST to BUILD_TUPLE optimization
def test_iterate_literal_list(self):
def forloop():
for x in [a, b]:
@@ -863,7 +861,6 @@ class TestMarkingVariablesAsUnKnown(BytecodeTestCase):
self.addCleanup(sys.settrace, sys.gettrace())
sys.settrace(None)
@unittest.expectedFailure # TODO: RUSTPYTHON; BINARY_OP 0 (+)
def test_load_fast_known_simple(self):
def f():
x = 1

View File

@@ -2414,7 +2414,6 @@ class StrTest(string_tests.StringLikeTest,
else:
self.fail("Should have raised UnicodeDecodeError")
@unittest.expectedFailure # TODO: RUSTPYTHON; AssertionError: <class 'str'> is not <class 'test.test_str.StrSubclass'>
def test_conversion(self):
# Make sure __str__() works properly
class StrWithStr(str):

View File

@@ -1903,7 +1903,6 @@ class RunFuncTestCase(BaseTestCase):
res = subprocess.run(args)
self.assertEqual(res.returncode, 57)
@unittest.skipIf(mswindows, "TODO: RUSTPYTHON; empty env block fails nondeterministically")
@unittest.skipUnless(mswindows, "Maybe test trigger a leak on Ubuntu")
def test_run_with_an_empty_env(self):
# gh-105436: fix subprocess.run(..., env={}) broken on Windows

View File

@@ -209,7 +209,6 @@ class TestSuper(unittest.TestCase):
self.assertIs(test_class, A)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test___classcell___expected_behaviour(self):
# See issue #23722
class Meta(type):

View File

@@ -878,7 +878,6 @@ class SysModuleTest(unittest.TestCase):
def test_sys_version_info_no_instantiation(self):
self.assert_raise_on_new_sys_type(sys.version_info)
@unittest.expectedFailure # TODO: RUSTPYTHON; TypeError not raised for getwindowsversion instantiation
def test_sys_getwindowsversion_no_instantiation(self):
# Skip if not being run on Windows.
test.support.get_attribute(sys, "getwindowsversion")

View File

@@ -1420,8 +1420,6 @@ class JumpTestCase(unittest.TestCase):
output.append(6)
output.append(7)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(4, 5, [3, 5])
async def test_jump_out_of_async_for_block_forwards(output):
for i in [1]:
@@ -1430,8 +1428,6 @@ class JumpTestCase(unittest.TestCase):
output.append(4)
output.append(5)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(5, 2, [2, 4, 2, 4, 5, 6])
async def test_jump_out_of_async_for_block_backwards(output):
for i in [1]:
@@ -1539,8 +1535,6 @@ class JumpTestCase(unittest.TestCase):
output.append(2)
output.append(3)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(2, 3, [1, 3])
async def test_jump_forwards_out_of_async_with_block(output):
async with asynctracecontext(output, 1):
@@ -1553,8 +1547,6 @@ class JumpTestCase(unittest.TestCase):
with tracecontext(output, 2):
output.append(3)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(3, 1, [1, 2, 1, 2, 3, -2])
async def test_jump_backwards_out_of_async_with_block(output):
output.append(1)
@@ -1624,8 +1616,6 @@ class JumpTestCase(unittest.TestCase):
with tracecontext(output, 4):
output.append(5)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(2, 4, [1, 4, 5, -4])
async def test_jump_across_async_with(output):
output.append(1)
@@ -1643,8 +1633,6 @@ class JumpTestCase(unittest.TestCase):
output.append(5)
output.append(6)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(4, 5, [1, 3, 5, 6])
async def test_jump_out_of_async_with_block_within_for_block(output):
output.append(1)
@@ -1663,8 +1651,6 @@ class JumpTestCase(unittest.TestCase):
output.append(5)
output.append(6)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(4, 5, [1, 2, 3, 5, -2, 6])
async def test_jump_out_of_async_with_block_within_with_block(output):
output.append(1)
@@ -1684,8 +1670,6 @@ class JumpTestCase(unittest.TestCase):
output.append(6)
output.append(7)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(5, 6, [2, 4, 6, 7])
async def test_jump_out_of_async_with_block_within_finally_block(output):
try:
@@ -1719,8 +1703,6 @@ class JumpTestCase(unittest.TestCase):
output.append(4)
output.append(5)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(3, 5, [1, 2, 5])
async def test_jump_out_of_async_with_assignment(output):
output.append(1)
@@ -1768,8 +1750,6 @@ class JumpTestCase(unittest.TestCase):
output.append(7)
output.append(8)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(1, 7, [7, 8])
async def test_jump_over_async_for_block_before_else(output):
output.append(1)
@@ -2053,8 +2033,6 @@ class JumpTestCase(unittest.TestCase):
with tracecontext(output, 4):
output.append(5)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@async_jump_test(3, 5, [1, 2, 5, -2])
async def test_jump_between_async_with_blocks(output):
output.append(1)
@@ -2063,8 +2041,6 @@ class JumpTestCase(unittest.TestCase):
async with asynctracecontext(output, 4):
output.append(5)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@jump_test(5, 7, [2, 4], (ValueError, "after"))
def test_no_jump_over_return_out_of_finally_block(output):
try:

File diff suppressed because it is too large Load Diff

View File

@@ -8,9 +8,10 @@ use num_traits::ToPrimitive;
use rustpython_compiler_core::{
OneIndexed, SourceLocation,
bytecode::{
AnyInstruction, Arg, CodeFlags, CodeObject, CodeUnit, CodeUnits, ConstantData,
ExceptionTableEntry, InstrDisplayContext, Instruction, InstructionMetadata, Label, OpArg,
PseudoInstruction, PyCodeLocationInfoKind, encode_exception_table, oparg,
AnyInstruction, Arg, CO_FAST_CELL, CO_FAST_FREE, CO_FAST_HIDDEN, CO_FAST_LOCAL, CodeFlags,
CodeObject, CodeUnit, CodeUnits, ConstantData, ExceptionTableEntry, InstrDisplayContext,
Instruction, InstructionMetadata, Label, OpArg, PseudoInstruction, PyCodeLocationInfoKind,
encode_exception_table, oparg,
},
varint::{write_signed_varint, write_varint},
};
@@ -188,16 +189,22 @@ impl CodeInfo {
mut self,
opts: &crate::compile::CompileOpts,
) -> crate::InternalResult<CodeObject> {
// Always fold tuple constants
// Constant folding passes
self.fold_unary_negative();
self.remove_nops(); // remove NOPs from unary folding so tuple/list/set see contiguous LOADs
self.fold_tuple_constants();
self.fold_list_constants();
self.fold_set_constants();
self.remove_nops(); // remove NOPs from collection folding
self.fold_const_iterable_for_iter();
self.convert_to_load_small_int();
self.remove_unused_consts();
self.remove_nops();
if opts.optimize > 0 {
self.dce();
self.peephole_optimize();
}
// DCE always runs (removes dead code after terminal instructions)
self.dce();
// Peephole optimizer creates superinstructions matching CPython
self.peephole_optimize();
// Always apply LOAD_FAST_BORROW optimization
self.optimize_load_fast_borrow();
@@ -207,10 +214,14 @@ impl CodeInfo {
label_exception_targets(&mut self.blocks);
push_cold_blocks_to_end(&mut self.blocks);
normalize_jumps(&mut self.blocks);
self.dce(); // re-run within-block DCE after normalize_jumps creates new instructions
self.eliminate_unreachable_blocks();
duplicate_end_returns(&mut self.blocks);
self.dce(); // truncate after terminal in blocks that got return duplicated
self.eliminate_unreachable_blocks(); // remove now-unreachable last block
self.optimize_load_global_push_null();
let max_stackdepth = self.max_stackdepth()?;
let cell2arg = self.cell2arg();
let Self {
flags,
@@ -236,7 +247,7 @@ impl CodeInfo {
varnames: varname_cache,
cellvars: cellvar_cache,
freevars: freevar_cache,
fast_hidden: _,
fast_hidden,
argcount: arg_count,
posonlyargcount: posonlyarg_count,
kwonlyargcount: kwonlyarg_count,
@@ -247,8 +258,12 @@ impl CodeInfo {
let mut locations = Vec::new();
let mut linetable_locations: Vec<LineTableLocation> = Vec::new();
// Convert pseudo ops and remove resulting NOPs (keep line-marker NOPs)
convert_pseudo_ops(&mut blocks, varname_cache.len() as u32);
// Build cellfixedoffsets for cell-local merging
let cellfixedoffsets =
build_cellfixedoffsets(&varname_cache, &cellvar_cache, &freevar_cache);
// Convert pseudo ops (LoadClosure uses cellfixedoffsets) and fixup DEREF opargs
convert_pseudo_ops(&mut blocks, &cellfixedoffsets);
fixup_deref_opargs(&mut blocks, &cellfixedoffsets);
// Remove redundant NOPs, keeping line-marker NOPs only when
// they are needed to preserve tracing.
let mut block_order = Vec::new();
@@ -327,6 +342,18 @@ impl CodeInfo {
blocks[bi].instructions = kept;
}
// Final DCE: truncate instructions after terminal ops in linearized blocks.
// This catches dead code created by normalize_jumps after the initial DCE.
for block in blocks.iter_mut() {
if let Some(pos) = block
.instructions
.iter()
.position(|ins| ins.instr.is_scope_exit() || ins.instr.is_unconditional_jump())
{
block.instructions.truncate(pos + 1);
}
}
// Pre-compute cache_entries for real (non-pseudo) instructions
for block in blocks.iter_mut() {
for instr in &mut block.instructions {
@@ -336,7 +363,7 @@ impl CodeInfo {
}
}
let mut block_to_offset = vec![Label::new(0); blocks.len()];
let mut block_to_offset = vec![Label::from_u32(0); blocks.len()];
// block_to_index: maps block idx to instruction index (for exception table)
// This is the index into the final instructions array, including EXTENDED_ARG and CACHE
let mut block_to_index = vec![0u32; blocks.len()];
@@ -345,7 +372,7 @@ impl CodeInfo {
loop {
let mut num_instructions = 0;
for (idx, block) in iter_blocks(&blocks) {
block_to_offset[idx.idx()] = Label::new(num_instructions as u32);
block_to_offset[idx.idx()] = Label::from_u32(num_instructions as u32);
// block_to_index uses the same value as block_to_offset but as u32
// because lasti in frame.rs is the index into instructions array
// and instructions array index == byte offset (each instruction is 1 CodeUnit)
@@ -482,6 +509,41 @@ impl CodeInfo {
// Generate exception table before moving source_path
let exceptiontable = generate_exception_table(&blocks, &block_to_index);
// Build localspluskinds with cell-local merging
let nlocals = varname_cache.len();
let ncells = cellvar_cache.len();
let nfrees = freevar_cache.len();
let numdropped = cellvar_cache
.iter()
.filter(|cv| varname_cache.contains(cv.as_str()))
.count();
let nlocalsplus = nlocals + ncells - numdropped + nfrees;
let mut localspluskinds = vec![0u8; nlocalsplus];
// Mark locals
for kind in localspluskinds.iter_mut().take(nlocals) {
*kind = CO_FAST_LOCAL;
}
// Mark cells (merged and non-merged)
for (i, cellvar) in cellvar_cache.iter().enumerate() {
let idx = cellfixedoffsets[i] as usize;
if varname_cache.contains(cellvar.as_str()) {
localspluskinds[idx] |= CO_FAST_CELL; // merged: LOCAL | CELL
} else {
localspluskinds[idx] = CO_FAST_CELL;
}
}
// Mark frees
for i in 0..nfrees {
let idx = cellfixedoffsets[ncells + i] as usize;
localspluskinds[idx] = CO_FAST_FREE;
}
// Apply CO_FAST_HIDDEN for inlined comprehension variables
for (name, &hidden) in &fast_hidden {
if hidden && let Some(idx) = varname_cache.get_index_of(name.as_str()) {
localspluskinds[idx] |= CO_FAST_HIDDEN;
}
}
Ok(CodeObject {
flags,
posonlyarg_count,
@@ -500,44 +562,14 @@ impl CodeInfo {
varnames: varname_cache.into_iter().collect(),
cellvars: cellvar_cache.into_iter().collect(),
freevars: freevar_cache.into_iter().collect(),
cell2arg,
localspluskinds: localspluskinds.into_boxed_slice(),
linetable,
exceptiontable,
})
}
fn cell2arg(&self) -> Option<Box<[i32]>> {
if self.metadata.cellvars.is_empty() {
return None;
}
let total_args = self.metadata.argcount
+ self.metadata.kwonlyargcount
+ self.flags.contains(CodeFlags::VARARGS) as u32
+ self.flags.contains(CodeFlags::VARKEYWORDS) as u32;
let mut found_cellarg = false;
let cell2arg = self
.metadata
.cellvars
.iter()
.map(|var| {
self.metadata
.varnames
.get_index_of(var)
// check that it's actually an arg
.filter(|i| *i < total_args as usize)
.map_or(-1, |i| {
found_cellarg = true;
i as i32
})
})
.collect::<Box<[_]>>();
if found_cellarg { Some(cell2arg) } else { None }
}
fn dce(&mut self) {
// Truncate instructions after terminal instructions within each block
for block in &mut self.blocks {
let mut last_instr = None;
for (i, ins) in block.instructions.iter().enumerate() {
@@ -552,6 +584,109 @@ impl CodeInfo {
}
}
/// Clear blocks that are unreachable (not entry, not a jump target,
/// and only reachable via fall-through from a terminal block).
fn eliminate_unreachable_blocks(&mut self) {
let mut reachable = vec![false; self.blocks.len()];
reachable[0] = true;
// Fixpoint: only mark targets of already-reachable blocks
let mut changed = true;
while changed {
changed = false;
for i in 0..self.blocks.len() {
if !reachable[i] {
continue;
}
// Mark jump targets and exception handlers
for ins in &self.blocks[i].instructions {
if ins.target != BlockIdx::NULL && !reachable[ins.target.idx()] {
reachable[ins.target.idx()] = true;
changed = true;
}
if let Some(eh) = &ins.except_handler
&& !reachable[eh.handler_block.idx()]
{
reachable[eh.handler_block.idx()] = true;
changed = true;
}
}
// Mark fall-through
let next = self.blocks[i].next;
if next != BlockIdx::NULL
&& !reachable[next.idx()]
&& !self.blocks[i].instructions.last().is_some_and(|ins| {
ins.instr.is_scope_exit() || ins.instr.is_unconditional_jump()
})
{
reachable[next.idx()] = true;
changed = true;
}
}
}
for (i, block) in self.blocks.iter_mut().enumerate() {
if !reachable[i] {
block.instructions.clear();
}
}
}
/// Fold LOAD_CONST/LOAD_SMALL_INT + UNARY_NEGATIVE → LOAD_CONST (negative value)
fn fold_unary_negative(&mut self) {
for block in &mut self.blocks {
let mut i = 0;
while i + 1 < block.instructions.len() {
let next = &block.instructions[i + 1];
let Some(Instruction::UnaryNegative) = next.instr.real() else {
i += 1;
continue;
};
let curr = &block.instructions[i];
let value = match curr.instr.real() {
Some(Instruction::LoadConst { .. }) => {
let idx = u32::from(curr.arg) as usize;
match self.metadata.consts.get_index(idx) {
Some(ConstantData::Integer { value }) => {
Some(ConstantData::Integer { value: -value })
}
Some(ConstantData::Float { value }) => {
Some(ConstantData::Float { value: -value })
}
_ => None,
}
}
Some(Instruction::LoadSmallInt { .. }) => {
let v = u32::from(curr.arg) as i32;
Some(ConstantData::Integer {
value: BigInt::from(-v),
})
}
_ => None,
};
if let Some(neg_const) = value {
let (const_idx, _) = self.metadata.consts.insert_full(neg_const);
// Replace LOAD_CONST/LOAD_SMALL_INT with new LOAD_CONST
let load_location = block.instructions[i].location;
block.instructions[i].instr = Instruction::LoadConst {
consti: Arg::marker(),
}
.into();
block.instructions[i].arg = OpArg::new(const_idx as u32);
// Replace UNARY_NEGATIVE with NOP, inheriting the LOAD_CONST
// location so that remove_nops can clean it up
block.instructions[i + 1].instr = Instruction::Nop.into();
block.instructions[i + 1].location = load_location;
block.instructions[i + 1].end_location = block.instructions[i].end_location;
// Skip the NOP, don't re-check
i += 2;
} else {
i += 1;
}
}
}
}
/// Constant folding: fold LOAD_CONST/LOAD_SMALL_INT + BUILD_TUPLE into LOAD_CONST tuple
/// fold_tuple_of_constants
fn fold_tuple_constants(&mut self) {
@@ -566,7 +701,20 @@ impl CodeInfo {
};
let tuple_size = u32::from(instr.arg) as usize;
if tuple_size == 0 || i < tuple_size {
if tuple_size == 0 {
// BUILD_TUPLE 0 → LOAD_CONST ()
let (const_idx, _) = self.metadata.consts.insert_full(ConstantData::Tuple {
elements: Vec::new(),
});
block.instructions[i].instr = Instruction::LoadConst {
consti: Arg::marker(),
}
.into();
block.instructions[i].arg = OpArg::new(const_idx as u32);
i += 1;
continue;
}
if i < tuple_size {
i += 1;
continue;
}
@@ -637,6 +785,238 @@ impl CodeInfo {
}
}
/// Fold constant list literals: LOAD_CONST* + BUILD_LIST N →
/// BUILD_LIST 0 + LOAD_CONST (tuple) + LIST_EXTEND 1
fn fold_list_constants(&mut self) {
for block in &mut self.blocks {
let mut i = 0;
while i < block.instructions.len() {
let instr = &block.instructions[i];
let Some(Instruction::BuildList { .. }) = instr.instr.real() else {
i += 1;
continue;
};
let list_size = u32::from(instr.arg) as usize;
if list_size == 0 || i < list_size {
i += 1;
continue;
}
let start_idx = i - list_size;
let mut elements = Vec::with_capacity(list_size);
let mut all_const = true;
for j in start_idx..i {
let load_instr = &block.instructions[j];
match load_instr.instr.real() {
Some(Instruction::LoadConst { .. }) => {
let const_idx = u32::from(load_instr.arg) as usize;
if let Some(constant) =
self.metadata.consts.get_index(const_idx).cloned()
{
elements.push(constant);
} else {
all_const = false;
break;
}
}
Some(Instruction::LoadSmallInt { .. }) => {
let value = u32::from(load_instr.arg) as i32;
elements.push(ConstantData::Integer {
value: BigInt::from(value),
});
}
_ => {
all_const = false;
break;
}
}
}
if !all_const || list_size < 3 {
i += 1;
continue;
}
let tuple_const = ConstantData::Tuple { elements };
let (const_idx, _) = self.metadata.consts.insert_full(tuple_const);
let folded_loc = block.instructions[i].location;
let end_loc = block.instructions[i].end_location;
let eh = block.instructions[i].except_handler;
// slot[start_idx] → BUILD_LIST 0
block.instructions[start_idx].instr = Instruction::BuildList {
count: Arg::marker(),
}
.into();
block.instructions[start_idx].arg = OpArg::new(0);
block.instructions[start_idx].location = folded_loc;
block.instructions[start_idx].end_location = end_loc;
block.instructions[start_idx].except_handler = eh;
// slot[start_idx+1] → LOAD_CONST (tuple)
block.instructions[start_idx + 1].instr = Instruction::LoadConst {
consti: Arg::marker(),
}
.into();
block.instructions[start_idx + 1].arg = OpArg::new(const_idx as u32);
block.instructions[start_idx + 1].location = folded_loc;
block.instructions[start_idx + 1].end_location = end_loc;
block.instructions[start_idx + 1].except_handler = eh;
// NOP the rest
for j in (start_idx + 2)..i {
block.instructions[j].instr = Instruction::Nop.into();
block.instructions[j].location = folded_loc;
}
// slot[i] (was BUILD_LIST) → LIST_EXTEND 1
block.instructions[i].instr = Instruction::ListExtend { i: Arg::marker() }.into();
block.instructions[i].arg = OpArg::new(1);
i += 1;
}
}
}
/// Convert constant list/set construction before GET_ITER to just LOAD_CONST tuple.
/// BUILD_LIST 0 + LOAD_CONST (tuple) + LIST_EXTEND 1 + GET_ITER
/// → LOAD_CONST (tuple) + GET_ITER
/// Also handles BUILD_SET 0 + LOAD_CONST + SET_UPDATE 1 + GET_ITER.
fn fold_const_iterable_for_iter(&mut self) {
for block in &mut self.blocks {
let mut i = 0;
while i + 3 < block.instructions.len() {
let is_build = matches!(
block.instructions[i].instr.real(),
Some(Instruction::BuildList { .. } | Instruction::BuildSet { .. })
) && u32::from(block.instructions[i].arg) == 0;
let is_const = matches!(
block.instructions[i + 1].instr.real(),
Some(Instruction::LoadConst { .. })
);
let is_extend = matches!(
block.instructions[i + 2].instr.real(),
Some(Instruction::ListExtend { .. } | Instruction::SetUpdate { .. })
) && u32::from(block.instructions[i + 2].arg) == 1;
let is_iter = matches!(
block.instructions[i + 3].instr.real(),
Some(Instruction::GetIter)
);
if is_build && is_const && is_extend && is_iter {
// Replace: BUILD_X 0 → NOP, keep LOAD_CONST, LIST_EXTEND → NOP
let loc = block.instructions[i].location;
block.instructions[i].instr = Instruction::Nop.into();
block.instructions[i].location = loc;
block.instructions[i + 2].instr = Instruction::Nop.into();
block.instructions[i + 2].location = loc;
i += 4;
} else {
i += 1;
}
}
}
}
/// Fold constant set literals: LOAD_CONST* + BUILD_SET N →
/// BUILD_SET 0 + LOAD_CONST (frozenset-as-tuple) + SET_UPDATE 1
fn fold_set_constants(&mut self) {
for block in &mut self.blocks {
let mut i = 0;
while i < block.instructions.len() {
let instr = &block.instructions[i];
let Some(Instruction::BuildSet { .. }) = instr.instr.real() else {
i += 1;
continue;
};
let set_size = u32::from(instr.arg) as usize;
if set_size < 3 || i < set_size {
i += 1;
continue;
}
let start_idx = i - set_size;
let mut elements = Vec::with_capacity(set_size);
let mut all_const = true;
for j in start_idx..i {
let load_instr = &block.instructions[j];
match load_instr.instr.real() {
Some(Instruction::LoadConst { .. }) => {
let const_idx = u32::from(load_instr.arg) as usize;
if let Some(constant) =
self.metadata.consts.get_index(const_idx).cloned()
{
elements.push(constant);
} else {
all_const = false;
break;
}
}
Some(Instruction::LoadSmallInt { .. }) => {
let value = u32::from(load_instr.arg) as i32;
elements.push(ConstantData::Integer {
value: BigInt::from(value),
});
}
_ => {
all_const = false;
break;
}
}
}
if !all_const {
i += 1;
continue;
}
// Use FrozenSet constant (stored as Tuple for now)
let const_data = ConstantData::Tuple { elements };
let (const_idx, _) = self.metadata.consts.insert_full(const_data);
let folded_loc = block.instructions[i].location;
let end_loc = block.instructions[i].end_location;
let eh = block.instructions[i].except_handler;
block.instructions[start_idx].instr = Instruction::BuildSet {
count: Arg::marker(),
}
.into();
block.instructions[start_idx].arg = OpArg::new(0);
block.instructions[start_idx].location = folded_loc;
block.instructions[start_idx].end_location = end_loc;
block.instructions[start_idx].except_handler = eh;
block.instructions[start_idx + 1].instr = Instruction::LoadConst {
consti: Arg::marker(),
}
.into();
block.instructions[start_idx + 1].arg = OpArg::new(const_idx as u32);
block.instructions[start_idx + 1].location = folded_loc;
block.instructions[start_idx + 1].end_location = end_loc;
block.instructions[start_idx + 1].except_handler = eh;
for j in (start_idx + 2)..i {
block.instructions[j].instr = Instruction::Nop.into();
block.instructions[j].location = folded_loc;
}
block.instructions[i].instr = Instruction::SetUpdate { i: Arg::marker() }.into();
block.instructions[i].arg = OpArg::new(1);
i += 1;
}
}
}
/// Peephole optimization: combine consecutive instructions into super-instructions
fn peephole_optimize(&mut self) {
for block in &mut self.blocks {
@@ -1107,12 +1487,19 @@ impl InstrDisplayContext for CodeInfo {
self.metadata.varnames[var_num.as_usize()].as_ref()
}
fn get_cell_name(&self, i: usize) -> &str {
self.metadata
.cellvars
.get_index(i)
.unwrap_or_else(|| &self.metadata.freevars[i - self.metadata.cellvars.len()])
.as_ref()
fn get_localsplus_name(&self, var_num: oparg::VarNum) -> &str {
let idx = var_num.as_usize();
let nlocals = self.metadata.varnames.len();
if idx < nlocals {
self.metadata.varnames[idx].as_ref()
} else {
let cell_idx = idx - nlocals;
self.metadata
.cellvars
.get_index(cell_idx)
.unwrap_or_else(|| &self.metadata.freevars[cell_idx - self.metadata.cellvars.len()])
.as_ref()
}
}
}
@@ -1632,6 +2019,76 @@ fn normalize_jumps(blocks: &mut [Block]) {
}
}
/// Duplicate `LOAD_CONST None + RETURN_VALUE` for blocks that fall through
/// to the final return block. Matches CPython's behavior of ensuring every
/// code path that reaches the end of a function/module has its own explicit
/// return instruction.
fn duplicate_end_returns(blocks: &mut [Block]) {
// Walk the block chain to find the last block
let mut last_block = BlockIdx(0);
let mut current = BlockIdx(0);
while current != BlockIdx::NULL {
last_block = current;
current = blocks[current.idx()].next;
}
// Check if the last block ends with LOAD_CONST + RETURN_VALUE (the implicit return)
let last_insts = &blocks[last_block.idx()].instructions;
// Only apply when the last block is EXACTLY a return-None epilogue
// AND the return instructions have no explicit line number (lineno <= 0)
let is_return_block = last_insts.len() == 2
&& matches!(
last_insts[0].instr,
AnyInstruction::Real(Instruction::LoadConst { .. })
)
&& matches!(
last_insts[1].instr,
AnyInstruction::Real(Instruction::ReturnValue)
);
if !is_return_block {
return;
}
// Get the return instructions to clone
let return_insts: Vec<InstructionInfo> = last_insts[last_insts.len() - 2..].to_vec();
// Find non-cold blocks that fall through to the last block
let mut blocks_to_fix = Vec::new();
current = BlockIdx(0);
while current != BlockIdx::NULL {
let block = &blocks[current.idx()];
if current != last_block && block.next == last_block && !block.cold && !block.except_handler
{
let last_ins = block.instructions.last();
let has_fallthrough = last_ins
.map(|ins| !ins.instr.is_scope_exit() && !ins.instr.is_unconditional_jump())
.unwrap_or(true);
// Don't duplicate if block already ends with the same return pattern
let already_has_return = block.instructions.len() >= 2 && {
let n = block.instructions.len();
matches!(
block.instructions[n - 2].instr,
AnyInstruction::Real(Instruction::LoadConst { .. })
) && matches!(
block.instructions[n - 1].instr,
AnyInstruction::Real(Instruction::ReturnValue)
)
};
if has_fallthrough && !already_has_return {
blocks_to_fix.push(current);
}
}
current = blocks[current.idx()].next;
}
// Duplicate the return instructions at the end of fall-through blocks
for block_idx in blocks_to_fix {
blocks[block_idx.idx()]
.instructions
.extend_from_slice(&return_insts);
}
}
/// Label exception targets: walk CFG with except stack, set per-instruction
/// handler info and block preserve_lasti flag. Converts POP_BLOCK to NOP.
/// flowgraph.c label_exception_targets + push_except_block
@@ -1715,28 +2172,35 @@ pub(crate) fn label_exception_targets(blocks: &mut [Block]) {
blocks[bi].instructions[i].except_handler = handler_info;
// Track YIELD_VALUE except stack depth
if matches!(
blocks[bi].instructions[i].instr.real(),
Some(Instruction::YieldValue { .. })
) {
last_yield_except_depth = stack.len() as i32;
// Only count for direct yield (arg=0), not yield-from/await (arg=1)
// The yield-from's internal SETUP_FINALLY is not an external except depth
if let Some(Instruction::YieldValue { .. }) =
blocks[bi].instructions[i].instr.real()
{
let yield_arg = u32::from(blocks[bi].instructions[i].arg);
if yield_arg == 0 {
// Direct yield: count actual except depth
last_yield_except_depth = stack.len() as i32;
} else {
// yield-from/await: subtract 1 for the internal SETUP_FINALLY
last_yield_except_depth = (stack.len() as i32) - 1;
}
}
// Set RESUME DEPTH1 flag based on last yield's except depth
if matches!(
blocks[bi].instructions[i].instr.real(),
Some(Instruction::Resume { .. })
) {
const RESUME_AT_FUNC_START: u32 = 0;
const RESUME_OPARG_LOCATION_MASK: u32 = 0x3;
const RESUME_OPARG_DEPTH1_MASK: u32 = 0x4;
if (u32::from(arg) & RESUME_OPARG_LOCATION_MASK) != RESUME_AT_FUNC_START {
if last_yield_except_depth == 1 {
blocks[bi].instructions[i].arg =
OpArg::new(u32::from(arg) | RESUME_OPARG_DEPTH1_MASK);
if let Some(Instruction::Resume { context }) =
blocks[bi].instructions[i].instr.real()
{
let location = context.get(arg).location();
match location {
oparg::ResumeLocation::AtFuncStart => {}
_ => {
if last_yield_except_depth == 1 {
blocks[bi].instructions[i].arg =
OpArg::new(oparg::ResumeContext::new(location, true).as_u32());
}
last_yield_except_depth = -1;
}
last_yield_except_depth = -1;
}
}
@@ -1768,7 +2232,7 @@ pub(crate) fn label_exception_targets(blocks: &mut [Block]) {
/// Convert remaining pseudo ops to real instructions or NOP.
/// flowgraph.c convert_pseudo_ops
pub(crate) fn convert_pseudo_ops(blocks: &mut [Block], varnames_len: u32) {
pub(crate) fn convert_pseudo_ops(blocks: &mut [Block], cellfixedoffsets: &[u32]) {
for block in blocks.iter_mut() {
for info in &mut block.instructions {
let Some(pseudo) = info.instr.pseudo() else {
@@ -1786,9 +2250,10 @@ pub(crate) fn convert_pseudo_ops(blocks: &mut [Block], varnames_len: u32) {
PseudoInstruction::PopBlock => {
info.instr = Instruction::Nop.into();
}
// LOAD_CLOSURE → LOAD_FAST (with varnames offset)
// LOAD_CLOSURE → LOAD_FAST (using cellfixedoffsets for merged layout)
PseudoInstruction::LoadClosure { i } => {
let new_idx = varnames_len + i.get(info.arg);
let cell_relative = i.get(info.arg) as usize;
let new_idx = cellfixedoffsets[cell_relative];
info.arg = OpArg::new(new_idx);
info.instr = Instruction::LoadFast {
var_num: Arg::marker(),
@@ -1808,3 +2273,54 @@ pub(crate) fn convert_pseudo_ops(blocks: &mut [Block], varnames_len: u32) {
}
}
}
/// Build cellfixedoffsets mapping: cell/free index -> localsplus index.
/// Merged cells (cellvar also in varnames) get the local slot index.
/// Non-merged cells get slots after nlocals. Free vars follow.
pub(crate) fn build_cellfixedoffsets(
varnames: &IndexSet<String>,
cellvars: &IndexSet<String>,
freevars: &IndexSet<String>,
) -> Vec<u32> {
let nlocals = varnames.len();
let ncells = cellvars.len();
let nfrees = freevars.len();
let mut fixed = Vec::with_capacity(ncells + nfrees);
let mut numdropped = 0usize;
for (i, cellvar) in cellvars.iter().enumerate() {
if let Some(local_idx) = varnames.get_index_of(cellvar) {
fixed.push(local_idx as u32);
numdropped += 1;
} else {
fixed.push((nlocals + i - numdropped) as u32);
}
}
for i in 0..nfrees {
fixed.push((nlocals + ncells - numdropped + i) as u32);
}
fixed
}
/// Convert DEREF instruction opargs from cell-relative indices to localsplus indices
/// using the cellfixedoffsets mapping.
pub(crate) fn fixup_deref_opargs(blocks: &mut [Block], cellfixedoffsets: &[u32]) {
for block in blocks.iter_mut() {
for info in &mut block.instructions {
let Some(instr) = info.instr.real() else {
continue;
};
let needs_fixup = matches!(
instr,
Instruction::LoadDeref { .. }
| Instruction::StoreDeref { .. }
| Instruction::DeleteDeref { .. }
| Instruction::LoadFromDictOrDeref { .. }
| Instruction::MakeCell { .. }
);
if needs_fixup {
let cell_relative = u32::from(info.arg) as usize;
info.arg = OpArg::new(cellfixedoffsets[cell_relative]);
}
}
}
}

View File

@@ -1,21 +1,23 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9100
assertion_line: 9553
expression: "compile_exec(\"\\\nif True and False and False:\n pass\n\")"
---
1 0 RESUME (0)
>> 1 LOAD_CONST (True)
2 POP_JUMP_IF_FALSE (9)
3 CACHE
1 LOAD_CONST (True)
2 POP_JUMP_IF_FALSE (11)
>> 3 CACHE
4 NOT_TAKEN
>> 5 LOAD_CONST (False)
6 POP_JUMP_IF_FALSE (5)
7 CACHE
5 LOAD_CONST (False)
6 POP_JUMP_IF_FALSE (7)
>> 7 CACHE
8 NOT_TAKEN
>> 9 LOAD_CONST (False)
10 POP_JUMP_IF_FALSE (1)
11 CACHE
9 LOAD_CONST (False)
10 POP_JUMP_IF_FALSE (3)
>> 11 CACHE
12 NOT_TAKEN
2 13 LOAD_CONST (None)
14 RETURN_VALUE
15 LOAD_CONST (None)
16 RETURN_VALUE

View File

@@ -1,25 +1,27 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9110
assertion_line: 9563
expression: "compile_exec(\"\\\nif (True and False) or (False and True):\n pass\n\")"
---
1 0 RESUME (0)
>> 1 LOAD_CONST (True)
1 LOAD_CONST (True)
2 POP_JUMP_IF_FALSE (5)
3 CACHE
>> 3 CACHE
4 NOT_TAKEN
>> 5 LOAD_CONST (False)
6 POP_JUMP_IF_TRUE (9)
7 CACHE
>> 7 CACHE
8 NOT_TAKEN
>> 9 LOAD_CONST (False)
10 POP_JUMP_IF_FALSE (5)
10 POP_JUMP_IF_FALSE (7)
11 CACHE
12 NOT_TAKEN
13 LOAD_CONST (True)
14 POP_JUMP_IF_FALSE (1)
14 POP_JUMP_IF_FALSE (3)
15 CACHE
16 NOT_TAKEN
2 17 LOAD_CONST (None)
18 RETURN_VALUE
19 LOAD_CONST (None)
20 RETURN_VALUE

View File

@@ -1,21 +1,23 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9090
assertion_line: 9543
expression: "compile_exec(\"\\\nif True or False or False:\n pass\n\")"
---
1 0 RESUME (0)
>> 1 LOAD_CONST (True)
1 LOAD_CONST (True)
2 POP_JUMP_IF_TRUE (9)
3 CACHE
>> 3 CACHE
4 NOT_TAKEN
>> 5 LOAD_CONST (False)
6 POP_JUMP_IF_TRUE (5)
7 CACHE
8 NOT_TAKEN
>> 9 LOAD_CONST (False)
10 POP_JUMP_IF_FALSE (1)
10 POP_JUMP_IF_FALSE (3)
11 CACHE
12 NOT_TAKEN
2 13 LOAD_CONST (None)
14 RETURN_VALUE
15 LOAD_CONST (None)
16 RETURN_VALUE

View File

@@ -1,5 +1,6 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9688
expression: "compile_exec(\"\\\nx = Test() and False or False\n\")"
---
1 0 RESUME (0)
@@ -9,18 +10,26 @@ expression: "compile_exec(\"\\\nx = Test() and False or False\n\")"
4 CACHE
5 CACHE
6 CACHE
>> 7 COPY (1)
8 POP_JUMP_IF_FALSE (7)
7 COPY (1)
8 TO_BOOL
9 CACHE
10 NOT_TAKEN
11 POP_TOP
12 LOAD_CONST (False)
13 COPY (1)
14 POP_JUMP_IF_TRUE (3)
15 CACHE
16 NOT_TAKEN
17 POP_TOP
18 LOAD_CONST (False)
19 STORE_NAME (1, x)
20 LOAD_CONST (None)
21 RETURN_VALUE
10 CACHE
>> 11 CACHE
12 POP_JUMP_IF_FALSE (11)
13 CACHE
14 NOT_TAKEN
15 POP_TOP
16 LOAD_CONST (False)
17 COPY (1)
18 TO_BOOL
19 CACHE
20 CACHE
21 CACHE
22 POP_JUMP_IF_TRUE (3)
23 CACHE
24 NOT_TAKEN
25 POP_TOP
26 LOAD_CONST (False)
27 STORE_NAME (1, x)
28 LOAD_CONST (None)
29 RETURN_VALUE

View File

@@ -1,6 +1,6 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9089
assertion_line: 9780
expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIteration('spam'), StopAsyncIteration('ham')):\n with self.subTest(type=type(stop_exc)):\n try:\n async with egg():\n raise stop_exc\n except Exception as ex:\n self.assertIs(ex, stop_exc)\n else:\n self.fail(f'{stop_exc} was suppressed')\n\")"
---
1 0 RESUME (0)
@@ -23,8 +23,8 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
15 CACHE
16 CACHE
17 CACHE
>> 18 LOAD_CONST ("ham")
19 CALL (1)
18 LOAD_CONST ("ham")
>> 19 CALL (1)
20 CACHE
21 CACHE
22 CACHE
@@ -32,15 +32,15 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
24 GET_ITER
25 FOR_ITER (71)
26 CACHE
>> 27 STORE_FAST (0, stop_exc)
27 STORE_FAST (0, stop_exc)
3 >> 28 LOAD_GLOBAL (4, self)
29 CACHE
30 CACHE
31 CACHE
32 CACHE
33 LOAD_ATTR (7, subTest, method=true)
>> 34 CACHE
>> 33 LOAD_ATTR (7, subTest, method=true)
34 CACHE
35 CACHE
36 CACHE
37 CACHE
@@ -52,8 +52,8 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
43 LOAD_GLOBAL (9, NULL + type)
44 CACHE
45 CACHE
46 CACHE
>> 47 CACHE
>> 46 CACHE
47 CACHE
48 LOAD_FAST (0, stop_exc)
49 CALL (1)
50 CACHE
@@ -67,8 +67,8 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
58 COPY (1)
59 LOAD_SPECIAL (__exit__)
60 SWAP (2)
61 LOAD_SPECIAL (__enter__)
62 PUSH_NULL
61 SWAP (3)
62 LOAD_SPECIAL (__enter__)
63 CALL (0)
64 CACHE
65 CACHE
@@ -89,8 +89,8 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
78 COPY (1)
79 LOAD_SPECIAL (__aexit__)
80 SWAP (2)
81 LOAD_SPECIAL (__aenter__)
82 PUSH_NULL
81 SWAP (3)
82 LOAD_SPECIAL (__aenter__)
83 CALL (0)
84 CACHE
85 CACHE
@@ -115,162 +115,140 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
5 102 CLEANUP_THROW
103 JUMP_BACKWARD_NO_INTERRUPT(10)
6 104 NOP
5 105 PUSH_NULL
106 LOAD_CONST (None)
104 PUSH_EXC_INFO
105 WITH_EXCEPT_START
106 GET_AWAITABLE (2)
107 LOAD_CONST (None)
108 LOAD_CONST (None)
109 CALL (3)
110 CACHE
111 CACHE
112 CACHE
113 GET_AWAITABLE (2)
114 LOAD_CONST (None)
115 SEND (4)
108 SEND (4)
109 CACHE
110 YIELD_VALUE (1)
111 RESUME (3)
112 JUMP_BACKWARD_NO_INTERRUPT(5)
113 CLEANUP_THROW
114 END_SEND
115 TO_BOOL
116 CACHE
117 YIELD_VALUE (1)
118 RESUME (3)
119 JUMP_BACKWARD_NO_INTERRUPT(5)
120 CLEANUP_THROW
121 END_SEND
122 POP_TOP
123 JUMP_FORWARD (27)
124 PUSH_EXC_INFO
125 WITH_EXCEPT_START
126 GET_AWAITABLE (2)
127 LOAD_CONST (None)
128 SEND (4)
129 CACHE
130 YIELD_VALUE (1)
131 RESUME (3)
132 JUMP_BACKWARD_NO_INTERRUPT(5)
133 CLEANUP_THROW
134 END_SEND
135 TO_BOOL
117 CACHE
118 CACHE
119 POP_JUMP_IF_TRUE (2)
120 CACHE
121 NOT_TAKEN
122 RERAISE (2)
123 POP_TOP
124 POP_EXCEPT
125 POP_TOP
126 POP_TOP
127 POP_TOP
128 JUMP_FORWARD (3)
129 COPY (3)
130 POP_EXCEPT
131 RERAISE (1)
132 JUMP_FORWARD (46)
133 PUSH_EXC_INFO
7 134 LOAD_GLOBAL (12, Exception)
135 CACHE
136 CACHE
137 CACHE
138 CACHE
139 POP_JUMP_IF_TRUE (2)
140 CACHE
141 NOT_TAKEN
142 RERAISE (2)
143 POP_TOP
144 POP_EXCEPT
145 POP_TOP
146 POP_TOP
147 JUMP_FORWARD (3)
148 COPY (3)
149 POP_EXCEPT
150 RERAISE (1)
151 JUMP_FORWARD (47)
152 PUSH_EXC_INFO
139 CHECK_EXC_MATCH
140 POP_JUMP_IF_FALSE (33)
141 CACHE
142 NOT_TAKEN
143 STORE_FAST (1, ex)
7 153 LOAD_GLOBAL (12, Exception)
8 144 LOAD_GLOBAL (4, self)
145 CACHE
146 CACHE
147 CACHE
148 CACHE
149 LOAD_ATTR (15, assertIs, method=true)
150 CACHE
151 CACHE
152 CACHE
153 CACHE
154 CACHE
155 CACHE
156 CACHE
157 CACHE
158 CHECK_EXC_MATCH
159 POP_JUMP_IF_FALSE (34)
160 CACHE
161 NOT_TAKEN
162 STORE_FAST (1, ex)
158 CACHE
159 LOAD_FAST_LOAD_FAST (ex, stop_exc)
160 CALL (2)
161 CACHE
162 CACHE
163 CACHE
164 POP_TOP
165 JUMP_FORWARD (4)
166 LOAD_CONST (None)
167 STORE_FAST (1, ex)
168 DELETE_FAST (1, ex)
169 RERAISE (1)
170 POP_EXCEPT
171 LOAD_CONST (None)
172 STORE_FAST (1, ex)
173 DELETE_FAST (1, ex)
174 JUMP_FORWARD (28)
175 RERAISE (0)
176 COPY (3)
177 POP_EXCEPT
178 RERAISE (1)
8 163 LOAD_GLOBAL (4, self)
164 CACHE
165 CACHE
166 CACHE
167 CACHE
168 LOAD_ATTR (15, assertIs, method=true)
169 CACHE
170 CACHE
171 CACHE
172 CACHE
173 CACHE
174 CACHE
175 CACHE
176 CACHE
177 CACHE
178 LOAD_FAST (1, ex)
179 LOAD_FAST (0, stop_exc)
180 CALL (2)
10 179 LOAD_GLOBAL (4, self)
180 CACHE
181 CACHE
182 CACHE
183 CACHE
184 POP_TOP
185 JUMP_FORWARD (4)
186 LOAD_CONST (None)
187 STORE_FAST (1, ex)
188 DELETE_FAST (1, ex)
189 RERAISE (1)
190 POP_EXCEPT
191 LOAD_CONST (None)
192 STORE_FAST (1, ex)
193 DELETE_FAST (1, ex)
194 JUMP_FORWARD (28)
195 RERAISE (0)
196 COPY (3)
197 POP_EXCEPT
198 RERAISE (1)
10 199 LOAD_GLOBAL (4, self)
184 LOAD_ATTR (17, fail, method=true)
185 CACHE
186 CACHE
187 CACHE
188 CACHE
189 CACHE
190 CACHE
191 CACHE
192 CACHE
193 CACHE
194 LOAD_FAST_BORROW (0, stop_exc)
195 FORMAT_SIMPLE
196 LOAD_CONST (" was suppressed")
197 BUILD_STRING (2)
198 CALL (1)
199 CACHE
200 CACHE
201 CACHE
202 CACHE
203 CACHE
204 LOAD_ATTR (17, fail, method=true)
205 CACHE
206 CACHE
207 CACHE
208 CACHE
209 CACHE
210 CACHE
211 CACHE
212 CACHE
213 CACHE
214 LOAD_FAST_BORROW (0, stop_exc)
215 FORMAT_SIMPLE
216 LOAD_CONST (" was suppressed")
217 BUILD_STRING (2)
218 CALL (1)
219 CACHE
220 CACHE
221 CACHE
222 POP_TOP
223 NOP
202 POP_TOP
203 NOP
3 224 PUSH_NULL
225 LOAD_CONST (None)
226 LOAD_CONST (None)
227 LOAD_CONST (None)
228 CALL (3)
>> 229 CACHE
230 CACHE
231 CACHE
232 POP_TOP
233 JUMP_FORWARD (18)
234 PUSH_EXC_INFO
235 WITH_EXCEPT_START
236 TO_BOOL
237 CACHE
238 CACHE
239 CACHE
240 POP_JUMP_IF_TRUE (2)
241 CACHE
242 NOT_TAKEN
243 RERAISE (2)
244 POP_TOP
245 POP_EXCEPT
246 POP_TOP
247 POP_TOP
248 JUMP_FORWARD (3)
249 COPY (3)
250 POP_EXCEPT
251 RERAISE (1)
252 JUMP_BACKWARD (229)
253 CACHE
3 204 LOAD_CONST (None)
205 LOAD_CONST (None)
206 LOAD_CONST (None)
207 CALL (3)
208 CACHE
>> 209 CACHE
210 CACHE
211 POP_TOP
212 JUMP_FORWARD (19)
213 PUSH_EXC_INFO
214 WITH_EXCEPT_START
215 TO_BOOL
216 CACHE
217 CACHE
218 CACHE
219 POP_JUMP_IF_TRUE (2)
220 CACHE
221 NOT_TAKEN
222 RERAISE (2)
223 POP_TOP
224 POP_EXCEPT
225 POP_TOP
226 POP_TOP
227 POP_TOP
228 JUMP_FORWARD (3)
229 COPY (3)
230 POP_EXCEPT
231 RERAISE (1)
232 JUMP_BACKWARD (209)
233 CACHE
2 MAKE_FUNCTION
3 STORE_NAME (0, test)

View File

@@ -54,6 +54,9 @@ pub struct SymbolTable {
/// Whether this type param scope can see the parent class scope
pub can_see_class_scope: bool,
/// Whether this scope contains yield/yield from (is a generator function)
pub is_generator: bool,
/// Whether this comprehension scope should be inlined (PEP 709)
/// True for list/set/dict comprehensions in non-generator expressions
pub comp_inlined: bool,
@@ -89,6 +92,7 @@ impl SymbolTable {
needs_class_closure: false,
needs_classdict: false,
can_see_class_scope: false,
is_generator: false,
comp_inlined: false,
annotation_block: None,
has_conditional_annotations: false,
@@ -292,6 +296,20 @@ fn drop_class_free(symbol_table: &mut SymbolTable, newfree: &mut IndexSet<String
symbol_table.needs_classdict = true;
}
// Classes with function definitions need __classdict__ for PEP 649
// (but not when `from __future__ import annotations` is active)
if !symbol_table.needs_classdict && !symbol_table.future_annotations {
let has_functions = symbol_table.sub_tables.iter().any(|t| {
matches!(
t.typ,
CompilerScope::Function | CompilerScope::AsyncFunction
)
});
if has_functions {
symbol_table.needs_classdict = true;
}
}
// Check if __conditional_annotations__ is in the free variables collected from children
// Remove it from free set - it's handled specially in class scope
if newfree.shift_remove("__conditional_annotations__") {
@@ -299,6 +317,88 @@ fn drop_class_free(symbol_table: &mut SymbolTable, newfree: &mut IndexSet<String
}
}
/// Check if an expression contains an `await` node (shallow, not into nested scopes).
fn expr_contains_await(expr: &ast::Expr) -> bool {
use ast::visitor::Visitor;
struct AwaitFinder(bool);
impl ast::visitor::Visitor<'_> for AwaitFinder {
fn visit_expr(&mut self, expr: &ast::Expr) {
if !self.0 {
if matches!(expr, ast::Expr::Await(_)) {
self.0 = true;
} else {
ast::visitor::walk_expr(self, expr);
}
}
}
}
let mut finder = AwaitFinder(false);
finder.visit_expr(expr);
finder.0
}
/// PEP 709: Merge symbols from an inlined comprehension into the parent scope.
/// Matches symtable.c inline_comprehension().
fn inline_comprehension(
parent_symbols: &mut SymbolMap,
comp: &SymbolTable,
comp_free: &mut IndexSet<String>,
inlined_cells: &mut IndexSet<String>,
parent_type: CompilerScope,
) {
for (name, sub_symbol) in &comp.symbols {
// Skip the .0 parameter
if sub_symbol.flags.contains(SymbolFlags::PARAMETER) {
continue;
}
// Track inlined cells
if sub_symbol.scope == SymbolScope::Cell
|| sub_symbol.flags.contains(SymbolFlags::COMP_CELL)
{
inlined_cells.insert(name.clone());
}
// Handle __class__ in ClassBlock
let scope = if sub_symbol.scope == SymbolScope::Free
&& parent_type == CompilerScope::Class
&& name == "__class__"
{
comp_free.swap_remove(name);
SymbolScope::GlobalImplicit
} else {
sub_symbol.scope
};
if let Some(existing) = parent_symbols.get(name) {
// Name exists in parent
if existing.is_bound() && parent_type != CompilerScope::Class {
// Check if the name is free in any child of the comprehension
let is_free_in_child = comp.sub_tables.iter().any(|child| {
child
.symbols
.get(name)
.is_some_and(|s| s.scope == SymbolScope::Free)
});
if !is_free_in_child {
comp_free.swap_remove(name);
}
}
} else {
// Name doesn't exist in parent, copy from comprehension.
// Reset scope to Unknown so analyze_symbol will resolve it
// in the parent's context.
let mut symbol = sub_symbol.clone();
symbol.scope = if sub_symbol.is_bound() {
SymbolScope::Unknown
} else {
scope
};
parent_symbols.insert(name.clone(), symbol);
}
}
}
type SymbolMap = IndexMap<String, Symbol>;
mod stack {
@@ -392,14 +492,9 @@ impl SymbolTableAnalyzer {
let symbols = core::mem::take(&mut symbol_table.symbols);
let sub_tables = &mut *symbol_table.sub_tables;
// Collect free variables from all child scopes
let mut newfree = IndexSet::default();
let annotation_block = &mut symbol_table.annotation_block;
// PEP 649: Determine class_entry to pass to children
// If current scope is a class with annotation block that can_see_class_scope,
// we need to pass class symbols to the annotation scope
let is_class = symbol_table.typ == CompilerScope::Class;
// Clone class symbols if needed for child scopes with can_see_class_scope
@@ -418,12 +513,16 @@ impl SymbolTableAnalyzer {
None
};
// Collect (child_free, is_inlined) pairs from child scopes.
// We need to process inlined comprehensions after the closure
// when we have access to symbol_table.symbols.
let mut child_frees: Vec<(IndexSet<String>, bool)> = Vec::new();
let mut annotation_free: Option<IndexSet<String>> = None;
let mut info = (symbols, symbol_table.typ);
self.tables.with_append(&mut info, |list| {
let inner_scope = unsafe { &mut *(list as *mut _ as *mut Self) };
// Analyze sub scopes and collect their free variables
for sub_table in sub_tables.iter_mut() {
// Pass class_entry to sub-scopes that can see the class scope
let child_class_entry = if sub_table.can_see_class_scope {
if is_class {
class_symbols_clone.as_ref()
@@ -434,12 +533,10 @@ impl SymbolTableAnalyzer {
None
};
let child_free = inner_scope.analyze_symbol_table(sub_table, child_class_entry)?;
// Propagate child's free variables to this scope
newfree.extend(child_free);
child_frees.push((child_free, sub_table.comp_inlined));
}
// PEP 649: Analyze annotation block if present
if let Some(annotation_table) = annotation_block {
// Pass class symbols to annotation scope if can_see_class_scope
let ann_class_entry = if annotation_table.can_see_class_scope {
if is_class {
class_symbols_clone.as_ref()
@@ -451,59 +548,59 @@ impl SymbolTableAnalyzer {
};
let child_free =
inner_scope.analyze_symbol_table(annotation_table, ann_class_entry)?;
// Propagate annotation's free variables to this scope
newfree.extend(child_free);
annotation_free = Some(child_free);
}
Ok(())
})?;
symbol_table.symbols = info.0;
// PEP 709: Merge symbols from inlined comprehensions into parent scope
// Only merge symbols that are actually bound in the comprehension,
// not references to outer scope variables (Free symbols).
const BOUND_FLAGS: SymbolFlags = SymbolFlags::ASSIGNED
.union(SymbolFlags::PARAMETER)
.union(SymbolFlags::ITER)
.union(SymbolFlags::ASSIGNED_IN_COMPREHENSION);
for sub_table in sub_tables.iter() {
if sub_table.comp_inlined {
for (name, sub_symbol) in &sub_table.symbols {
// Skip the .0 parameter - it's internal to the comprehension
if name == ".0" {
continue;
}
// Only merge symbols that are bound in the comprehension
// Skip Free references to outer scope variables
if !sub_symbol.flags.intersects(BOUND_FLAGS) {
continue;
}
// If the symbol doesn't exist in parent, add it
if !symbol_table.symbols.contains_key(name) {
let mut symbol = sub_symbol.clone();
// Mark as local in parent scope
symbol.scope = SymbolScope::Local;
symbol_table.symbols.insert(name.clone(), symbol);
}
}
// PEP 709: Process inlined comprehensions.
// Merge symbols from inlined comps into parent scope without bail-out.
let mut inlined_cells: IndexSet<String> = IndexSet::default();
let mut newfree = IndexSet::default();
for (idx, (mut child_free, is_inlined)) in child_frees.into_iter().enumerate() {
if is_inlined {
inline_comprehension(
&mut symbol_table.symbols,
&sub_tables[idx],
&mut child_free,
&mut inlined_cells,
symbol_table.typ,
);
}
newfree.extend(child_free);
}
if let Some(ann_free) = annotation_free {
// Propagate annotation-scope free names to this scope so
// implicit class-scope cells (__classdict__/__conditional_annotations__)
// can be materialized by drop_class_free when needed.
newfree.extend(ann_free);
}
let sub_tables = &*symbol_table.sub_tables;
// Analyze symbols in current scope
for symbol in symbol_table.symbols.values_mut() {
self.analyze_symbol(symbol, symbol_table.typ, sub_tables, class_entry)?;
// Collect free variables from this scope
// These will be propagated to the parent scope
if symbol.scope == SymbolScope::Free || symbol.flags.contains(SymbolFlags::FREE_CLASS) {
newfree.insert(symbol.name.clone());
}
}
// PEP 709: Promote LOCAL to CELL and set COMP_CELL for inlined cell vars
for symbol in symbol_table.symbols.values_mut() {
if inlined_cells.contains(&symbol.name) {
if symbol.scope == SymbolScope::Local {
symbol.scope = SymbolScope::Cell;
}
symbol.flags.insert(SymbolFlags::COMP_CELL);
}
}
// Handle class-specific implicit cells
// This removes __class__ and __classdict__ from newfree if present
// and sets the corresponding flags on the symbol table
if symbol_table.typ == CompilerScope::Class {
drop_class_free(symbol_table, &mut newfree);
}
@@ -665,6 +762,12 @@ impl SymbolTableAnalyzer {
if let Some(decl_depth) = decl_depth {
// decl_depth is the number of tables between the current one and
// the one that declared the cell var
// For implicit class scope variables (__classdict__, __conditional_annotations__),
// only propagate free to annotation/type-param scopes, not regular functions.
// Regular method functions don't need these in their freevars.
let is_class_implicit =
name == "__classdict__" || name == "__conditional_annotations__";
for (table, typ) in self.tables.iter_mut().rev().take(decl_depth) {
if let CompilerScope::Class = typ {
if let Some(free_class) = table.get_mut(name) {
@@ -675,10 +778,19 @@ impl SymbolTableAnalyzer {
symbol.scope = SymbolScope::Free;
table.insert(name.to_owned(), symbol);
}
} else if is_class_implicit
&& matches!(
typ,
CompilerScope::Function
| CompilerScope::AsyncFunction
| CompilerScope::Lambda
)
{
// Skip: don't add __classdict__/__conditional_annotations__
// as free vars in regular functions — only annotation/type scopes need them
} else if !table.contains_key(name) {
let mut symbol = Symbol::new(name);
symbol.scope = SymbolScope::Free;
// symbol.is_referenced = true;
table.insert(name.to_owned(), symbol);
}
}
@@ -694,6 +806,11 @@ impl SymbolTableAnalyzer {
st_typ: CompilerScope,
) -> Option<SymbolScope> {
sub_tables.iter().find_map(|st| {
// PEP 709: For inlined comprehensions, check their children
// instead of the comp itself (its symbols are merged into parent).
if st.comp_inlined {
return self.found_in_inner_scope(&st.sub_tables, name, st_typ);
}
let sym = st.symbols.get(name)?;
if sym.scope == SymbolScope::Free || sym.flags.contains(SymbolFlags::FREE_CLASS) {
if st_typ == CompilerScope::Class && name != "__class__" {
@@ -918,6 +1035,7 @@ impl SymbolTableBuilder {
.and_then(|t| t.mangled_names.clone())
.filter(|_| typ != CompilerScope::Class);
let mut table = SymbolTable::new(name.to_owned(), typ, line_number, is_nested);
table.future_annotations = self.future_annotations;
table.mangled_names = inherited_mangled_names;
self.tables.push(table);
// Save parent's varnames and start fresh for the new scope
@@ -1128,20 +1246,30 @@ impl SymbolTableBuilder {
}
fn scan_annotation(&mut self, annotation: &ast::Expr) -> SymbolTableResult {
self.scan_annotation_inner(annotation, false)
}
/// Scan an annotation from an AnnAssign statement (can be conditional)
fn scan_ann_assign_annotation(&mut self, annotation: &ast::Expr) -> SymbolTableResult {
self.scan_annotation_inner(annotation, true)
}
fn scan_annotation_inner(
&mut self,
annotation: &ast::Expr,
is_ann_assign: bool,
) -> SymbolTableResult {
let current_scope = self.tables.last().map(|t| t.typ);
// PEP 649: Check if this is a conditional annotation
// Module-level: always conditional (module may be partially executed)
// Class-level: conditional only when inside if/for/while/etc.
if !self.future_annotations {
// PEP 649: Only AnnAssign annotations can be conditional.
// Function parameter/return annotations are never conditional.
if is_ann_assign && !self.future_annotations {
let is_conditional = matches!(current_scope, Some(CompilerScope::Module))
|| (matches!(current_scope, Some(CompilerScope::Class))
&& self.in_conditional_block);
if is_conditional && !self.tables.last().unwrap().has_conditional_annotations {
self.tables.last_mut().unwrap().has_conditional_annotations = true;
// Register __conditional_annotations__ as both Assigned and Used so that
// it becomes a Cell variable in class scope (children reference it as Free)
self.register_name(
"__conditional_annotations__",
SymbolUsage::Assigned,
@@ -1473,7 +1601,7 @@ impl SymbolTableBuilder {
// sub_tables that cause mismatch in the annotation scope's sub_table index.
let is_simple_name = *simple && matches!(&**target, Expr::Name(_));
if is_simple_name {
self.scan_annotation(annotation)?;
self.scan_ann_assign_annotation(annotation)?;
} else {
// Still validate annotation for forbidden expressions
// (yield, await, named) even for non-simple targets.
@@ -1729,6 +1857,7 @@ impl SymbolTableBuilder {
node_index: _,
range: _,
}) => {
self.tables.last_mut().unwrap().is_generator = true;
if let Some(expression) = value {
self.scan_expression(expression, context)?;
}
@@ -1738,6 +1867,7 @@ impl SymbolTableBuilder {
node_index: _,
range: _,
}) => {
self.tables.last_mut().unwrap().is_generator = true;
self.scan_expression(value, context)?;
}
Expr::UnaryOp(ExprUnaryOp {
@@ -2036,14 +2166,34 @@ impl SymbolTableBuilder {
CompilerScope::Comprehension,
self.line_index_start(range),
);
// Generator expressions need the is_generator flag
self.tables.last_mut().unwrap().is_generator = is_generator;
// PEP 709: inlined comprehensions are not yet implemented in the
// compiler (is_inlined_comprehension_context always returns false),
// so do NOT mark comp_inlined here. Setting it would cause the
// symbol-table analyzer to merge comprehension-local symbols into
// the parent scope, while the compiler still emits a separate code
// object — leading to the merged symbols being missing from the
// comprehension's own symbol table lookup.
// PEP 709: Mark non-generator comprehensions for inlining,
// but only inside function-like scopes (fastlocals).
// Module/class scope uses STORE_NAME which is incompatible
// with LOAD_FAST_AND_CLEAR / STORE_FAST save/restore.
// Async comprehensions cannot be inlined because they need
// their own coroutine scope.
// Note: tables.last() is the comprehension scope we just pushed,
// so we check the second-to-last for the parent scope.
let element_has_await = expr_contains_await(elt1) || elt2.is_some_and(expr_contains_await);
if !is_generator && !has_async_gen && !element_has_await {
let parent = self.tables.iter().rev().nth(1);
let parent_is_func = parent.is_some_and(|t| {
matches!(
t.typ,
CompilerScope::Function
| CompilerScope::AsyncFunction
| CompilerScope::Lambda
| CompilerScope::Comprehension
)
});
let parent_can_see_class = parent.is_some_and(|t| t.can_see_class_scope);
if parent_is_func && !parent_can_see_class {
self.tables.last_mut().unwrap().comp_inlined = true;
}
}
// Register the passed argument to the generator function as the name ".0"
self.register_name(".0", SymbolUsage::Parameter, range)?;

View File

@@ -19,7 +19,7 @@ itertools = { workspace = true }
malachite-bigint = { workspace = true }
num-complex = { workspace = true }
lz4_flex = "0.12"
lz4_flex = "0.13"
[lints]
workspace = true

View File

@@ -3,7 +3,7 @@
use crate::{
marshal::MarshalError,
varint::{read_varint, read_varint_with_start, write_varint, write_varint_with_start},
varint::{read_varint, read_varint_with_start, write_varint_be, write_varint_with_start},
{OneIndexed, SourceLocation},
};
use alloc::{borrow::ToOwned, boxed::Box, collections::BTreeSet, fmt, string::String, vec::Vec};
@@ -27,7 +27,7 @@ pub use crate::bytecode::{
BinaryOperator, BuildSliceArgCount, CommonConstant, ComparisonOperator, ConvertValueOparg,
IntrinsicFunction1, IntrinsicFunction2, Invert, Label, LoadAttr, LoadSuperAttr,
MakeFunctionFlag, MakeFunctionFlags, NameIdx, OpArg, OpArgByte, OpArgState, OpArgType,
RaiseKind, ResumeType, SpecialMethod, UnpackExArgs,
RaiseKind, SpecialMethod, UnpackExArgs,
},
};
@@ -71,9 +71,9 @@ pub fn encode_exception_table(entries: &[ExceptionTableEntry]) -> alloc::boxed::
let depth_lasti = ((entry.depth as u32) << 1) | (entry.push_lasti as u32);
write_varint_with_start(&mut data, entry.start);
write_varint(&mut data, size);
write_varint(&mut data, entry.target);
write_varint(&mut data, depth_lasti);
write_varint_be(&mut data, size);
write_varint_be(&mut data, entry.target);
write_varint_be(&mut data, depth_lasti);
}
data.into_boxed_slice()
}
@@ -204,7 +204,7 @@ impl PyCodeLocationInfoKind {
}
}
pub trait Constant: Sized {
pub trait Constant: Sized + Clone {
type Name: AsRef<str>;
/// Transforms the given Constant to a BorrowedConstant
@@ -334,6 +334,12 @@ impl<T> IndexMut<oparg::VarNum> for [T] {
}
}
/// Per-slot kind flags for localsplus (co_localspluskinds).
pub const CO_FAST_HIDDEN: u8 = 0x10;
pub const CO_FAST_LOCAL: u8 = 0x20;
pub const CO_FAST_CELL: u8 = 0x40;
pub const CO_FAST_FREE: u8 = 0x80;
/// Primary container of a single code object. Each python function has
/// a code object. Also a module has a code object.
#[derive(Clone)]
@@ -352,12 +358,14 @@ pub struct CodeObject<C: Constant = ConstantData> {
pub obj_name: C::Name,
/// Qualified name of the object (like CPython's co_qualname)
pub qualname: C::Name,
pub cell2arg: Option<Box<[i32]>>,
pub constants: Constants<C>,
pub names: Box<[C::Name]>,
pub varnames: Box<[C::Name]>,
pub cellvars: Box<[C::Name]>,
pub freevars: Box<[C::Name]>,
/// Per-slot kind flags: CO_FAST_LOCAL, CO_FAST_CELL, CO_FAST_FREE, CO_FAST_HIDDEN.
/// Length = nlocalsplus (nlocals + ncells + nfrees).
pub localspluskinds: Box<[u8]>,
/// Line number table (CPython 3.11+ format)
pub linetable: Box<[u8]>,
/// Exception handling table
@@ -559,6 +567,14 @@ impl Deref for CodeUnits {
}
impl CodeUnits {
/// Disable adaptive specialization by setting all counters to unreachable.
/// Used for CPython-compiled bytecode where specialization may not be safe.
pub fn disable_specialization(&self) {
for counter in self.adaptive_counters.iter() {
counter.store(UNREACHABLE_BACKOFF, Ordering::Relaxed);
}
}
/// Replace the opcode at `index` in-place without changing the arg byte.
/// Uses atomic Release store to ensure prior cache writes are visible
/// to threads that subsequently read the new opcode with Acquire.
@@ -1025,7 +1041,7 @@ impl<C: Constant> CodeObject<C> {
}
// arrow and offset
let arrow = if label_targets.contains(&Label::new(offset as u32)) {
let arrow = if label_targets.contains(&Label::from_u32(offset as u32)) {
">>"
} else {
" "
@@ -1080,7 +1096,7 @@ impl<C: Constant> CodeObject<C> {
kwonlyarg_count: self.kwonlyarg_count,
first_line_number: self.first_line_number,
max_stackdepth: self.max_stackdepth,
cell2arg: self.cell2arg,
localspluskinds: self.localspluskinds,
linetable: self.linetable,
exceptiontable: self.exceptiontable,
}
@@ -1112,7 +1128,7 @@ impl<C: Constant> CodeObject<C> {
kwonlyarg_count: self.kwonlyarg_count,
first_line_number: self.first_line_number,
max_stackdepth: self.max_stackdepth,
cell2arg: self.cell2arg.clone(),
localspluskinds: self.localspluskinds.clone(),
linetable: self.linetable.clone(),
exceptiontable: self.exceptiontable.clone(),
}
@@ -1141,7 +1157,8 @@ pub trait InstrDisplayContext {
fn get_varname(&self, var_num: oparg::VarNum) -> &str;
fn get_cell_name(&self, i: usize) -> &str;
/// Get name for a localsplus index (used by DEREF instructions).
fn get_localsplus_name(&self, var_num: oparg::VarNum) -> &str;
}
impl<C: Constant> InstrDisplayContext for CodeObject<C> {
@@ -1159,11 +1176,18 @@ impl<C: Constant> InstrDisplayContext for CodeObject<C> {
self.varnames[var_num].as_ref()
}
fn get_cell_name(&self, i: usize) -> &str {
self.cellvars
.get(i)
.unwrap_or_else(|| &self.freevars[i - self.cellvars.len()])
.as_ref()
fn get_localsplus_name(&self, var_num: oparg::VarNum) -> &str {
let idx = var_num.as_usize();
let nlocals = self.varnames.len();
if idx < nlocals {
self.varnames[idx].as_ref()
} else {
let cell_idx = idx - nlocals;
self.cellvars
.get(cell_idx)
.unwrap_or_else(|| &self.freevars[cell_idx - self.cellvars.len()])
.as_ref()
}
}
}

View File

@@ -130,7 +130,7 @@ pub enum Instruction {
namei: Arg<NameIdx>,
} = 61,
DeleteDeref {
i: Arg<NameIdx>,
i: Arg<oparg::VarNum>,
} = 62,
DeleteFast {
var_num: Arg<oparg::VarNum>,
@@ -189,7 +189,7 @@ pub enum Instruction {
consti: Arg<oparg::ConstIdx>,
} = 82,
LoadDeref {
i: Arg<NameIdx>,
i: Arg<oparg::VarNum>,
} = 83,
LoadFast {
var_num: Arg<oparg::VarNum>,
@@ -210,7 +210,7 @@ pub enum Instruction {
var_nums: Arg<oparg::VarNums>,
} = 89,
LoadFromDictOrDeref {
i: Arg<NameIdx>,
i: Arg<oparg::VarNum>,
} = 90,
LoadFromDictOrGlobals {
i: Arg<NameIdx>,
@@ -231,7 +231,7 @@ pub enum Instruction {
namei: Arg<LoadSuperAttr>,
} = 96,
MakeCell {
i: Arg<NameIdx>,
i: Arg<oparg::VarNum>,
} = 97,
MapAdd {
i: Arg<u32>,
@@ -273,7 +273,7 @@ pub enum Instruction {
namei: Arg<NameIdx>,
} = 110,
StoreDeref {
i: Arg<NameIdx>,
i: Arg<oparg::VarNum>,
} = 111,
StoreFast {
var_num: Arg<oparg::VarNum>,
@@ -304,7 +304,7 @@ pub enum Instruction {
} = 120,
// CPython 3.14 RESUME (128)
Resume {
context: Arg<oparg::ResumeType>,
context: Arg<oparg::ResumeContext>,
} = 128,
// CPython 3.14 specialized opcodes (129-211)
BinaryOpAddFloat = 129, // Placeholder
@@ -1020,7 +1020,7 @@ impl InstructionMetadata for Instruction {
Self::LoadLocals => (1, 0),
Self::LoadName { .. } => (1, 0),
Self::LoadSmallInt { .. } => (1, 0),
Self::LoadSpecial { .. } => (1, 1),
Self::LoadSpecial { .. } => (2, 1),
Self::LoadSuperAttr { .. } => (1 + (oparg & 1), 3),
Self::LoadSuperAttrAttr => (1, 3),
Self::LoadSuperAttrMethod => (2, 3),
@@ -1085,7 +1085,7 @@ impl InstructionMetadata for Instruction {
Self::UnpackSequenceList => (oparg, 1),
Self::UnpackSequenceTuple => (oparg, 1),
Self::UnpackSequenceTwoTuple => (2, 1),
Self::WithExceptStart => (6, 5),
Self::WithExceptStart => (7, 6),
Self::YieldValue { .. } => (1, 1),
};
@@ -1128,7 +1128,7 @@ impl InstructionMetadata for Instruction {
let varname = |var_num: oparg::VarNum| ctx.get_varname(var_num);
let name = |i: u32| ctx.get_name(i as usize);
let cell_name = |i: u32| ctx.get_cell_name(i as usize);
let cell_name = |i: oparg::VarNum| ctx.get_localsplus_name(i);
let fmt_const = |op: &str,
arg: OpArg,

View File

@@ -276,48 +276,6 @@ impl fmt::Display for ConvertValueOparg {
}
}
/// Resume type for the RESUME instruction
#[derive(Copy, Clone, Debug, Hash, PartialEq, Eq)]
pub enum ResumeType {
AtFuncStart,
AfterYield,
AfterYieldFrom,
AfterAwait,
Other(u32),
}
impl From<u32> for ResumeType {
fn from(value: u32) -> Self {
match value {
0 => Self::AtFuncStart,
1 => Self::AfterYield,
2 => Self::AfterYieldFrom,
3 => Self::AfterAwait,
_ => Self::Other(value),
}
}
}
impl From<ResumeType> for u32 {
fn from(typ: ResumeType) -> Self {
match typ {
ResumeType::AtFuncStart => 0,
ResumeType::AfterYield => 1,
ResumeType::AfterYieldFrom => 2,
ResumeType::AfterAwait => 3,
ResumeType::Other(v) => v,
}
}
}
impl core::fmt::Display for ResumeType {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
u32::from(*self).fmt(f)
}
}
impl OpArgType for ResumeType {}
pub type NameIdx = u32;
impl OpArgType for u32 {}
@@ -382,16 +340,20 @@ oparg_enum!(
);
bitflagset::bitflag! {
/// `SET_FUNCTION_ATTRIBUTE` flags.
/// Bitmask: Defaults=0x01, KwOnly=0x02, Annotations=0x04,
/// Closure=0x08, TypeParams=0x10, Annotate=0x20.
/// Stored as bit position (0-5) by `bitflag!` macro.
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
#[repr(u8)]
pub enum MakeFunctionFlag {
Closure = 0,
Annotations = 1,
KwOnlyDefaults = 2,
Defaults = 3,
TypeParams = 4,
Defaults = 0,
KwOnlyDefaults = 1,
Annotations = 2,
Closure = 3,
/// PEP 649: __annotate__ function closure (instead of __annotations__ dict)
Annotate = 5,
Annotate = 4,
TypeParams = 5,
}
}
@@ -403,33 +365,86 @@ bitflagset::bitflagset! {
impl TryFrom<u32> for MakeFunctionFlag {
type Error = MarshalError;
/// Decode from CPython-compatible power-of-two value
fn try_from(value: u32) -> Result<Self, Self::Error> {
Self::try_from(value as u8).map_err(|_| MarshalError::InvalidBytecode)
match value {
0x01 => Ok(Self::Defaults),
0x02 => Ok(Self::KwOnlyDefaults),
0x04 => Ok(Self::Annotations),
0x08 => Ok(Self::Closure),
0x10 => Ok(Self::Annotate),
0x20 => Ok(Self::TypeParams),
_ => Err(MarshalError::InvalidBytecode),
}
}
}
impl From<MakeFunctionFlag> for u32 {
/// Encode as CPython-compatible power-of-two value
fn from(flag: MakeFunctionFlag) -> Self {
flag as u32
1u32 << (flag as u32)
}
}
impl OpArgType for MakeFunctionFlag {}
oparg_enum!(
/// The possible comparison operators.
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
pub enum ComparisonOperator {
// be intentional with bits so that we can do eval_ord with just a bitwise and
// bits: | Equal | Greater | Less |
Less = 0b001,
Greater = 0b010,
NotEqual = 0b011,
Equal = 0b100,
LessOrEqual = 0b101,
GreaterOrEqual = 0b110,
/// `COMPARE_OP` arg is `(cmp_index << 5) | mask`. Only the upper
/// 3 bits identify the comparison; the lower 5 bits are an inline
/// cache mask for adaptive specialization.
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
pub enum ComparisonOperator {
Less,
LessOrEqual,
Equal,
NotEqual,
Greater,
GreaterOrEqual,
}
impl TryFrom<u8> for ComparisonOperator {
type Error = MarshalError;
fn try_from(value: u8) -> Result<Self, Self::Error> {
Self::try_from(value as u32)
}
);
}
impl TryFrom<u32> for ComparisonOperator {
type Error = MarshalError;
/// Decode from `COMPARE_OP` arg: `(cmp_index << 5) | mask`.
fn try_from(value: u32) -> Result<Self, Self::Error> {
match value >> 5 {
0 => Ok(Self::Less),
1 => Ok(Self::LessOrEqual),
2 => Ok(Self::Equal),
3 => Ok(Self::NotEqual),
4 => Ok(Self::Greater),
5 => Ok(Self::GreaterOrEqual),
_ => Err(MarshalError::InvalidBytecode),
}
}
}
impl From<ComparisonOperator> for u8 {
/// Encode as `cmp_index << 5` (mask bits zero).
fn from(value: ComparisonOperator) -> Self {
match value {
ComparisonOperator::Less => 0,
ComparisonOperator::LessOrEqual => 1 << 5,
ComparisonOperator::Equal => 2 << 5,
ComparisonOperator::NotEqual => 3 << 5,
ComparisonOperator::Greater => 4 << 5,
ComparisonOperator::GreaterOrEqual => 5 << 5,
}
}
}
impl From<ComparisonOperator> for u32 {
fn from(value: ComparisonOperator) -> Self {
Self::from(u8::from(value))
}
}
impl OpArgType for ComparisonOperator {}
oparg_enum!(
/// The possible Binary operators
@@ -699,14 +714,8 @@ macro_rules! newtype_oparg {
impl $name {
/// Creates a new [`$name`] instance.
#[must_use]
pub const fn new(value: u32) -> Self {
Self(value)
}
/// Alias to [`$name::new`].
#[must_use]
pub const fn from_u32(value: u32) -> Self {
Self::new(value)
Self(value)
}
/// Returns the oparg as a `u32` value.
@@ -786,15 +795,119 @@ newtype_oparg!(
pub struct Label(u32)
);
newtype_oparg!(
/// Context for [`Instruction::Resume`].
///
/// The oparg consists of two parts:
/// 1. [`ResumeContext::location`]: Indicates where the instruction occurs.
/// 2. [`ResumeContext::is_exception_depth1`]: Is the instruction is at except-depth 1.
#[derive(Clone, Copy)]
#[repr(transparent)]
pub struct ResumeContext(u32)
);
impl ResumeContext {
/// [CPython `RESUME_OPARG_LOCATION_MASK`](https://github.com/python/cpython/blob/v3.14.3/Include/internal/pycore_opcode_utils.h#L84)
pub const LOCATION_MASK: u32 = 0x3;
/// [CPython `RESUME_OPARG_DEPTH1_MASK`](https://github.com/python/cpython/blob/v3.14.3/Include/internal/pycore_opcode_utils.h#L85)
pub const DEPTH1_MASK: u32 = 0x4;
#[must_use]
pub const fn new(location: ResumeLocation, is_exception_depth1: bool) -> Self {
let value = if is_exception_depth1 {
Self::DEPTH1_MASK
} else {
0
};
Self::from_u32(location.as_u32() | value)
}
/// Resume location is determined by [`Self::LOCATION_MASK`].
#[must_use]
pub fn location(&self) -> ResumeLocation {
// SAFETY: The mask should return a value that is in range.
unsafe { ResumeLocation::try_from(self.as_u32() & Self::LOCATION_MASK).unwrap_unchecked() }
}
/// True if the bit at [`Self::DEPTH1_MASK`] is on.
#[must_use]
pub const fn is_exception_depth1(&self) -> bool {
(self.as_u32() & Self::DEPTH1_MASK) != 0
}
}
#[derive(Copy, Clone)]
pub enum ResumeLocation {
/// At the start of a function, which is neither a generator, coroutine nor an async generator.
AtFuncStart,
/// After a `yield` expression.
AfterYield,
/// After a `yield from` expression.
AfterYieldFrom,
/// After an `await` expression.
AfterAwait,
}
impl From<ResumeLocation> for ResumeContext {
fn from(location: ResumeLocation) -> Self {
Self::new(location, false)
}
}
impl TryFrom<u32> for ResumeLocation {
type Error = MarshalError;
fn try_from(value: u32) -> Result<Self, Self::Error> {
Ok(match value {
0 => Self::AtFuncStart,
1 => Self::AfterYield,
2 => Self::AfterYieldFrom,
3 => Self::AfterAwait,
_ => return Err(Self::Error::InvalidBytecode),
})
}
}
impl ResumeLocation {
#[must_use]
pub const fn as_u8(&self) -> u8 {
match self {
Self::AtFuncStart => 0,
Self::AfterYield => 1,
Self::AfterYieldFrom => 2,
Self::AfterAwait => 3,
}
}
#[must_use]
pub const fn as_u32(&self) -> u32 {
self.as_u8() as u32
}
}
impl From<ResumeLocation> for u8 {
fn from(location: ResumeLocation) -> Self {
location.as_u8()
}
}
impl From<ResumeLocation> for u32 {
fn from(location: ResumeLocation) -> Self {
location.as_u32()
}
}
impl VarNums {
#[must_use]
pub const fn idx_1(self) -> VarNum {
VarNum::new(self.0 >> 4)
VarNum::from_u32(self.0 >> 4)
}
#[must_use]
pub const fn idx_2(self) -> VarNum {
VarNum::new(self.0 & 15)
VarNum::from_u32(self.0 & 15)
}
#[must_use]
@@ -830,7 +943,7 @@ impl LoadAttrBuilder {
#[must_use]
pub const fn build(self) -> LoadAttr {
let value = (self.name_idx << 1) | (self.is_method as u32);
LoadAttr::new(value)
LoadAttr::from_u32(value)
}
#[must_use]
@@ -880,7 +993,7 @@ impl LoadSuperAttrBuilder {
pub const fn build(self) -> LoadSuperAttr {
let value =
(self.name_idx << 2) | ((self.has_class as u32) << 1) | (self.is_load_method as u32);
LoadSuperAttr::new(value)
LoadSuperAttr::from_u32(value)
}
#[must_use]

File diff suppressed because it is too large Load Diff

View File

@@ -1,12 +1,14 @@
//! Variable-length integer encoding utilities.
//!
//! Uses 6-bit chunks with a continuation bit (0x40) to encode integers.
//! Used for exception tables and line number tables.
//! Two encodings are used:
//! - **Little-endian** (low bits first): linetable
//! - **Big-endian** (high bits first): exception tables
//!
//! Both use 6-bit chunks with 0x40 as the continuation bit.
use alloc::vec::Vec;
/// Write a variable-length unsigned integer using 6-bit chunks.
/// Returns the number of bytes written.
/// Write a little-endian varint (used by linetable).
#[inline]
pub fn write_varint(buf: &mut Vec<u8>, mut val: u32) -> usize {
let start_len = buf.len();
@@ -18,12 +20,10 @@ pub fn write_varint(buf: &mut Vec<u8>, mut val: u32) -> usize {
buf.len() - start_len
}
/// Write a variable-length signed integer.
/// Returns the number of bytes written.
/// Write a little-endian signed varint.
#[inline]
pub fn write_signed_varint(buf: &mut Vec<u8>, val: i32) -> usize {
let uval = if val < 0 {
// (0 - val as u32) handles INT_MIN correctly
((0u32.wrapping_sub(val as u32)) << 1) | 1
} else {
(val as u32) << 1
@@ -31,70 +31,72 @@ pub fn write_signed_varint(buf: &mut Vec<u8>, val: i32) -> usize {
write_varint(buf, uval)
}
/// Write a variable-length unsigned integer with a start marker (0x80 bit).
/// Used for exception table entries where each entry starts with the marker.
/// Write a big-endian varint (used by exception tables).
pub fn write_varint_be(buf: &mut Vec<u8>, val: u32) -> usize {
let start_len = buf.len();
if val >= 1 << 30 {
buf.push(0x40 | ((val >> 30) & 0x3f) as u8);
}
if val >= 1 << 24 {
buf.push(0x40 | ((val >> 24) & 0x3f) as u8);
}
if val >= 1 << 18 {
buf.push(0x40 | ((val >> 18) & 0x3f) as u8);
}
if val >= 1 << 12 {
buf.push(0x40 | ((val >> 12) & 0x3f) as u8);
}
if val >= 1 << 6 {
buf.push(0x40 | ((val >> 6) & 0x3f) as u8);
}
buf.push((val & 0x3f) as u8);
buf.len() - start_len
}
/// Write a big-endian varint with the start marker (0x80) on the first byte.
pub fn write_varint_with_start(data: &mut Vec<u8>, val: u32) {
let start_pos = data.len();
write_varint(data, val);
// Set start bit on first byte
write_varint_be(data, val);
if let Some(first) = data.get_mut(start_pos) {
*first |= 0x80;
}
}
/// Read a variable-length unsigned integer that starts with a start marker (0x80 bit).
/// Returns None if not at a valid start byte or end of data.
/// Read a big-endian varint with start marker (0x80).
pub fn read_varint_with_start(data: &[u8], pos: &mut usize) -> Option<u32> {
if *pos >= data.len() {
return None;
}
let first = data[*pos];
if first & 0x80 == 0 {
return None; // Not a start byte
return None;
}
*pos += 1;
let mut val = (first & 0x3f) as u32;
let mut shift = 6;
let mut has_continuation = first & 0x40 != 0;
while has_continuation && *pos < data.len() {
let byte = data[*pos];
if byte & 0x80 != 0 {
break; // Next entry start
}
let mut cont = first & 0x40 != 0;
while cont && *pos < data.len() {
let b = data[*pos];
*pos += 1;
val |= ((byte & 0x3f) as u32) << shift;
shift += 6;
has_continuation = byte & 0x40 != 0;
val = (val << 6) | (b & 0x3f) as u32;
cont = b & 0x40 != 0;
}
Some(val)
}
/// Read a variable-length unsigned integer.
/// Returns None if end of data or malformed.
/// Read a big-endian varint (no start marker).
pub fn read_varint(data: &[u8], pos: &mut usize) -> Option<u32> {
if *pos >= data.len() {
return None;
}
let mut val = 0u32;
let mut shift = 0;
loop {
if *pos >= data.len() {
return None;
}
let byte = data[*pos];
if byte & 0x80 != 0 && shift > 0 {
break; // Next entry start
}
let first = data[*pos];
*pos += 1;
let mut val = (first & 0x3f) as u32;
let mut cont = first & 0x40 != 0;
while cont && *pos < data.len() {
let b = data[*pos];
*pos += 1;
val |= ((byte & 0x3f) as u32) << shift;
shift += 6;
if byte & 0x40 == 0 {
break;
}
val = (val << 6) | (b & 0x3f) as u32;
cont = b & 0x40 != 0;
}
Some(val)
}
@@ -104,37 +106,39 @@ mod tests {
use super::*;
#[test]
fn test_write_read_varint() {
fn test_le_varint_roundtrip() {
// Little-endian is only used internally in linetable,
// no read function needed outside of linetable parsing.
let mut buf = Vec::new();
write_varint(&mut buf, 0);
write_varint(&mut buf, 63);
write_varint(&mut buf, 64);
write_varint(&mut buf, 4095);
// Values: 0, 63, 64, 4095
assert_eq!(buf.len(), 1 + 1 + 2 + 2);
}
#[test]
fn test_write_read_signed_varint() {
let mut buf = Vec::new();
write_signed_varint(&mut buf, 0);
write_signed_varint(&mut buf, 1);
write_signed_varint(&mut buf, -1);
write_signed_varint(&mut buf, i32::MIN);
assert!(!buf.is_empty());
fn test_be_varint_roundtrip() {
for &val in &[0u32, 1, 63, 64, 127, 128, 4095, 4096, 1_000_000] {
let mut buf = Vec::new();
write_varint_be(&mut buf, val);
let mut pos = 0;
assert_eq!(read_varint(&buf, &mut pos), Some(val), "val={val}");
assert_eq!(pos, buf.len());
}
}
#[test]
fn test_varint_with_start() {
fn test_be_varint_with_start() {
let mut buf = Vec::new();
write_varint_with_start(&mut buf, 42);
write_varint_with_start(&mut buf, 100);
write_varint_with_start(&mut buf, 71);
let mut pos = 0;
assert_eq!(read_varint_with_start(&buf, &mut pos), Some(42));
assert_eq!(read_varint_with_start(&buf, &mut pos), Some(100));
assert_eq!(read_varint_with_start(&buf, &mut pos), Some(71));
assert_eq!(read_varint_with_start(&buf, &mut pos), None);
}
}

View File

@@ -17,9 +17,9 @@ num-traits = { workspace = true }
thiserror = { workspace = true }
libffi = { workspace = true }
cranelift = "0.129.1"
cranelift-jit = "0.129.1"
cranelift-module = "0.129.1"
cranelift = "0.130.0"
cranelift-jit = "0.130.0"
cranelift-module = "0.130.0"
[dev-dependencies]
rustpython-derive = { workspace = true }

View File

@@ -162,7 +162,7 @@ impl<'a, 'b> FunctionCompiler<'a, 'b> {
let target = after
.checked_add(u32::from(arg))
.ok_or(JitCompileError::BadBytecode)?;
Ok(Label::new(target))
Ok(Label::from_u32(target))
}
fn jump_target_backward(
@@ -177,7 +177,7 @@ impl<'a, 'b> FunctionCompiler<'a, 'b> {
let target = after
.checked_sub(u32::from(arg))
.ok_or(JitCompileError::BadBytecode)?;
Ok(Label::new(target))
Ok(Label::from_u32(target))
}
fn instruction_target(
@@ -232,7 +232,7 @@ impl<'a, 'b> FunctionCompiler<'a, 'b> {
let mut in_unreachable_code = false;
for (offset, &raw_instr) in clean_instructions.iter().enumerate() {
let label = Label::new(offset as u32);
let label = Label::from_u32(offset as u32);
let (instruction, arg) = arg_state.get(raw_instr);
// If this is a label that some earlier jump can target,
@@ -624,7 +624,10 @@ impl<'a, 'b> FunctionCompiler<'a, 'b> {
_ => Err(JitCompileError::NotSupported),
}
}
Instruction::ExtendedArg | Instruction::Cache => Ok(()),
Instruction::ExtendedArg
| Instruction::Cache
| Instruction::MakeCell { .. }
| Instruction::CopyFreeVars { .. } => Ok(()),
Instruction::JumpBackward { .. }
| Instruction::JumpBackwardNoInterrupt { .. }
@@ -733,6 +736,14 @@ impl<'a, 'b> FunctionCompiler<'a, 'b> {
let val = self.stack.pop().ok_or(JitCompileError::BadBytecode)?;
self.store_variable(var_num.get(arg), val)
}
Instruction::StoreFastStoreFast { var_nums } => {
let oparg = var_nums.get(arg);
let (idx1, idx2) = oparg.indexes();
let val1 = self.stack.pop().ok_or(JitCompileError::BadBytecode)?;
self.store_variable(idx1, val1)?;
let val2 = self.stack.pop().ok_or(JitCompileError::BadBytecode)?;
self.store_variable(idx2, val2)
}
Instruction::Swap { i: index } => {
let len = self.stack.len();
let i = len - 1;

View File

@@ -134,12 +134,12 @@ x509-parser = { version = "0.18", optional = true }
der = { version = "0.7", features = ["alloc", "oid"], optional = true }
pem-rfc7468 = { version = "1.0", features = ["alloc"], optional = true }
webpki-roots = { version = "1.0", optional = true }
aws-lc-rs = { version = "1.16.0", optional = true }
aws-lc-rs = { version = "1.16.2", optional = true }
oid-registry = { version = "0.8", features = ["x509", "pkcs1", "nist_algs"], optional = true }
pkcs8 = { version = "0.10", features = ["encryption", "pkcs5", "pem"], optional = true }
[target.'cfg(not(any(target_os = "android", target_arch = "wasm32")))'.dependencies]
libsqlite3-sys = { version = "0.36", features = ["bundled"], optional = true }
libsqlite3-sys = { version = "0.37", features = ["bundled"], optional = true }
liblzma = "0.4"
liblzma-sys = "0.4"

View File

@@ -1204,7 +1204,7 @@ mod mmap {
// Check if this is a Named mmap - these cannot be resized
if let Some(MmapObj::Named(_)) = mmap_guard.as_ref() {
return Err(vm.new_system_error("mmap: cannot resize a named memory mapping"));
return Err(vm.new_os_error("mmap: cannot resize a named memory mapping"));
}
let is_anonymous = handle == INVALID_HANDLE_VALUE as isize;

View File

@@ -128,6 +128,7 @@ features = [
"Win32_System_Environment",
"Win32_System_IO",
"Win32_System_Ioctl",
"Win32_System_JobObjects",
"Win32_System_Kernel",
"Win32_System_LibraryLoader",
"Win32_System_Memory",

View File

@@ -215,6 +215,15 @@ impl PyByteArray {
size_of::<Self>() + self.borrow_buf().len() * size_of::<u8>()
}
#[pyslot]
fn slot_str(zelf: &PyObject, vm: &VirtualMachine) -> PyResult<PyStrRef> {
let zelf = zelf.downcast_ref::<Self>().expect("expected bytearray");
PyBytesInner::warn_on_str("str() on a bytearray instance", vm)?;
let class_name = zelf.class().name();
let repr = zelf.inner().repr_with_name(&class_name, vm)?;
Ok(vm.ctx.new_str(repr))
}
fn __add__(&self, other: ArgBytesLike) -> Self {
self.inner().add(&other.borrow_buf()).into()
}

View File

@@ -224,6 +224,13 @@ impl PyBytes {
size_of::<Self>() + self.len() * size_of::<u8>()
}
#[pyslot]
fn slot_str(zelf: &PyObject, vm: &VirtualMachine) -> PyResult<PyStrRef> {
let zelf = zelf.downcast_ref::<Self>().expect("expected bytes");
PyBytesInner::warn_on_str("str() on a bytes instance", vm)?;
Ok(vm.ctx.new_str(zelf.inner.repr_bytes(vm)?))
}
fn __add__(&self, other: ArgBytesLike) -> Vec<u8> {
self.inner.add(&other.borrow_buf())
}

View File

@@ -194,6 +194,12 @@ impl From<Literal> for PyObjectRef {
}
}
impl From<PyObjectRef> for Literal {
fn from(obj: PyObjectRef) -> Self {
Literal(obj)
}
}
fn borrow_obj_constant(obj: &PyObject) -> BorrowedConstant<'_, Literal> {
match_class!(match obj {
ref i @ super::int::PyInt => {
@@ -633,6 +639,38 @@ impl Constructor for PyCode {
)],
> = vec![(loc, loc); instructions.len()].into_boxed_slice();
// Build localspluskinds with cell-local merging
let localspluskinds = {
use rustpython_compiler_core::bytecode::*;
let nlocals = varnames.len();
let ncells = cellvars.len();
let nfrees = freevars.len();
let numdropped = cellvars
.iter()
.filter(|cv| varnames.iter().any(|v| *v == **cv))
.count();
let nlocalsplus = nlocals + ncells - numdropped + nfrees;
let mut kinds = vec![0u8; nlocalsplus];
for kind in kinds.iter_mut().take(nlocals) {
*kind = CO_FAST_LOCAL;
}
let mut cell_numdropped = 0usize;
for (i, cv) in cellvars.iter().enumerate() {
let merged_idx = varnames.iter().position(|v| **v == **cv);
if let Some(local_idx) = merged_idx {
kinds[local_idx] |= CO_FAST_CELL;
cell_numdropped += 1;
} else {
kinds[nlocals + i - cell_numdropped] = CO_FAST_CELL;
}
}
let free_start = nlocals + ncells - numdropped;
for i in 0..nfrees {
kinds[free_start + i] = CO_FAST_FREE;
}
kinds.into_boxed_slice()
};
// Build the CodeObject
let code = CodeObject {
instructions,
@@ -650,12 +688,12 @@ impl Constructor for PyCode {
max_stackdepth: args.stacksize,
obj_name: vm.ctx.intern_str(args.name.as_wtf8()),
qualname: vm.ctx.intern_str(args.qualname.as_wtf8()),
cell2arg: None, // TODO: reuse `fn cell2arg`
constants,
names,
varnames,
cellvars,
freevars,
localspluskinds,
linetable: args.linetable.as_bytes().to_vec().into_boxed_slice(),
exceptiontable: args.exceptiontable.as_bytes().to_vec().into_boxed_slice(),
};
@@ -1237,7 +1275,7 @@ impl PyCode {
.collect(),
cellvars,
freevars,
cell2arg: self.code.cell2arg.clone(),
localspluskinds: self.code.localspluskinds.clone(),
linetable,
exceptiontable,
};
@@ -1252,22 +1290,34 @@ impl PyCode {
let idx = usize::try_from(opcode).map_err(|_| idx_err(vm))?;
let varnames_len = self.code.varnames.len();
let cellvars_len = self.code.cellvars.len();
// Non-parameter cells: cellvars that are NOT also in varnames
let nonparam_cellvars: Vec<_> = self
.code
.cellvars
.iter()
.filter(|s| {
let s_str: &str = s.as_ref();
!self.code.varnames.iter().any(|v| {
let v_str: &str = v.as_ref();
v_str == s_str
})
})
.collect();
let nonparam_len = nonparam_cellvars.len();
let name = if idx < varnames_len {
// Index in varnames
// Index in varnames (includes parameter cells)
self.code.varnames.get(idx).ok_or_else(|| idx_err(vm))?
} else if idx < varnames_len + cellvars_len {
// Index in cellvars
self.code
.cellvars
} else if idx < varnames_len + nonparam_len {
// Index in non-parameter cellvars
*nonparam_cellvars
.get(idx - varnames_len)
.ok_or_else(|| idx_err(vm))?
} else {
// Index in freevars
self.code
.freevars
.get(idx - varnames_len - cellvars_len)
.get(idx - varnames_len - nonparam_len)
.ok_or_else(|| idx_err(vm))?
};
Ok(name.to_object())

View File

@@ -64,7 +64,7 @@ pub struct PyFunction {
code: PyAtomicRef<PyCode>,
globals: PyDictRef,
builtins: PyObjectRef,
closure: Option<PyRef<PyTuple<PyCellRef>>>,
pub(crate) closure: Option<PyRef<PyTuple<PyCellRef>>>,
defaults_and_kwdefaults: PyMutex<(Option<PyTupleRef>, Option<PyDictRef>)>,
name: PyMutex<PyStrRef>,
qualname: PyMutex<PyStrRef>,
@@ -443,13 +443,6 @@ impl PyFunction {
}
}
if let Some(cell2arg) = code.cell2arg.as_deref() {
for (cell_idx, arg_idx) in cell2arg.iter().enumerate().filter(|(_, i)| **i != -1) {
let x = fastlocals[*arg_idx as usize].take();
frame.set_cell_contents(cell_idx, x);
}
}
Ok(())
}
@@ -725,14 +718,6 @@ impl Py<PyFunction> {
}
}
if let Some(cell2arg) = code.cell2arg.as_deref() {
let fastlocals = unsafe { frame.fastlocals_mut() };
for (cell_idx, arg_idx) in cell2arg.iter().enumerate().filter(|(_, i)| **i != -1) {
let x = fastlocals[*arg_idx as usize].take();
frame.set_cell_contents(cell_idx, x);
}
}
frame
}
@@ -780,11 +765,7 @@ pub(crate) fn datastack_frame_size_bytes_for_code(code: &Py<PyCode>) -> Option<u
{
return None;
}
let nlocalsplus = code
.varnames
.len()
.checked_add(code.cellvars.len())?
.checked_add(code.freevars.len())?;
let nlocalsplus = code.localspluskinds.len();
let capacity = nlocalsplus.checked_add(code.max_stackdepth as usize)?;
capacity.checked_mul(core::mem::size_of::<usize>())
}
@@ -1216,6 +1197,17 @@ impl GetAttr for PyBoundMethod {
}
}
impl GetDescriptor for PyBoundMethod {
fn descr_get(
zelf: PyObjectRef,
_obj: Option<PyObjectRef>,
_cls: Option<PyObjectRef>,
_vm: &VirtualMachine,
) -> PyResult {
Ok(zelf)
}
}
#[derive(FromArgs)]
pub struct PyBoundMethodNewArgs {
#[pyarg(positional)]
@@ -1230,8 +1222,14 @@ impl Constructor for PyBoundMethod {
fn py_new(
_cls: &Py<PyType>,
Self::Args { function, object }: Self::Args,
_vm: &VirtualMachine,
vm: &VirtualMachine,
) -> PyResult<Self> {
if !function.is_callable() {
return Err(vm.new_type_error("first argument must be callable".to_owned()));
}
if vm.is_none(&object) {
return Err(vm.new_type_error("instance must not be None".to_owned()));
}
Ok(Self::new(object, function))
}
}
@@ -1258,7 +1256,15 @@ impl PyBoundMethod {
}
#[pyclass(
with(Callable, Comparable, Hashable, GetAttr, Constructor, Representable),
with(
Callable,
Comparable,
Hashable,
GetAttr,
GetDescriptor,
Constructor,
Representable
),
flags(IMMUTABLETYPE, HAS_WEAKREF)
)]
impl PyBoundMethod {
@@ -1266,11 +1272,11 @@ impl PyBoundMethod {
fn __reduce__(
&self,
vm: &VirtualMachine,
) -> (Option<PyObjectRef>, (PyObjectRef, Option<PyObjectRef>)) {
let builtins_getattr = vm.builtins.get_attr("getattr", vm).ok();
) -> PyResult<(PyObjectRef, (PyObjectRef, PyObjectRef))> {
let builtins_getattr = vm.builtins.get_attr("getattr", vm)?;
let func_self = self.object.clone();
let func_name = self.function.get_attr("__name__", vm).ok();
(builtins_getattr, (func_self, func_name))
let func_name = self.function.get_attr("__name__", vm)?;
Ok((builtins_getattr, (func_self, func_name)))
}
#[pygetset]

View File

@@ -394,13 +394,10 @@ impl Constructor for PyStr {
type Args = StrArgs;
fn slot_new(cls: PyTypeRef, func_args: FuncArgs, vm: &VirtualMachine) -> PyResult {
// Optimization: return exact str as-is (only when no encoding/errors provided)
if cls.is(vm.ctx.types.str_type)
&& func_args.args.len() == 1
&& func_args.kwargs.is_empty()
&& func_args.args[0].class().is(vm.ctx.types.str_type)
// Optimization: for exact str, return PyObject_Str result as-is
if cls.is(vm.ctx.types.str_type) && func_args.args.len() == 1 && func_args.kwargs.is_empty()
{
return Ok(func_args.args[0].clone());
return func_args.args[0].str(vm).map(Into::into);
}
let args: Self::Args = func_args.bind(vm)?;

View File

@@ -7,6 +7,7 @@ See also [CPython source code.](https://github.com/python/cpython/blob/50b48572d
use super::{PyStr, PyType, PyTypeRef};
use crate::{
AsObject, Context, Py, PyObjectRef, PyPayload, PyRef, PyResult, VirtualMachine,
builtins::function::PyCell,
class::PyClassImpl,
common::lock::PyRwLock,
function::{FuncArgs, IntoFuncArgs, OptionalArg},
@@ -86,27 +87,33 @@ impl Initializer for PySuper {
return Err(vm.new_runtime_error("super(): no arguments"));
}
// SAFETY: Frame is current and not concurrently mutated.
use rustpython_compiler_core::bytecode::CO_FAST_CELL;
let obj = unsafe { frame.fastlocals() }[0]
.clone()
.or_else(|| {
if let Some(cell2arg) = frame.code.cell2arg.as_deref() {
cell2arg[..frame.code.cellvars.len()]
.iter()
.enumerate()
.find(|(_, arg_idx)| **arg_idx == 0)
.and_then(|(cell_idx, _)| frame.get_cell_contents(cell_idx))
.and_then(|val| {
// If slot 0 is a merged cell (LOCAL|CELL), extract value from cell
if frame
.code
.localspluskinds
.first()
.is_some_and(|&k| k & CO_FAST_CELL != 0)
{
val.downcast_ref::<PyCell>().and_then(|c| c.get())
} else {
None
Some(val)
}
})
.ok_or_else(|| vm.new_runtime_error("super(): arg[0] deleted"))?;
let mut typ = None;
// Search for __class__ in freevars using localspluskinds
let nlocalsplus = frame.code.localspluskinds.len();
let nfrees = frame.code.freevars.len();
let free_start = nlocalsplus - nfrees;
for (i, var) in frame.code.freevars.iter().enumerate() {
if var.as_bytes() == b"__class__" {
let i = frame.code.cellvars.len() + i;
let class = frame
.get_cell_contents(i)
.get_cell_contents(free_start + i)
.ok_or_else(|| vm.new_runtime_error("super(): empty __class__ cell"))?;
typ = Some(class.downcast().map_err(|o| {
vm.new_type_error(format!(

View File

@@ -1868,14 +1868,16 @@ impl Constructor for PyType {
};
let qualname = dict
.pop_item(identifier!(vm, __qualname__).as_object(), vm)?
.get_item_opt(identifier!(vm, __qualname__), vm)?
.map(|obj| downcast_qualname(obj, vm))
.transpose()?
.unwrap_or_else(|| {
// If __qualname__ is not provided, we can use the name as default
name.clone().into_wtf8()
});
let mut attributes = dict.to_attributes(vm);
attributes.shift_remove(identifier!(vm, __qualname__));
// Check __doc__ for surrogates - raises UnicodeEncodeError during type creation
if let Some(doc) = attributes.get(identifier!(vm, __doc__))
@@ -2133,15 +2135,29 @@ impl Constructor for PyType {
}
}
if let Some(cell) = typ.attributes.write().get(identifier!(vm, __classcell__)) {
let cell = PyCellRef::try_from_object(vm, cell.clone()).map_err(|_| {
vm.new_type_error(format!(
"__classcell__ must be a nonlocal cell, not {}",
cell.class().name()
))
})?;
cell.set(Some(typ.clone().into()));
};
{
let mut attrs = typ.attributes.write();
if let Some(cell) = attrs.get(identifier!(vm, __classcell__)) {
let cell = PyCellRef::try_from_object(vm, cell.clone()).map_err(|_| {
vm.new_type_error(format!(
"__classcell__ must be a nonlocal cell, not {}",
cell.class().name()
))
})?;
cell.set(Some(typ.clone().into()));
attrs.shift_remove(identifier!(vm, __classcell__));
}
if let Some(cell) = attrs.get(identifier!(vm, __classdictcell__)) {
let cell = PyCellRef::try_from_object(vm, cell.clone()).map_err(|_| {
vm.new_type_error(format!(
"__classdictcell__ must be a nonlocal cell, not {}",
cell.class().name()
))
})?;
cell.set(Some(dict.clone().into()));
attrs.shift_remove(identifier!(vm, __classdictcell__));
}
}
// All *classes* should have a dict. Exceptions are *instances* of
// classes that define __slots__ and instances of built-in classes

View File

@@ -237,6 +237,18 @@ impl PyBytesInner {
vm.new_overflow_error("bytes object is too large to make repr")
}
pub(crate) fn warn_on_str(message: &'static str, vm: &VirtualMachine) -> PyResult<()> {
if vm.state.config.settings.bytes_warning > 0 {
crate::stdlib::_warnings::warn(
vm.ctx.exceptions.bytes_warning,
message.to_owned(),
1,
vm,
)?;
}
Ok(())
}
pub fn repr_with_name(&self, class_name: &str, vm: &VirtualMachine) -> PyResult<String> {
const DECORATION_LEN: isize = 2 + 3; // 2 for (), 3 for b"" => bytearray(b"")
let escape = crate::literal::escape::AsciiEscape::new_repr(&self.elements);

View File

@@ -10,7 +10,6 @@ use crate::{
PyBaseException, PyBaseExceptionRef, PyBaseObject, PyCode, PyCoroutine, PyDict, PyDictRef,
PyFloat, PyFrozenSet, PyGenerator, PyInt, PyInterpolation, PyList, PyModule, PyProperty,
PySet, PySlice, PyStr, PyStrInterned, PyTemplate, PyTraceback, PyType, PyUtf8Str,
asyncgenerator::PyAsyncGenWrappedValue,
builtin_func::PyNativeFunction,
descriptor::{MemberGetter, PyMemberDescriptor, PyMethodDescriptor},
frame::stack_analysis,
@@ -41,7 +40,6 @@ use crate::{
use alloc::fmt;
use bstr::ByteSlice;
use core::cell::UnsafeCell;
use core::iter::zip;
use core::sync::atomic;
use core::sync::atomic::AtomicPtr;
use core::sync::atomic::Ordering::{Acquire, Relaxed};
@@ -684,14 +682,7 @@ impl Frame {
use_datastack: bool,
vm: &VirtualMachine,
) -> Self {
let nlocals = code.varnames.len();
let num_cells = code.cellvars.len();
let nfrees = closure.len();
let nlocalsplus = nlocals
.checked_add(num_cells)
.and_then(|v| v.checked_add(nfrees))
.expect("Frame::new: nlocalsplus overflow");
let nlocalsplus = code.localspluskinds.len();
let max_stackdepth = code.max_stackdepth as usize;
let mut localsplus = if use_datastack {
LocalsPlus::new_on_datastack(nlocalsplus, max_stackdepth, vm)
@@ -699,15 +690,32 @@ impl Frame {
LocalsPlus::new(nlocalsplus, max_stackdepth)
};
// Store cell/free variable objects directly in localsplus
let fastlocals = localsplus.fastlocals_mut();
for i in 0..num_cells {
fastlocals[nlocals + i] = Some(PyCell::default().into_ref(&vm.ctx).into());
}
for (i, cell) in closure.iter().enumerate() {
fastlocals[nlocals + num_cells + i] = Some(cell.clone().into());
// Pre-copy closure cells into free var slots so that locals() works
// even before COPY_FREE_VARS runs (e.g. coroutine before first send).
// COPY_FREE_VARS will overwrite these on first execution.
{
let nfrees = code.freevars.len();
if nfrees > 0 {
let freevar_start = nlocalsplus - nfrees;
let fastlocals = localsplus.fastlocals_mut();
for (i, cell) in closure.iter().enumerate() {
fastlocals[freevar_start + i] = Some(cell.clone().into());
}
}
}
// For generators/coroutines, initialize prev_line to the def line
// so that preamble instructions (RETURN_GENERATOR, POP_TOP) don't
// fire spurious LINE events.
let prev_line = if code
.flags
.intersects(bytecode::CodeFlags::GENERATOR | bytecode::CodeFlags::COROUTINE)
{
code.first_line_number.map_or(0, |line| line.get() as u32)
} else {
0
};
let iframe = InterpreterFrame {
localsplus,
locals: match scope.locals {
@@ -722,7 +730,7 @@ impl Frame {
code,
func_obj,
lasti: Radium::new(0),
prev_line: 0,
prev_line,
trace: PyMutex::new(vm.ctx.none()),
trace_lines: PyMutex::new(true),
trace_opcodes: PyMutex::new(false),
@@ -791,30 +799,17 @@ impl Frame {
}
}
/// Get cell contents by cell index. Reads through fastlocals (no state lock needed).
pub(crate) fn get_cell_contents(&self, cell_idx: usize) -> Option<PyObjectRef> {
let nlocals = self.code.varnames.len();
/// Get cell contents by localsplus index.
pub(crate) fn get_cell_contents(&self, localsplus_idx: usize) -> Option<PyObjectRef> {
// SAFETY: Frame not executing; no concurrent mutation.
let fastlocals = unsafe { (*self.iframe.get()).localsplus.fastlocals() };
fastlocals
.get(nlocals + cell_idx)
.get(localsplus_idx)
.and_then(|slot| slot.as_ref())
.and_then(|obj| obj.downcast_ref::<PyCell>())
.and_then(|cell| cell.get())
}
/// Set cell contents by cell index. Only safe to call before frame execution starts.
pub(crate) fn set_cell_contents(&self, cell_idx: usize, value: Option<PyObjectRef>) {
let nlocals = self.code.varnames.len();
// SAFETY: Called before frame execution starts.
let fastlocals = unsafe { (*self.iframe.get()).localsplus.fastlocals() };
fastlocals[nlocals + cell_idx]
.as_ref()
.and_then(|obj| obj.downcast_ref::<PyCell>())
.expect("cell slot empty or not a PyCell")
.set(value);
}
/// Store a borrowed back-reference to the owning generator/coroutine.
/// The caller must ensure the generator outlives the frame.
pub fn set_generator(&self, generator: &PyObject) {
@@ -888,41 +883,102 @@ impl Frame {
}
pub fn locals(&self, vm: &VirtualMachine) -> PyResult<ArgMapping> {
use rustpython_compiler_core::bytecode::{
CO_FAST_CELL, CO_FAST_FREE, CO_FAST_HIDDEN, CO_FAST_LOCAL,
};
// SAFETY: Either the frame is not executing (caller checked owner),
// or we're in a trace callback on the same thread that's executing.
let locals = &self.locals;
let code = &**self.code;
let map = &code.varnames;
let j = core::cmp::min(map.len(), code.varnames.len());
let locals_map = locals.mapping(vm);
if !code.varnames.is_empty() {
let fastlocals = unsafe { (*self.iframe.get()).localsplus.fastlocals() };
for (&k, v) in zip(&map[..j], fastlocals) {
match locals_map.ass_subscript(k, v.clone(), vm) {
Ok(()) => {}
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
Err(e) => return Err(e),
let fastlocals = unsafe { (*self.iframe.get()).localsplus.fastlocals() };
// Iterate through all localsplus slots using localspluskinds
let nlocalsplus = code.localspluskinds.len();
let nfrees = code.freevars.len();
let free_start = nlocalsplus - nfrees;
let is_optimized = code.flags.contains(bytecode::CodeFlags::OPTIMIZED);
// Track which non-merged cellvar index we're at
let mut nonmerged_cell_idx = 0;
for (i, &kind) in code.localspluskinds.iter().enumerate() {
if kind & CO_FAST_HIDDEN != 0 {
// Hidden variables are only skipped when their slot is empty.
// After a comprehension restores values, they should appear in locals().
let slot_empty = match fastlocals[i].as_ref() {
None => true,
Some(obj) => {
if kind & (CO_FAST_CELL | CO_FAST_FREE) != 0 {
// If it's a PyCell, check if the cell is empty.
// If it's a raw value (merged cell during inlined comp), not empty.
obj.downcast_ref::<PyCell>()
.is_some_and(|cell| cell.get().is_none())
} else {
false
}
}
};
if slot_empty {
continue;
}
}
}
if !code.cellvars.is_empty() || !code.freevars.is_empty() {
for (i, &k) in code.cellvars.iter().enumerate() {
let cell_value = self.get_cell_contents(i);
match locals_map.ass_subscript(k, cell_value, vm) {
Ok(()) => {}
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
Err(e) => return Err(e),
}
// Free variables only included for optimized (function-like) scopes.
// Class/module scopes should not expose free vars in locals().
if kind == CO_FAST_FREE && !is_optimized {
continue;
}
if code.flags.contains(bytecode::CodeFlags::OPTIMIZED) {
for (i, &k) in code.freevars.iter().enumerate() {
let cell_value = self.get_cell_contents(code.cellvars.len() + i);
match locals_map.ass_subscript(k, cell_value, vm) {
Ok(()) => {}
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
Err(e) => return Err(e),
// Get the name for this slot
let name = if kind & CO_FAST_LOCAL != 0 {
code.varnames[i]
} else if kind & CO_FAST_FREE != 0 {
code.freevars[i - free_start]
} else if kind & CO_FAST_CELL != 0 {
// Non-merged cell: find the name by skipping merged cellvars
let mut found_name = None;
let mut skip = nonmerged_cell_idx;
for cv in code.cellvars.iter() {
let is_merged = code.varnames.contains(cv);
if !is_merged {
if skip == 0 {
found_name = Some(*cv);
break;
}
skip -= 1;
}
}
nonmerged_cell_idx += 1;
match found_name {
Some(n) => n,
None => continue,
}
} else {
continue;
};
// Get the value
let value = if kind & (CO_FAST_CELL | CO_FAST_FREE) != 0 {
// Cell or free var: extract value from PyCell.
// During inlined comprehensions, a merged cell slot may hold a raw
// value (not a PyCell) after LOAD_FAST_AND_CLEAR + STORE_FAST.
fastlocals[i].as_ref().and_then(|obj| {
if let Some(cell) = obj.downcast_ref::<PyCell>() {
cell.get()
} else {
Some(obj.clone())
}
})
} else {
// Regular local
fastlocals[i].clone()
};
match locals_map.ass_subscript(name, value, vm) {
Ok(()) => {}
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
Err(e) => return Err(e),
}
}
Ok(locals.clone_mapping(vm))
@@ -1325,13 +1381,12 @@ impl ExecutingFrame<'_> {
self.lasti.load(Relaxed)
}
/// Access the PyCellRef at the given cell/free variable index.
/// `cell_idx` is 0-based: 0..ncells for cellvars, ncells.. for freevars.
/// Access the PyCellRef at the given localsplus index.
#[inline(always)]
fn cell_ref(&self, cell_idx: usize) -> &PyCell {
let nlocals = self.code.varnames.len();
self.localsplus.fastlocals()[nlocals + cell_idx]
.as_ref()
fn cell_ref(&self, localsplus_idx: usize) -> &PyCell {
let fastlocals = self.localsplus.fastlocals();
let slot = &fastlocals[localsplus_idx];
slot.as_ref()
.expect("cell slot empty")
.downcast_ref::<PyCell>()
.expect("cell slot is not a PyCell")
@@ -1871,18 +1926,72 @@ impl ExecutingFrame<'_> {
}
}
fn unbound_cell_exception(&self, i: usize, vm: &VirtualMachine) -> PyBaseExceptionRef {
if let Some(&name) = self.code.cellvars.get(i) {
vm.new_exception_msg(
vm.ctx.exceptions.unbound_local_error.to_owned(),
format!("local variable '{name}' referenced before assignment").into(),
)
} else {
let name = self.code.freevars[i - self.code.cellvars.len()];
fn unbound_cell_exception(
&self,
localsplus_idx: usize,
vm: &VirtualMachine,
) -> PyBaseExceptionRef {
use rustpython_compiler_core::bytecode::CO_FAST_FREE;
let kind = self
.code
.localspluskinds
.get(localsplus_idx)
.copied()
.unwrap_or(0);
if kind & CO_FAST_FREE != 0 {
let name = self.localsplus_name(localsplus_idx);
vm.new_name_error(
format!("cannot access free variable '{name}' where it is not associated with a value in enclosing scope"),
name.to_owned(),
)
} else {
// Both merged cells (LOCAL|CELL) and non-merged cells get unbound local error
let name = self.localsplus_name(localsplus_idx);
vm.new_exception_msg(
vm.ctx.exceptions.unbound_local_error.to_owned(),
format!("local variable '{name}' referenced before assignment").into(),
)
}
}
/// Get the variable name for a localsplus index.
fn localsplus_name(&self, idx: usize) -> &'static PyStrInterned {
use rustpython_compiler_core::bytecode::{CO_FAST_CELL, CO_FAST_FREE, CO_FAST_LOCAL};
let nlocals = self.code.varnames.len();
let kind = self.code.localspluskinds.get(idx).copied().unwrap_or(0);
if kind & CO_FAST_LOCAL != 0 {
// Merged cell or regular local: name is in varnames
self.code.varnames[idx]
} else if kind & CO_FAST_FREE != 0 {
// Free var: slots are at the end of localsplus
let nlocalsplus = self.code.localspluskinds.len();
let nfrees = self.code.freevars.len();
let free_start = nlocalsplus - nfrees;
self.code.freevars[idx - free_start]
} else if kind & CO_FAST_CELL != 0 {
// Non-merged cell: count how many non-merged cell slots are before
// this index to find the corresponding cellvars entry.
// Non-merged cellvars appear in their original order (skipping merged ones).
let nonmerged_pos = self.code.localspluskinds[nlocals..idx]
.iter()
.filter(|&&k| k == CO_FAST_CELL)
.count();
// Skip merged cellvars to find the right one
let mut cv_idx = 0;
let mut nonmerged_count = 0;
for (i, name) in self.code.cellvars.iter().enumerate() {
let is_merged = self.code.varnames.contains(name);
if !is_merged {
if nonmerged_count == nonmerged_pos {
cv_idx = i;
break;
}
nonmerged_count += 1;
}
}
self.code.cellvars[cv_idx]
} else {
self.code.varnames[idx]
}
}
@@ -2153,13 +2262,29 @@ impl ExecutingFrame<'_> {
self.push_stackref_opt(value);
Ok(None)
}
Instruction::CopyFreeVars { .. } => {
// Free vars are already set up at frame creation time in RustPython
Instruction::CopyFreeVars { n } => {
let n = n.get(arg) as usize;
if n > 0 {
let closure = self
.object
.func_obj
.as_ref()
.and_then(|f| f.downcast_ref::<PyFunction>())
.and_then(|f| f.closure.as_ref());
let nlocalsplus = self.code.localspluskinds.len();
let freevar_start = nlocalsplus - n;
let fastlocals = self.localsplus.fastlocals_mut();
if let Some(closure) = closure {
for i in 0..n {
fastlocals[freevar_start + i] = Some(closure[i].clone().into());
}
}
}
Ok(None)
}
Instruction::DeleteAttr { namei: idx } => self.delete_attr(vm, idx.get(arg)),
Instruction::DeleteDeref { i } => {
self.cell_ref(i.get(arg) as usize).set(None);
self.cell_ref(i.get(arg).as_usize()).set(None);
Ok(None)
}
Instruction::DeleteFast { var_num } => {
@@ -2311,7 +2436,7 @@ impl ExecutingFrame<'_> {
}
Instruction::ForIter { .. } => {
// Relative forward jump: target = lasti + caches + delta
let target = bytecode::Label::new(self.lasti() + 1 + u32::from(arg));
let target = bytecode::Label::from_u32(self.lasti() + 1 + u32::from(arg));
self.adaptive(|s, ii, cb| s.specialize_for_iter(vm, u32::from(arg), ii, cb));
self.execute_for_iter(vm, target)?;
Ok(None)
@@ -2509,7 +2634,7 @@ impl ExecutingFrame<'_> {
}
Instruction::ListAppend { i } => {
let item = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let list: &Py<PyList> = unsafe {
// SAFETY: trust compiler
obj.downcast_unchecked_ref()
@@ -2519,7 +2644,7 @@ impl ExecutingFrame<'_> {
}
Instruction::ListExtend { i } => {
let iterable = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let list: &Py<PyList> = unsafe {
// SAFETY: compiler guarantees correct type
obj.downcast_unchecked_ref()
@@ -2585,12 +2710,8 @@ impl ExecutingFrame<'_> {
Instruction::LoadFromDictOrDeref { i } => {
// Pop dict from stack (locals or classdict depending on context)
let class_dict = self.pop_value();
let i = i.get(arg) as usize;
let name = if i < self.code.cellvars.len() {
self.code.cellvars[i]
} else {
self.code.freevars[i - self.code.cellvars.len()]
};
let idx = i.get(arg).as_usize();
let name = self.localsplus_name(idx);
// Only treat KeyError as "not found", propagate other exceptions
let value = if let Some(dict_obj) = class_dict.downcast_ref::<PyDict>() {
dict_obj.get_item_opt(name, vm)?
@@ -2604,9 +2725,9 @@ impl ExecutingFrame<'_> {
self.push_value(match value {
Some(v) => v,
None => self
.cell_ref(i)
.cell_ref(idx)
.get()
.ok_or_else(|| self.unbound_cell_exception(i, vm))?,
.ok_or_else(|| self.unbound_cell_exception(idx, vm))?,
});
Ok(None)
}
@@ -2672,7 +2793,7 @@ impl ExecutingFrame<'_> {
Ok(None)
}
Instruction::LoadDeref { i } => {
let idx = i.get(arg) as usize;
let idx = i.get(arg).as_usize();
let x = self
.cell_ref(idx)
.get()
@@ -2699,13 +2820,12 @@ impl ExecutingFrame<'_> {
Ok(None)
}
Instruction::LoadFastAndClear { var_num } => {
// Load value and clear the slot (for inlined comprehensions)
// If slot is empty, push None (not an error - variable may not exist yet)
// Save current slot value and clear it (for inlined comprehensions).
// Pushes NULL (None at Option level) if slot was empty, so that
// StoreFast can restore the empty state after the comprehension.
let idx = var_num.get(arg);
let x = self.localsplus.fastlocals_mut()[idx]
.take()
.unwrap_or_else(|| vm.ctx.none());
self.push_value(x);
let x = self.localsplus.fastlocals_mut()[idx].take();
self.push_value_opt(x);
Ok(None)
}
Instruction::LoadFastCheck { var_num } => {
@@ -2825,23 +2945,23 @@ impl ExecutingFrame<'_> {
Ok(None)
}
Instruction::LoadSpecial { method } => {
// Stack effect: 0 (replaces TOS with bound method)
// Input: [..., obj]
// Output: [..., bound_method]
// Pops obj, pushes (callable, self_or_null) for CALL convention.
// Push order: callable first (deeper), self_or_null on top.
use crate::vm::PyMethod;
let obj = self.pop_value();
let oparg = method.get(arg);
let method_name = get_special_method_name(oparg, vm);
let bound = match vm.get_special_method(&obj, method_name)? {
match vm.get_special_method(&obj, method_name)? {
Some(PyMethod::Function { target, func }) => {
// Create bound method: PyBoundMethod(object=target, function=func)
crate::builtins::PyBoundMethod::new(target, func)
.into_ref(&vm.ctx)
.into()
self.push_value(func); // callable (deeper)
self.push_value(target); // self (TOS)
}
Some(PyMethod::Attribute(bound)) => {
self.push_value(bound); // callable (deeper)
self.push_null(); // NULL (TOS)
}
Some(PyMethod::Attribute(bound)) => bound,
None => {
return Err(vm.new_type_error(get_special_method_error_msg(
oparg,
@@ -2850,18 +2970,24 @@ impl ExecutingFrame<'_> {
)));
}
};
self.push_value(bound);
Ok(None)
}
Instruction::MakeFunction => self.execute_make_function(vm),
Instruction::MakeCell { .. } => {
// Cell creation is handled at frame creation time in RustPython
Instruction::MakeCell { i } => {
// Wrap the current slot value (if any) in a new PyCell.
// For merged cells (LOCAL|CELL), this wraps the argument value.
// For non-merged cells, this creates an empty cell.
let idx = i.get(arg).as_usize();
let fastlocals = self.localsplus.fastlocals_mut();
let initial = fastlocals[idx].take();
let cell = PyCell::new(initial).into_ref(&vm.ctx).into();
fastlocals[idx] = Some(cell);
Ok(None)
}
Instruction::MapAdd { i } => {
let value = self.pop_value();
let key = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let dict: &Py<PyDict> = unsafe {
// SAFETY: trust compiler
obj.downcast_unchecked_ref()
@@ -3192,7 +3318,7 @@ impl ExecutingFrame<'_> {
}
Instruction::SetAdd { i } => {
let item = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let set: &Py<PySet> = unsafe {
// SAFETY: trust compiler
obj.downcast_unchecked_ref()
@@ -3202,7 +3328,7 @@ impl ExecutingFrame<'_> {
}
Instruction::SetUpdate { i } => {
let iterable = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let set: &Py<PySet> = unsafe {
// SAFETY: compiler guarantees correct type
obj.downcast_unchecked_ref()
@@ -3294,13 +3420,14 @@ impl ExecutingFrame<'_> {
}
Instruction::StoreDeref { i } => {
let value = self.pop_value();
self.cell_ref(i.get(arg) as usize).set(Some(value));
self.cell_ref(i.get(arg).as_usize()).set(Some(value));
Ok(None)
}
Instruction::StoreFast { var_num } => {
let value = self.pop_value();
// pop_value_opt: allows NULL from LoadFastAndClear restore path
let value = self.pop_value_opt();
let fastlocals = self.localsplus.fastlocals_mut();
fastlocals[var_num.get(arg)] = Some(value);
fastlocals[var_num.get(arg)] = value;
Ok(None)
}
Instruction::StoreFastLoadFast { var_nums } => {
@@ -3318,11 +3445,12 @@ impl ExecutingFrame<'_> {
Instruction::StoreFastStoreFast { var_nums } => {
let oparg = var_nums.get(arg);
let (idx1, idx2) = oparg.indexes();
let value1 = self.pop_value();
let value2 = self.pop_value();
// pop_value_opt: allows NULL from LoadFastAndClear restore path
let value1 = self.pop_value_opt();
let value2 = self.pop_value_opt();
let fastlocals = self.localsplus.fastlocals_mut();
fastlocals[idx1] = Some(value1);
fastlocals[idx2] = Some(value2);
fastlocals[idx1] = value1;
fastlocals[idx2] = value2;
Ok(None)
}
Instruction::StoreGlobal { namei: idx } => {
@@ -3393,29 +3521,33 @@ impl ExecutingFrame<'_> {
self.unpack_sequence(expected, vm)
}
Instruction::WithExceptStart => {
// Stack: [..., __exit__, lasti, prev_exc, exc]
// Call __exit__(type, value, tb) and push result
// __exit__ is at TOS-3 (below lasti, prev_exc, and exc)
// Stack: [..., exit_func, self_or_null, lasti, prev_exc, exc]
// exit_func at TOS-4, self_or_null at TOS-3
let exc = vm.current_exception();
let stack_len = self.localsplus.stack_len();
let exit = expect_unchecked(
self.localsplus.stack_index(stack_len - 4).clone(),
"WithExceptStart: __exit__ is NULL",
let exit_func = expect_unchecked(
self.localsplus.stack_index(stack_len - 5).clone(),
"WithExceptStart: exit_func is NULL",
);
let self_or_null = self.localsplus.stack_index(stack_len - 4).clone();
let args = if let Some(ref exc) = exc {
let (tp, val, tb) = if let Some(ref exc) = exc {
vm.split_exception(exc.clone())
} else {
(vm.ctx.none(), vm.ctx.none(), vm.ctx.none())
};
let exit_res = exit.call(args, vm)?;
// Push result on top of stack
let exit_res = if let Some(self_exit) = self_or_null {
exit_func.call((self_exit.to_pyobj(), tp, val, tb), vm)?
} else {
exit_func.call((tp, val, tb), vm)?
};
self.push_value(exit_res);
Ok(None)
}
Instruction::YieldValue { arg: oparg } => {
Instruction::YieldValue { .. } => {
debug_assert!(
self.localsplus
.stack_as_slice()
@@ -3424,21 +3556,12 @@ impl ExecutingFrame<'_> {
.all(|sr| !sr.is_borrowed()),
"borrowed refs on stack at yield point"
);
let value = self.pop_value();
// arg=0: direct yield (wrapped for async generators)
// arg=1: yield from await/yield-from (NOT wrapped)
let wrap = oparg.get(arg) == 0;
let value = if wrap && self.code.flags.contains(bytecode::CodeFlags::COROUTINE) {
PyAsyncGenWrappedValue(value).into_pyobject(vm)
} else {
value
};
Ok(Some(ExecutionResult::Yield(value)))
Ok(Some(ExecutionResult::Yield(self.pop_value())))
}
Instruction::Send { .. } => {
// (receiver, v -- receiver, retval)
self.adaptive(|s, ii, cb| s.specialize_send(vm, ii, cb));
let exit_label = bytecode::Label::new(self.lasti() + 1 + u32::from(arg));
let exit_label = bytecode::Label::from_u32(self.lasti() + 1 + u32::from(arg));
let receiver = self.nth_value(1);
let can_fast_send = !self.specialization_eval_frame_active(vm)
&& (receiver.downcast_ref_if_exact::<PyGenerator>(vm).is_some()
@@ -3476,7 +3599,7 @@ impl ExecutingFrame<'_> {
}
}
Instruction::SendGen => {
let exit_label = bytecode::Label::new(self.lasti() + 1 + u32::from(arg));
let exit_label = bytecode::Label::from_u32(self.lasti() + 1 + u32::from(arg));
// Stack: [receiver, val] — peek receiver before popping
let receiver = self.nth_value(1);
let can_fast_send = !self.specialization_eval_frame_active(vm)
@@ -3607,7 +3730,7 @@ impl ExecutingFrame<'_> {
}
// Specialized LOAD_ATTR opcodes
Instruction::LoadAttrMethodNoDict => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let owner = self.top_value();
@@ -3626,7 +3749,7 @@ impl ExecutingFrame<'_> {
}
}
Instruction::LoadAttrMethodLazyDict => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let owner = self.top_value();
@@ -3646,7 +3769,7 @@ impl ExecutingFrame<'_> {
}
}
Instruction::LoadAttrMethodWithValues => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let attr_name = self.code.names[oparg.name_idx() as usize];
@@ -3681,7 +3804,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrInstanceValue => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let attr_name = self.code.names[oparg.name_idx() as usize];
@@ -3703,7 +3826,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrWithHint => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let attr_name = self.code.names[oparg.name_idx() as usize];
@@ -3728,7 +3851,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrModule => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let attr_name = self.code.names[oparg.name_idx() as usize];
@@ -3752,7 +3875,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrNondescriptorNoDict => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let owner = self.top_value();
@@ -3774,7 +3897,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrNondescriptorWithValues => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let attr_name = self.code.names[oparg.name_idx() as usize];
@@ -3812,7 +3935,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrClass => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let owner = self.top_value();
@@ -3835,7 +3958,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrClassWithMetaclassCheck => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let owner = self.top_value();
@@ -3861,7 +3984,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrGetattributeOverridden => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let owner = self.top_value();
let type_version = self.code.instructions.read_cache_u32(cache_base + 1);
@@ -3888,7 +4011,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrSlot => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let owner = self.top_value();
@@ -3912,7 +4035,7 @@ impl ExecutingFrame<'_> {
self.load_attr_slow(vm, oparg)
}
Instruction::LoadAttrProperty => {
let oparg = LoadAttr::new(u32::from(arg));
let oparg = LoadAttr::from_u32(u32::from(arg));
let cache_base = self.lasti() as usize;
let owner = self.top_value();
@@ -5114,7 +5237,7 @@ impl ExecutingFrame<'_> {
return Ok(None);
}
}
let oparg = LoadSuperAttr::new(oparg);
let oparg = LoadSuperAttr::from_u32(oparg);
self.load_super_attr(vm, oparg)
}
Instruction::LoadSuperAttrMethod => {
@@ -5181,7 +5304,7 @@ impl ExecutingFrame<'_> {
return Ok(None);
}
}
let oparg = LoadSuperAttr::new(oparg);
let oparg = LoadSuperAttr::from_u32(oparg);
self.load_super_attr(vm, oparg)
}
Instruction::CompareOpInt => {
@@ -5448,7 +5571,7 @@ impl ExecutingFrame<'_> {
self.unpack_sequence(size as u32, vm)
}
Instruction::ForIterRange => {
let target = bytecode::Label::new(self.lasti() + 1 + u32::from(arg));
let target = bytecode::Label::from_u32(self.lasti() + 1 + u32::from(arg));
let iter = self.top_value();
if let Some(range_iter) = iter.downcast_ref_if_exact::<PyRangeIterator>(vm) {
if let Some(value) = range_iter.fast_next() {
@@ -5463,7 +5586,7 @@ impl ExecutingFrame<'_> {
}
}
Instruction::ForIterList => {
let target = bytecode::Label::new(self.lasti() + 1 + u32::from(arg));
let target = bytecode::Label::from_u32(self.lasti() + 1 + u32::from(arg));
let iter = self.top_value();
if let Some(list_iter) = iter.downcast_ref_if_exact::<PyListIterator>(vm) {
if let Some(value) = list_iter.fast_next() {
@@ -5478,7 +5601,7 @@ impl ExecutingFrame<'_> {
}
}
Instruction::ForIterTuple => {
let target = bytecode::Label::new(self.lasti() + 1 + u32::from(arg));
let target = bytecode::Label::from_u32(self.lasti() + 1 + u32::from(arg));
let iter = self.top_value();
if let Some(tuple_iter) = iter.downcast_ref_if_exact::<PyTupleIterator>(vm) {
if let Some(value) = tuple_iter.fast_next() {
@@ -5493,7 +5616,7 @@ impl ExecutingFrame<'_> {
}
}
Instruction::ForIterGen => {
let target = bytecode::Label::new(self.lasti() + 1 + u32::from(arg));
let target = bytecode::Label::from_u32(self.lasti() + 1 + u32::from(arg));
let iter = self.top_value();
if self.specialization_eval_frame_active(vm) {
self.execute_for_iter(vm, target)?;
@@ -5667,13 +5790,6 @@ impl ExecutingFrame<'_> {
let offset = (self.lasti() - 1) * 2;
monitoring::fire_py_yield(vm, self.code, offset, &value)?;
}
let oparg = u32::from(arg);
let wrap = oparg == 0;
let value = if wrap && self.code.flags.contains(bytecode::CodeFlags::COROUTINE) {
PyAsyncGenWrappedValue(value).into_pyobject(vm)
} else {
value
};
Ok(Some(ExecutionResult::Yield(value)))
}
Instruction::InstrumentedCall => {
@@ -5735,7 +5851,7 @@ impl ExecutingFrame<'_> {
Instruction::InstrumentedJumpForward => {
let src_offset = (self.lasti() - 1) * 2;
let target_idx = self.lasti() + u32::from(arg);
let target = bytecode::Label::new(target_idx);
let target = bytecode::Label::from_u32(target_idx);
self.jump(target);
if self.monitoring_mask & monitoring::EVENT_JUMP != 0 {
monitoring::fire_jump(vm, self.code, src_offset, target.as_u32() * 2)?;
@@ -5745,7 +5861,7 @@ impl ExecutingFrame<'_> {
Instruction::InstrumentedJumpBackward => {
let src_offset = (self.lasti() - 1) * 2;
let target_idx = self.lasti() + 1 - u32::from(arg);
let target = bytecode::Label::new(target_idx);
let target = bytecode::Label::from_u32(target_idx);
self.jump(target);
if self.monitoring_mask & monitoring::EVENT_JUMP != 0 {
monitoring::fire_jump(vm, self.code, src_offset, target.as_u32() * 2)?;
@@ -5754,7 +5870,7 @@ impl ExecutingFrame<'_> {
}
Instruction::InstrumentedForIter => {
let src_offset = (self.lasti() - 1) * 2;
let target = bytecode::Label::new(self.lasti() + 1 + u32::from(arg));
let target = bytecode::Label::from_u32(self.lasti() + 1 + u32::from(arg));
let continued = self.execute_for_iter(vm, target)?;
if continued {
if self.monitoring_mask & monitoring::EVENT_BRANCH_LEFT != 0 {
@@ -5804,7 +5920,7 @@ impl ExecutingFrame<'_> {
let obj = self.pop_value();
let value = obj.try_to_bool(vm)?;
if value {
self.jump(bytecode::Label::new(target_idx));
self.jump(bytecode::Label::from_u32(target_idx));
if self.monitoring_mask & monitoring::EVENT_BRANCH_RIGHT != 0 {
monitoring::fire_branch_right(vm, self.code, src_offset, target_idx * 2)?;
}
@@ -5817,7 +5933,7 @@ impl ExecutingFrame<'_> {
let obj = self.pop_value();
let value = obj.try_to_bool(vm)?;
if !value {
self.jump(bytecode::Label::new(target_idx));
self.jump(bytecode::Label::from_u32(target_idx));
if self.monitoring_mask & monitoring::EVENT_BRANCH_RIGHT != 0 {
monitoring::fire_branch_right(vm, self.code, src_offset, target_idx * 2)?;
}
@@ -5829,7 +5945,7 @@ impl ExecutingFrame<'_> {
let target_idx = self.lasti() + 1 + u32::from(arg);
let value = self.pop_value();
if vm.is_none(&value) {
self.jump(bytecode::Label::new(target_idx));
self.jump(bytecode::Label::from_u32(target_idx));
if self.monitoring_mask & monitoring::EVENT_BRANCH_RIGHT != 0 {
monitoring::fire_branch_right(vm, self.code, src_offset, target_idx * 2)?;
}
@@ -5841,7 +5957,7 @@ impl ExecutingFrame<'_> {
let target_idx = self.lasti() + 1 + u32::from(arg);
let value = self.pop_value();
if !vm.is_none(&value) {
self.jump(bytecode::Label::new(target_idx));
self.jump(bytecode::Label::from_u32(target_idx));
if self.monitoring_mask & monitoring::EVENT_BRANCH_RIGHT != 0 {
monitoring::fire_branch_right(vm, self.code, src_offset, target_idx * 2)?;
}
@@ -6233,7 +6349,7 @@ impl ExecutingFrame<'_> {
self.push_value(exception.into());
// 4. Jump to handler
self.jump(bytecode::Label::new(entry.target));
self.jump(bytecode::Label::from_u32(entry.target));
Ok(None)
} else {
@@ -6838,7 +6954,7 @@ impl ExecutingFrame<'_> {
bytecode::Instruction::EndFor | bytecode::Instruction::InstrumentedEndFor
)
{
return bytecode::Label::new(target.as_u32() + 1);
return bytecode::Label::from_u32(target.as_u32() + 1);
}
target
}
@@ -8819,7 +8935,7 @@ impl ExecutingFrame<'_> {
unit.op,
bytecode::Instruction::EndFor | bytecode::Instruction::InstrumentedEndFor
) {
bytecode::Label::new(target.as_u32() + 1)
bytecode::Label::from_u32(target.as_u32() + 1)
} else {
target
}

View File

@@ -457,12 +457,20 @@ impl GcState {
}
// Step 3: Subtract internal references
// Pre-compute referent pointers once per object so that both step 3
// (subtract refs) and step 4 (BFS reachability) see the same snapshot
// of each object's children. Without this, a dict whose write lock is
// held during one traversal but not the other can yield inconsistent
// results, causing live objects to be incorrectly collected.
let mut referents_map: std::collections::HashMap<GcPtr, Vec<NonNull<PyObject>>> =
std::collections::HashMap::new();
for &ptr in &collecting {
let obj = unsafe { ptr.0.as_ref() };
if obj.strong_count() == 0 {
continue;
}
let referent_ptrs = unsafe { obj.gc_get_referent_ptrs() };
referents_map.insert(ptr, referent_ptrs.clone());
for child_ptr in referent_ptrs {
let gc_ptr = GcPtr(child_ptr);
if collecting.contains(&gc_ptr)
@@ -487,7 +495,13 @@ impl GcState {
while let Some(ptr) = worklist.pop() {
let obj = unsafe { ptr.0.as_ref() };
if obj.is_gc_tracked() {
let referent_ptrs = unsafe { obj.gc_get_referent_ptrs() };
// Reuse the pre-computed referent pointers from step 3.
// For objects that were skipped in step 3 (strong_count was 0),
// compute them now as a fallback.
let referent_ptrs = referents_map
.get(&ptr)
.cloned()
.unwrap_or_else(|| unsafe { obj.gc_get_referent_ptrs() });
for child_ptr in referent_ptrs {
let gc_ptr = GcPtr(child_ptr);
if collecting.contains(&gc_ptr) && reachable.insert(gc_ptr) {

View File

@@ -200,33 +200,34 @@ impl PyNumberMethods {
}
}
/// Matches the NB_* constants ordering from opcode.h / BinaryOperator.
#[derive(Copy, Clone)]
pub enum PyNumberBinaryOp {
Add,
Subtract,
And,
FloorDivide,
Lshift,
MatrixMultiply,
Multiply,
Remainder,
Divmod,
Lshift,
Rshift,
And,
Xor,
Or,
Rshift,
Subtract,
TrueDivide,
Xor,
InplaceAdd,
InplaceSubtract,
InplaceAnd,
InplaceFloorDivide,
InplaceLshift,
InplaceMatrixMultiply,
InplaceMultiply,
InplaceRemainder,
InplaceLshift,
InplaceRshift,
InplaceAnd,
InplaceXor,
InplaceOr,
FloorDivide,
TrueDivide,
InplaceFloorDivide,
InplaceRshift,
InplaceSubtract,
InplaceTrueDivide,
MatrixMultiply,
InplaceMatrixMultiply,
InplaceXor,
Divmod,
}
impl PyNumberBinaryOp {

View File

@@ -298,7 +298,9 @@ impl PyObject {
) -> PyResult<Either<PyObjectRef, bool>> {
let swapped = op.swapped();
let call_cmp = |obj: &Self, other: &Self, op| {
let cmp = obj.class().slots.richcompare.load().unwrap();
let Some(cmp) = obj.class().slots.richcompare.load() else {
return Ok(PyArithmeticValue::NotImplemented);
};
let r = match cmp(obj, other, op, vm)? {
Either::A(obj) => PyArithmeticValue::from_object(vm, obj).map(Either::A),
Either::B(arithmetic) => arithmetic.map(Either::B),

View File

@@ -445,6 +445,11 @@ mod _winapi {
for (_, entry) in entries {
out.push(entry);
}
// Each entry ends with \0, so one more \0 terminates the block.
// For empty env, we need \0\0 as a valid empty environment block.
if out.is_empty() {
out.push_str("\0");
}
out.push_str("\0");
Ok(out.into_vec())
}
@@ -610,6 +615,78 @@ mod _winapi {
})
}
#[pyfunction]
fn CreateJobObject(
_security_attributes: PyObjectRef,
name: OptionalArg<Option<PyStrRef>>,
vm: &VirtualMachine,
) -> PyResult<WinHandle> {
let handle = unsafe {
match name.flatten() {
Some(name) => {
let name_wide = name.as_wtf8().to_wide_with_nul();
windows_sys::Win32::System::JobObjects::CreateJobObjectW(
null(),
name_wide.as_ptr(),
)
}
None => windows_sys::Win32::System::JobObjects::CreateJobObjectW(null(), null()),
}
};
if handle.is_null() {
return Err(vm.new_last_os_error());
}
Ok(WinHandle(handle))
}
#[pyfunction]
fn AssignProcessToJobObject(
job: WinHandle,
process: WinHandle,
vm: &VirtualMachine,
) -> PyResult<()> {
let ret = unsafe {
windows_sys::Win32::System::JobObjects::AssignProcessToJobObject(job.0, process.0)
};
if ret == 0 {
return Err(vm.new_last_os_error());
}
Ok(())
}
#[pyfunction]
fn TerminateJobObject(job: WinHandle, exit_code: u32, vm: &VirtualMachine) -> PyResult<()> {
let ret =
unsafe { windows_sys::Win32::System::JobObjects::TerminateJobObject(job.0, exit_code) };
if ret == 0 {
return Err(vm.new_last_os_error());
}
Ok(())
}
#[pyfunction]
fn SetJobObjectKillOnClose(job: WinHandle, vm: &VirtualMachine) -> PyResult<()> {
use windows_sys::Win32::System::JobObjects::{
JOBOBJECT_EXTENDED_LIMIT_INFORMATION, JobObjectExtendedLimitInformation,
SetInformationJobObject,
};
let mut info: JOBOBJECT_EXTENDED_LIMIT_INFORMATION = unsafe { core::mem::zeroed() };
info.BasicLimitInformation.LimitFlags =
windows_sys::Win32::System::JobObjects::JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
let ret = unsafe {
SetInformationJobObject(
job.0,
JobObjectExtendedLimitInformation,
&info as *const _ as *const core::ffi::c_void,
core::mem::size_of::<JOBOBJECT_EXTENDED_LIMIT_INFORMATION>() as u32,
)
};
if ret == 0 {
return Err(vm.new_last_os_error());
}
Ok(())
}
#[pyfunction]
fn GetModuleFileName(handle: isize, vm: &VirtualMachine) -> PyResult<String> {
let mut path: Vec<u16> = vec![0; MAX_PATH as usize];

View File

@@ -39,12 +39,11 @@ pub mod errors {
WSAENOMORE, WSAENOPROTOOPT, WSAENOTCONN, WSAENOTEMPTY, WSAENOTSOCK, WSAEOPNOTSUPP,
WSAEPFNOSUPPORT, WSAEPROCLIM, WSAEPROTONOSUPPORT, WSAEPROTOTYPE,
WSAEPROVIDERFAILEDINIT, WSAEREFUSED, WSAEREMOTE, WSAESHUTDOWN, WSAESOCKTNOSUPPORT,
WSAESTALE, WSAETIMEDOUT, WSAETOOMANYREFS, WSAEUSERS, WSAEWOULDBLOCK, WSAHOST_NOT_FOUND,
WSAID_ACCEPTEX, WSAID_CONNECTEX, WSAID_DISCONNECTEX, WSAID_GETACCEPTEXSOCKADDRS,
WSAID_TRANSMITFILE, WSAID_TRANSMITPACKETS, WSAID_WSAPOLL, WSAID_WSARECVMSG, WSANO_DATA,
WSANO_RECOVERY, WSANOTINITIALISED, WSAPROTOCOL_LEN, WSASERVICE_NOT_FOUND,
WSASYS_STATUS_LEN, WSASYSCALLFAILURE, WSASYSNOTREADY, WSATRY_AGAIN, WSATYPE_NOT_FOUND,
WSAVERNOTSUPPORTED,
WSAESTALE, WSAETIMEDOUT, WSAETOOMANYREFS, WSAEUSERS, WSAEWOULDBLOCK, WSAID_ACCEPTEX,
WSAID_CONNECTEX, WSAID_DISCONNECTEX, WSAID_GETACCEPTEXSOCKADDRS, WSAID_TRANSMITFILE,
WSAID_TRANSMITPACKETS, WSAID_WSAPOLL, WSAID_WSARECVMSG, WSANO_DATA, WSANO_RECOVERY,
WSANOTINITIALISED, WSAPROTOCOL_LEN, WSASERVICE_NOT_FOUND, WSASYS_STATUS_LEN,
WSASYSCALLFAILURE, WSASYSNOTREADY, WSATRY_AGAIN, WSATYPE_NOT_FOUND, WSAVERNOTSUPPORTED,
},
};
#[cfg(windows)]
@@ -64,8 +63,6 @@ pub mod errors {
ETIMEDOUT, ETOOMANYREFS, EUSERS, EWOULDBLOCK,
// TODO: EBADF should be here once winerrs are translated to errnos but it messes up some things atm
}
#[cfg(windows)]
pub const WSAHOS: i32 = WSAHOST_NOT_FOUND;
}
#[cfg(any(unix, windows, target_os = "wasi"))]
@@ -566,7 +563,7 @@ const ERROR_CODES: &[(&str, i32)] = &[
e!(cfg(windows), WSAEDISCON),
e!(cfg(windows), WSAEINTR),
e!(cfg(windows), WSAEPROTOTYPE),
e!(cfg(windows), WSAHOS),
// TODO: e!(cfg(windows), WSAHOS),
e!(cfg(windows), WSAEADDRINUSE),
e!(cfg(windows), WSAEADDRNOTAVAIL),
e!(cfg(windows), WSAEALREADY),

View File

@@ -5,20 +5,19 @@ pub(crate) use decl::module_def;
mod decl {
use crate::builtins::code::{CodeObject, Literal, PyObjBag};
use crate::class::StaticType;
use crate::common::wtf8::Wtf8;
use crate::{
PyObjectRef, PyResult, TryFromObject, VirtualMachine,
builtins::{
PyBool, PyByteArray, PyBytes, PyCode, PyComplex, PyDict, PyEllipsis, PyFloat,
PyFrozenSet, PyInt, PyList, PyNone, PySet, PyStopIteration, PyStr, PyTuple,
},
common::wtf8::Wtf8,
convert::ToPyObject,
function::{ArgBytesLike, OptionalArg},
object::{AsObject, PyPayload},
protocol::PyBuffer,
};
use malachite_bigint::BigInt;
use num_complex::Complex64;
use num_traits::Zero;
use rustpython_compiler_core::marshal;
@@ -91,34 +90,290 @@ mod decl {
}
}
#[pyfunction]
fn dumps(
#[derive(FromArgs)]
struct DumpsArgs {
value: PyObjectRef,
#[pyarg(any, optional)]
_version: OptionalArg<i32>,
vm: &VirtualMachine,
) -> PyResult<PyBytes> {
use marshal::Dumpable;
let mut buf = Vec::new();
value
.with_dump(|val| marshal::serialize_value(&mut buf, val))
.unwrap_or_else(Err)
.map_err(|DumpError| {
vm.new_not_implemented_error(
"TODO: not implemented yet or marshal unsupported type",
)
})?;
Ok(PyBytes::from(buf))
#[pyarg(named, default = true)]
allow_code: bool,
}
#[pyfunction]
fn dump(
value: PyObjectRef,
f: PyObjectRef,
version: OptionalArg<i32>,
fn dumps(args: DumpsArgs, vm: &VirtualMachine) -> PyResult<PyBytes> {
let DumpsArgs {
value,
allow_code,
_version,
} = args;
let version = _version.unwrap_or(marshal::FORMAT_VERSION as i32);
if !allow_code {
check_no_code(&value, vm)?;
}
check_exact_type(&value, vm)?;
let mut buf = Vec::new();
let mut refs = if version >= 3 {
Some(WriterRefTable::new())
} else {
None
};
write_object(&mut buf, &value, &mut refs, version, vm)?;
Ok(PyBytes::from(buf))
}
struct WriterRefTable {
map: std::collections::HashMap<usize, u32>,
next_idx: u32,
}
impl WriterRefTable {
fn new() -> Self {
Self {
map: std::collections::HashMap::new(),
next_idx: 0,
}
}
fn try_ref(&mut self, buf: &mut Vec<u8>, obj: &PyObjectRef) -> bool {
use marshal::Write;
let id = obj.get_id();
if let Some(&idx) = self.map.get(&id) {
buf.write_u8(b'r');
buf.write_u32(idx);
true
} else {
false
}
}
fn reserve(&mut self, obj: &PyObjectRef) -> u32 {
let idx = self.next_idx;
self.map.insert(obj.get_id(), idx);
self.next_idx += 1;
idx
}
}
fn write_object(
buf: &mut Vec<u8>,
obj: &PyObjectRef,
refs: &mut Option<WriterRefTable>,
version: i32,
vm: &VirtualMachine,
) -> PyResult<()> {
let dumped = dumps(value, version, vm)?;
vm.call_method(&f, "write", (dumped,))?;
write_object_depth(
buf,
obj,
refs,
version,
vm,
marshal::MAX_MARSHAL_STACK_DEPTH,
)
}
fn write_object_depth(
buf: &mut Vec<u8>,
obj: &PyObjectRef,
refs: &mut Option<WriterRefTable>,
version: i32,
vm: &VirtualMachine,
depth: usize,
) -> PyResult<()> {
use marshal::Write;
if depth == 0 {
return Err(vm.new_value_error("object too deeply nested to marshal".to_string()));
}
// Singletons: no FLAG_REF needed
let is_singleton = vm.is_none(obj)
|| obj.class().is(PyBool::static_type())
|| obj.is(PyStopIteration::static_type())
|| obj.downcast_ref::<crate::builtins::PyEllipsis>().is_some();
// FLAG_REF: check if already written, otherwise reserve slot
if !is_singleton
&& let Some(rt) = refs.as_mut()
&& rt.try_ref(buf, obj)
{
return Ok(());
}
let type_pos = buf.len();
let use_ref = refs.is_some() && !is_singleton;
if use_ref {
refs.as_mut().unwrap().reserve(obj);
}
if vm.is_none(obj) {
buf.write_u8(b'N');
} else if obj.is(PyStopIteration::static_type()) {
buf.write_u8(b'S');
} else if obj.class().is(PyBool::static_type()) {
let val = obj
.downcast_ref::<PyInt>()
.is_some_and(|i| !i.as_bigint().is_zero());
buf.write_u8(if val { b'T' } else { b'F' });
} else if obj.downcast_ref::<crate::builtins::PyEllipsis>().is_some() {
buf.write_u8(b'.');
} else if let Some(i) = obj.downcast_ref::<PyInt>() {
// TYPE_INT for i32 range, TYPE_LONG for larger
if let Ok(val) = i32::try_from(i.as_bigint()) {
buf.write_u8(b'i');
buf.write_u32(val as u32);
} else {
buf.write_u8(b'l');
let (sign, raw) = i.as_bigint().to_bytes_le();
let mut digits = Vec::new();
let mut accum: u32 = 0;
let mut bits = 0u32;
for &byte in &raw {
accum |= (byte as u32) << bits;
bits += 8;
while bits >= 15 {
digits.push((accum & 0x7fff) as u16);
accum >>= 15;
bits -= 15;
}
}
if accum > 0 || digits.is_empty() {
digits.push(accum as u16);
}
while digits.len() > 1 && *digits.last().unwrap() == 0 {
digits.pop();
}
let n = digits.len() as i32;
let n = if sign == malachite_bigint::Sign::Minus {
-n
} else {
n
};
buf.write_u32(n as u32);
for d in &digits {
buf.write_u16(*d);
}
}
} else if let Some(f) = obj.downcast_ref::<PyFloat>() {
buf.write_u8(b'g');
buf.write_u64(f.to_f64().to_bits());
} else if let Some(c) = obj.downcast_ref::<PyComplex>() {
buf.write_u8(b'y');
let cv = c.to_complex64();
buf.write_u64(cv.re.to_bits());
buf.write_u64(cv.im.to_bits());
} else if let Some(s) = obj.downcast_ref::<PyStr>() {
let bytes = s.as_wtf8().as_bytes();
let interned = version >= 3;
if bytes.len() < 256 && bytes.is_ascii() {
buf.write_u8(if interned { b'Z' } else { b'z' });
buf.write_u8(bytes.len() as u8);
} else {
buf.write_u8(if interned { b't' } else { b'u' });
buf.write_u32(bytes.len() as u32);
}
buf.write_slice(bytes);
} else if let Some(b) = obj.downcast_ref::<PyBytes>() {
buf.write_u8(b's');
let data = b.as_bytes();
buf.write_u32(data.len() as u32);
buf.write_slice(data);
} else if let Some(b) = obj.downcast_ref::<PyByteArray>() {
buf.write_u8(b's');
let data = b.borrow_buf();
buf.write_u32(data.len() as u32);
buf.write_slice(&data);
} else if let Some(t) = obj.downcast_ref::<PyTuple>() {
buf.write_u8(b'(');
buf.write_u32(t.len() as u32);
for elem in t.as_slice() {
write_object_depth(buf, elem, refs, version, vm, depth - 1)?;
}
} else if let Some(l) = obj.downcast_ref::<PyList>() {
buf.write_u8(b'[');
let items = l.borrow_vec();
buf.write_u32(items.len() as u32);
for elem in items.iter() {
write_object_depth(buf, elem, refs, version, vm, depth - 1)?;
}
} else if let Some(d) = obj.downcast_ref::<PyDict>() {
buf.write_u8(b'{');
for (k, v) in d.into_iter() {
write_object_depth(buf, &k, refs, version, vm, depth - 1)?;
write_object_depth(buf, &v, refs, version, vm, depth - 1)?;
}
buf.write_u8(b'0'); // TYPE_NULL terminator
} else if let Some(s) = obj.downcast_ref::<PySet>() {
buf.write_u8(b'<');
let elems = s.elements();
buf.write_u32(elems.len() as u32);
for elem in &elems {
write_object_depth(buf, elem, refs, version, vm, depth - 1)?;
}
} else if let Some(s) = obj.downcast_ref::<PyFrozenSet>() {
buf.write_u8(b'>');
let elems = s.elements();
buf.write_u32(elems.len() as u32);
for elem in &elems {
write_object_depth(buf, elem, refs, version, vm, depth - 1)?;
}
} else if let Some(co) = obj.downcast_ref::<PyCode>() {
buf.write_u8(b'c');
marshal::serialize_code(buf, &co.code);
} else if let Some(sl) = obj.downcast_ref::<crate::builtins::PySlice>() {
if version < 5 {
return Err(vm.new_value_error("unmarshallable object".to_string()));
}
buf.write_u8(b':');
let none: PyObjectRef = vm.ctx.none();
write_object_depth(
buf,
sl.start.as_ref().unwrap_or(&none),
refs,
version,
vm,
depth - 1,
)?;
write_object_depth(buf, &sl.stop, refs, version, vm, depth - 1)?;
write_object_depth(
buf,
sl.step.as_ref().unwrap_or(&none),
refs,
version,
vm,
depth - 1,
)?;
} else if let Ok(bytes_like) = ArgBytesLike::try_from_object(vm, obj.clone()) {
buf.write_u8(b's');
let data = bytes_like.borrow_buf();
buf.write_u32(data.len() as u32);
buf.write_slice(&data);
} else {
return Err(vm.new_value_error("unmarshallable object".to_string()));
}
if use_ref {
buf[type_pos] |= marshal::FLAG_REF;
}
Ok(())
}
#[derive(FromArgs)]
struct DumpArgs {
value: PyObjectRef,
f: PyObjectRef,
#[pyarg(any, optional)]
_version: OptionalArg<i32>,
#[pyarg(named, default = true)]
allow_code: bool,
}
#[pyfunction]
fn dump(args: DumpArgs, vm: &VirtualMachine) -> PyResult<()> {
let dumped = dumps(
DumpsArgs {
value: args.value,
_version: args._version,
allow_code: args.allow_code,
},
vm,
)?;
vm.call_method(&args.f, "write", (dumped,))?;
Ok(())
}
@@ -132,121 +387,219 @@ mod decl {
fn make_bool(&self, value: bool) -> Self::Value {
self.0.ctx.new_bool(value).into()
}
fn make_none(&self) -> Self::Value {
self.0.ctx.none()
}
fn make_ellipsis(&self) -> Self::Value {
self.0.ctx.ellipsis.clone().into()
}
fn make_float(&self, value: f64) -> Self::Value {
self.0.ctx.new_float(value).into()
}
fn make_complex(&self, value: Complex64) -> Self::Value {
fn make_complex(&self, value: num_complex::Complex64) -> Self::Value {
self.0.ctx.new_complex(value).into()
}
fn make_str(&self, value: &Wtf8) -> Self::Value {
self.0.ctx.new_str(value).into()
}
fn make_bytes(&self, value: &[u8]) -> Self::Value {
self.0.ctx.new_bytes(value.to_vec()).into()
}
fn make_int(&self, value: BigInt) -> Self::Value {
self.0.ctx.new_int(value).into()
}
fn make_tuple(&self, elements: impl Iterator<Item = Self::Value>) -> Self::Value {
let elements = elements.collect();
self.0.ctx.new_tuple(elements).into()
self.0.ctx.new_tuple(elements.collect()).into()
}
fn make_code(&self, code: CodeObject) -> Self::Value {
self.0.ctx.new_code(code).into()
}
fn make_stop_iter(&self) -> Result<Self::Value, marshal::MarshalError> {
Ok(self.0.ctx.exceptions.stop_iteration.to_owned().into())
}
fn make_list(
&self,
it: impl Iterator<Item = Self::Value>,
) -> Result<Self::Value, marshal::MarshalError> {
Ok(self.0.ctx.new_list(it.collect()).into())
}
fn make_set(
&self,
it: impl Iterator<Item = Self::Value>,
) -> Result<Self::Value, marshal::MarshalError> {
let vm = self.0;
let set = PySet::default().into_ref(&vm.ctx);
let set = PySet::default().into_ref(&self.0.ctx);
for elem in it {
set.add(elem, vm).unwrap()
set.add(elem, self.0).unwrap()
}
Ok(set.into())
}
fn make_frozenset(
&self,
it: impl Iterator<Item = Self::Value>,
) -> Result<Self::Value, marshal::MarshalError> {
let vm = self.0;
Ok(PyFrozenSet::from_iter(vm, it).unwrap().to_pyobject(vm))
Ok(PyFrozenSet::from_iter(self.0, it)
.unwrap()
.to_pyobject(self.0))
}
fn make_dict(
&self,
it: impl Iterator<Item = (Self::Value, Self::Value)>,
) -> Result<Self::Value, marshal::MarshalError> {
let vm = self.0;
let dict = vm.ctx.new_dict();
let dict = self.0.ctx.new_dict();
for (k, v) in it {
dict.set_item(&*k, v, vm).unwrap()
dict.set_item(&*k, v, self.0).unwrap()
}
Ok(dict.into())
}
fn make_slice(
&self,
start: Self::Value,
stop: Self::Value,
step: Self::Value,
) -> Result<Self::Value, marshal::MarshalError> {
use crate::builtins::PySlice;
let vm = self.0;
Ok(PySlice {
start: if vm.is_none(&start) {
None
} else {
Some(start)
},
stop,
step: if vm.is_none(&step) { None } else { Some(step) },
}
.into_ref(&vm.ctx)
.into())
}
fn constant_bag(self) -> Self::ConstantBag {
PyObjBag(&self.0.ctx)
}
}
#[pyfunction]
fn loads(pybuffer: PyBuffer, vm: &VirtualMachine) -> PyResult<PyObjectRef> {
let buf = pybuffer.as_contiguous().ok_or_else(|| {
vm.new_buffer_error("Buffer provided to marshal.loads() is not contiguous")
})?;
marshal::deserialize_value(&mut &buf[..], PyMarshalBag(vm)).map_err(|e| match e {
marshal::MarshalError::Eof => vm.new_exception_msg(
vm.ctx.exceptions.eof_error.to_owned(),
"marshal data too short".into(),
),
marshal::MarshalError::InvalidBytecode => {
vm.new_value_error("Couldn't deserialize python bytecode")
}
marshal::MarshalError::InvalidUtf8 => {
vm.new_value_error("invalid utf8 in marshalled string")
}
marshal::MarshalError::InvalidLocation => {
vm.new_value_error("invalid location in marshalled object")
}
marshal::MarshalError::BadType => {
vm.new_value_error("bad marshal data (unknown type code)")
}
})
#[derive(FromArgs)]
struct LoadsArgs {
#[pyarg(any)]
data: PyBuffer,
#[pyarg(named, default = true)]
allow_code: bool,
}
#[pyfunction]
fn load(f: PyObjectRef, vm: &VirtualMachine) -> PyResult<PyObjectRef> {
let read_res = vm.call_method(&f, "read", ())?;
fn loads(args: LoadsArgs, vm: &VirtualMachine) -> PyResult<PyObjectRef> {
let LoadsArgs {
data: pybuffer,
allow_code,
} = args;
let buf = pybuffer.as_contiguous().ok_or_else(|| {
vm.new_buffer_error("Buffer provided to marshal.loads() is not contiguous")
})?;
let result =
marshal::deserialize_value(&mut &buf[..], PyMarshalBag(vm)).map_err(|e| match e {
marshal::MarshalError::Eof => vm.new_exception_msg(
vm.ctx.exceptions.eof_error.to_owned(),
"marshal data too short".into(),
),
_ => vm.new_value_error("bad marshal data"),
})?;
if !allow_code {
check_no_code(&result, vm)?;
}
Ok(result)
}
#[derive(FromArgs)]
struct LoadArgs {
f: PyObjectRef,
#[pyarg(named, default = true)]
allow_code: bool,
}
#[pyfunction]
fn load(args: LoadArgs, vm: &VirtualMachine) -> PyResult<PyObjectRef> {
// Read from file object into a buffer, one object at a time.
// We read all available data, deserialize one object, then seek
// back to just after the consumed bytes.
let tell_before = vm
.call_method(&args.f, "tell", ())?
.try_into_value::<i64>(vm)?;
let read_res = vm.call_method(&args.f, "read", ())?;
let bytes = ArgBytesLike::try_from_object(vm, read_res)?;
loads(PyBuffer::from(bytes), vm)
let buf = bytes.borrow_buf();
let mut rdr: &[u8] = &buf;
let len_before = rdr.len();
let result =
marshal::deserialize_value(&mut rdr, PyMarshalBag(vm)).map_err(|e| match e {
marshal::MarshalError::Eof => vm.new_exception_msg(
vm.ctx.exceptions.eof_error.to_owned(),
"marshal data too short".into(),
),
_ => vm.new_value_error("bad marshal data"),
})?;
let consumed = len_before - rdr.len();
// Seek file to just after the consumed bytes
let new_pos = tell_before + consumed as i64;
vm.call_method(&args.f, "seek", (new_pos,))?;
if !args.allow_code {
check_no_code(&result, vm)?;
}
Ok(result)
}
/// Reject subclasses of marshallable types (int, float, complex, tuple, etc.).
/// Recursively check that no code objects are present.
fn check_no_code(obj: &PyObjectRef, vm: &VirtualMachine) -> PyResult<()> {
if obj.downcast_ref::<PyCode>().is_some() {
return Err(vm.new_value_error("unmarshalling code objects is disallowed".to_string()));
}
if let Some(tup) = obj.downcast_ref::<PyTuple>() {
for elem in tup.as_slice() {
check_no_code(elem, vm)?;
}
} else if let Some(list) = obj.downcast_ref::<PyList>() {
for elem in list.borrow_vec().iter() {
check_no_code(elem, vm)?;
}
} else if let Some(set) = obj.downcast_ref::<PySet>() {
for elem in set.elements() {
check_no_code(&elem, vm)?;
}
} else if let Some(fset) = obj.downcast_ref::<PyFrozenSet>() {
for elem in fset.elements() {
check_no_code(&elem, vm)?;
}
} else if let Some(dict) = obj.downcast_ref::<PyDict>() {
for (k, v) in dict.into_iter() {
check_no_code(&k, vm)?;
check_no_code(&v, vm)?;
}
}
Ok(())
}
fn check_exact_type(obj: &PyObjectRef, vm: &VirtualMachine) -> PyResult<()> {
let cls = obj.class();
// bool is a subclass of int but is marshallable
if cls.is(PyBool::static_type()) {
return Ok(());
}
for base in [
PyInt::static_type(),
PyFloat::static_type(),
PyComplex::static_type(),
PyTuple::static_type(),
PyList::static_type(),
PyDict::static_type(),
PySet::static_type(),
PyFrozenSet::static_type(),
] {
if cls.fast_issubclass(base) && !cls.is(base) {
return Err(vm.new_value_error("unmarshallable object".to_string()));
}
}
Ok(())
}
}

View File

@@ -321,12 +321,19 @@ pub(super) mod _os {
}
#[pyfunction]
fn write(
fd: crt_fd::Borrowed<'_>,
data: ArgBytesLike,
vm: &VirtualMachine,
) -> io::Result<usize> {
data.with_ref(|b| vm.allow_threads(|| crt_fd::write(fd, b)))
fn write(fd: crt_fd::Borrowed<'_>, data: ArgBytesLike, vm: &VirtualMachine) -> PyResult<usize> {
data.with_ref(|b| {
loop {
match vm.allow_threads(|| crt_fd::write(fd, b)) {
Ok(n) => return Ok(n),
Err(e) if e.raw_os_error() == Some(libc::EINTR) => {
vm.check_signals()?;
continue;
}
Err(e) => return Err(e.into_pyexception(vm)),
}
}
})
}
#[cfg(not(windows))]

View File

@@ -1807,7 +1807,12 @@ mod sys {
#[cfg(windows)]
#[pyclass(with(PyStructSequence))]
impl PyWindowsVersion {}
impl PyWindowsVersion {
#[pyslot]
fn slot_new(_cls: PyTypeRef, _args: FuncArgs, vm: &VirtualMachine) -> PyResult {
Err(vm.new_type_error("cannot create 'sys.getwindowsversion' instances"))
}
}
#[derive(Debug)]
#[pystruct_sequence_data(try_from_object)]

View File

@@ -1865,12 +1865,14 @@ impl PyComparisonOp {
}
pub fn eval_ord(self, ord: Ordering) -> bool {
let bit = match ord {
Ordering::Less => Self::Lt,
Ordering::Equal => Self::Eq,
Ordering::Greater => Self::Gt,
};
u8::from(self.0) & u8::from(bit.0) != 0
match self {
Self::Lt => ord == Ordering::Less,
Self::Le => ord != Ordering::Greater,
Self::Eq => ord == Ordering::Equal,
Self::Ne => ord != Ordering::Equal,
Self::Gt => ord == Ordering::Greater,
Self::Ge => ord != Ordering::Less,
}
}
pub const fn swapped(self) -> Self {

View File

@@ -129,7 +129,8 @@ pub fn get_git_datetime() -> String {
}
// Must be aligned to Lib/importlib/_bootstrap_external.py
pub const PYC_MAGIC_NUMBER: u16 = 2996;
// Bumped to 2997 for MAKE_CELL/COPY_FREE_VARS prolog and cell-local merging
pub const PYC_MAGIC_NUMBER: u16 = 2997;
// CPython format: magic_number | ('\r' << 16) | ('\n' << 24)
// This protects against text-mode file reads

View File

@@ -118,6 +118,7 @@ declare_const_name! {
__class__,
__class_getitem__,
__classcell__,
__classdictcell__,
__complex__,
__contains__,
__copy__,
@@ -427,12 +428,12 @@ impl Context {
max_stackdepth: 2,
obj_name: names.__init__,
qualname: names.__init__,
cell2arg: None,
constants: core::iter::empty().collect(),
names: Vec::new().into_boxed_slice(),
varnames: Vec::new().into_boxed_slice(),
cellvars: Vec::new().into_boxed_slice(),
freevars: Vec::new().into_boxed_slice(),
localspluskinds: Vec::new().into_boxed_slice(),
linetable: Vec::new().into_boxed_slice(),
exceptiontable: Vec::new().into_boxed_slice(),
};

View File

@@ -10,6 +10,10 @@ use core::sync::atomic::Ordering;
type InitFunc = Box<dyn FnOnce(&mut VirtualMachine)>;
/// Exit code used when stdout/stderr flush fails during interpreter shutdown.
/// Matches CPython's behavior (see cpython/Python/pylifecycle.c).
const EXITCODE_FLUSH_FAILURE: u32 = 120;
/// Configuration builder for constructing an Interpreter.
///
/// This is the preferred way to configure and create an interpreter with custom modules.
@@ -401,7 +405,7 @@ impl Interpreter {
/// Note that calling `finalize` is not necessary by purpose though.
pub fn finalize(self, exc: Option<PyBaseExceptionRef>) -> u32 {
self.enter(|vm| {
vm.flush_std();
let mut flush_status = vm.flush_std();
// See if any exception leaked out:
let exit_code = if let Some(exc) = exc {
@@ -439,9 +443,16 @@ impl Interpreter {
// (while builtins is still available for __del__), then clear module dicts.
vm.finalize_modules();
vm.flush_std();
if vm.flush_std() < 0 && flush_status == 0 {
flush_status = -1;
}
exit_code
// Match CPython: if exit_code is 0 and stdout flush failed, exit 120
if exit_code == 0 && flush_status < 0 {
EXITCODE_FLUSH_FAILURE
} else {
exit_code
}
})
}
}

View File

@@ -22,7 +22,7 @@ use crate::{
self, PyBaseExceptionRef, PyDict, PyDictRef, PyInt, PyList, PyModule, PyStr, PyStrInterned,
PyStrRef, PyTypeRef, PyUtf8Str, PyUtf8StrInterned, PyWeak,
code::PyCode,
dict::{PyDictItems, PyDictValues},
dict::{PyDictItems, PyDictKeys, PyDictValues},
pystr::AsPyStr,
tuple::PyTuple,
},
@@ -1822,7 +1822,9 @@ impl VirtualMachine {
where
F: Fn(PyObjectRef) -> PyResult<T>,
{
// Extract elements from item, if possible:
// Type-specific fast paths corresponding to _list_extend() in CPython
// Objects/listobject.c. Each branch takes an atomic snapshot to avoid
// race conditions from concurrent mutation (no GIL).
let cls = value.class();
let list_borrow;
let slice = if cls.is(self.ctx.types.tuple_type) {
@@ -1830,8 +1832,13 @@ impl VirtualMachine {
} else if cls.is(self.ctx.types.list_type) {
list_borrow = value.downcast_ref::<PyList>().unwrap().borrow_vec();
&list_borrow
} else if cls.is(self.ctx.types.dict_type) {
let keys = value.downcast_ref::<PyDict>().unwrap().keys_vec();
return keys.into_iter().map(func).collect();
} else if cls.is(self.ctx.types.dict_keys_type) {
let keys = value.downcast_ref::<PyDictKeys>().unwrap().dict.keys_vec();
return keys.into_iter().map(func).collect();
} else if cls.is(self.ctx.types.dict_values_type) {
// Atomic snapshot of dict values - prevents race condition during iteration
let values = value
.downcast_ref::<PyDictValues>()
.unwrap()
@@ -1839,7 +1846,6 @@ impl VirtualMachine {
.values_vec();
return values.into_iter().map(func).collect();
} else if cls.is(self.ctx.types.dict_items_type) {
// Atomic snapshot of dict items - prevents race condition during iteration
let items = value
.downcast_ref::<PyDictItems>()
.unwrap()

View File

@@ -52,14 +52,31 @@ impl VirtualMachine {
}
}
pub(crate) fn flush_std(&self) {
/// Returns true if the file object's `closed` attribute is truthy.
fn file_is_closed(&self, file: &PyObject) -> bool {
file.get_attr("closed", self)
.ok()
.is_some_and(|v| v.try_to_bool(self).unwrap_or(false))
}
pub(crate) fn flush_std(&self) -> i32 {
let vm = self;
if let Ok(stdout) = sys::get_stdout(vm) {
let _ = vm.call_method(&stdout, identifier!(vm, flush).as_str(), ());
let mut status = 0;
if let Ok(stdout) = sys::get_stdout(vm)
&& !vm.is_none(&stdout)
&& !vm.file_is_closed(&stdout)
&& let Err(e) = vm.call_method(&stdout, identifier!(vm, flush).as_str(), ())
{
vm.run_unraisable(e, None, stdout);
status = -1;
}
if let Ok(stderr) = sys::get_stderr(vm) {
if let Ok(stderr) = sys::get_stderr(vm)
&& !vm.is_none(&stderr)
&& !vm.file_is_closed(&stderr)
{
let _ = vm.call_method(&stderr, identifier!(vm, flush).as_str(), ());
}
status
}
#[track_caller]

View File

@@ -1,4 +1,4 @@
use rustpython_vm::{Interpreter};
use rustpython_vm::Interpreter;
unsafe extern "C" {
fn kv_get(kp: i32, kl: i32, vp: i32, vl: i32) -> i32;
@@ -37,12 +37,7 @@ pub unsafe extern "C" fn eval(s: *const u8, l: usize) -> i32 {
let msg = format!("eval result: {result}");
unsafe {
print(
msg.as_str().as_ptr() as usize as i32,
msg.len() as i32,
)
};
unsafe { print(msg.as_str().as_ptr() as usize as i32, msg.len() as i32) };
0
}

View File

@@ -38,6 +38,16 @@ from functools import reduce
from unittest.runner import registerResult, result
def _get_method_dict(test):
"""Get the __dict__ of the underlying function for a test method.
Works for both bound methods (__func__.__dict__) and plain functions.
"""
method = getattr(test, test._testMethodName)
func = getattr(method, "__func__", method)
return func.__dict__
class TablePrinter(object):
# Modified from https://github.com/agramian/table-printer, same license as above
"Print a list of dicts as a table"
@@ -325,9 +335,7 @@ class CustomTextTestResult(result.TestResult):
self.stream.writeln("TEST SUITE: %s" % self.suite)
self.stream.writeln("Description: %s" % self.getSuiteDescription(test))
try:
name_override = getattr(test, test._testMethodName).__func__.__dict__[
"test_case_name"
]
name_override = _get_method_dict(test)["test_case_name"]
except:
name_override = None
self.case = name_override if name_override else self.case
@@ -345,7 +353,11 @@ class CustomTextTestResult(result.TestResult):
self.results["suites"][self.suite_number] = {
"name": self.suite,
"class": test.__class__.__name__,
"module": re.compile(".* \((.*)\)").match(str(test)).group(1),
"module": (
m.group(1)
if (m := re.compile(r".* \((.*)\)").match(str(test)))
else str(test)
),
"description": self.getSuiteDescription(test),
"cases": {},
"used_case_names": {},
@@ -380,34 +392,22 @@ class CustomTextTestResult(result.TestResult):
if "test_type" in getattr(
test, test._testMethodName
).__func__.__dict__ and set([s.lower() for s in self.test_types]) == set(
[
s.lower()
for s in getattr(test, test._testMethodName).__func__.__dict__[
"test_type"
]
]
[s.lower() for s in _get_method_dict(test)["test_type"]]
):
pass
else:
getattr(test, test._testMethodName).__func__.__dict__[
"__unittest_skip_why__"
] = 'Test run specified to only run tests of type "%s"' % ",".join(
self.test_types
_get_method_dict(test)["__unittest_skip_why__"] = (
'Test run specified to only run tests of type "%s"'
% ",".join(self.test_types)
)
getattr(test, test._testMethodName).__func__.__dict__[
"__unittest_skip__"
] = True
if "skip_device" in getattr(test, test._testMethodName).__func__.__dict__:
for device in getattr(test, test._testMethodName).__func__.__dict__[
"skip_device"
]:
_get_method_dict(test)["__unittest_skip__"] = True
if "skip_device" in _get_method_dict(test):
for device in _get_method_dict(test)["skip_device"]:
if self.config and device.lower() in self.config["device_name"].lower():
getattr(test, test._testMethodName).__func__.__dict__[
"__unittest_skip_why__"
] = "Test is marked to be skipped on %s" % device
getattr(test, test._testMethodName).__func__.__dict__[
"__unittest_skip__"
] = True
_get_method_dict(test)["__unittest_skip_why__"] = (
"Test is marked to be skipped on %s" % device
)
_get_method_dict(test)["__unittest_skip__"] = True
break
def stopTest(self, test):

View File

@@ -19,6 +19,33 @@ assert type(str(y)) is str, "Str of a str-subtype should be a str."
assert y + " other" == "1 other"
assert y.x == "substr"
class ReprStrSubclass(str):
pass
class WithStr:
def __init__(self, value):
self.value = value
def __str__(self):
return self.value
class WithRepr:
def __init__(self, value):
self.value = value
def __repr__(self):
return self.value
str_value = ReprStrSubclass("abc")
assert str(WithStr(str_value)) is str_value
repr_value = ReprStrSubclass("<abc>")
assert str(WithRepr(repr_value)) is repr_value
## Base strings currently get an attribute dict, but shouldn't.
# with assert_raises(AttributeError):
# "hello".x = 5

View File

@@ -45,3 +45,37 @@ assert 'c' not in globals()
def f():
# Test no panic occurred.
[[x := 1 for j in range(5)] for i in range(5)]
# Nested inlined comprehensions with lambda in the first iterator expression.
# The lambda's sub_table must be consumed before the inner comprehension's
# sub_table is spliced in, otherwise scope ordering is wrong.
def test_nested_comp_with_lambda():
import itertools
offsets = {0: [0], 1: [1], 3: [2]}
grouped = [
[x for _, x in group]
for _, group in itertools.groupby(
enumerate(sorted(offsets.keys())), lambda x: x[1] - x[0]
)
]
assert grouped == [[0, 1], [3]], f"got {grouped}"
test_nested_comp_with_lambda()
# Nested inlined comprehensions with throwaway `_` in both levels.
def test_nested_comp_underscore():
data = [(1, "a", "x"), (2, "b", "y")]
result = [[v for _, v in zip(range(2), row)] for _, *row in data]
assert result == [["a", "x"], ["b", "y"]], f"got {result}"
test_nested_comp_underscore()
# Simple nested inlined comprehensions.
def test_simple_nested_comp():
result = [[j * i for j in range(3)] for i in range(3)]
assert result == [[0, 0, 0], [0, 1, 2], [0, 2, 4]]
test_simple_nested_comp()

View File

@@ -1,15 +1,46 @@
#!/usr/bin/env python
import argparse
import ast
import glob
import os
import pathlib
import sys
ROOT = pathlib.Path(__file__).parents[1]
TEST_DIR = ROOT / "Lib" / "test"
IS_GH_CI = "GITHUB_ACTIONS" in os.environ
def main():
def build_argparser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="check_redundant_patches")
parser.add_argument(
"patterns",
nargs="*",
default=[f"{TEST_DIR}/**/*.py"],
help="Glob patterns (e.g. foo/bar/**.py a/b/file.py)",
)
return parser
def iter_files(patterns: list[str]):
seen = set()
for pattern in set(patterns):
matches = glob.glob(pattern, recursive=True)
for path in matches:
if path in seen:
continue
seen.add(path)
yield path
def main(patterns: list[str]):
exit_status = 0
for file in TEST_DIR.rglob("**/*.py"):
for file in map(pathlib.Path, iter_files(patterns)):
if file.is_dir():
continue
try:
contents = file.read_text(encoding="utf-8")
except UnicodeDecodeError:
@@ -20,7 +51,11 @@ def main():
except SyntaxError:
continue
cls_name = None
for node in ast.walk(tree):
if isinstance(node, ast.ClassDef):
cls_name = node.name
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
@@ -39,14 +74,19 @@ def main():
f"super().{name}()",
):
exit_status += 1
rel = file.relative_to(ROOT)
lineno = node.lineno
print(
f"{rel}:{name}:{lineno} is a test patch that can be safely removed",
file=sys.stderr,
)
msg = f"{file}:{lineno}:{cls_name}.{name} is a test patch that can be safely removed"
if IS_GH_CI:
end_lineno = node.end_lineno
msg = f"::error file={file},line={lineno},endLine={end_lineno},title=Redundant Test Patch::{msg}"
print(msg, file=sys.stderr)
return exit_status
if __name__ == "__main__":
exit(main())
parser = build_argparser()
args = parser.parse_args()
exit(main(args.patterns))

View File

@@ -56,6 +56,9 @@ mod interpreter;
mod settings;
mod shell;
#[cfg(feature = "rustpython-pylib")]
pub use rustpython_pylib as pylib;
use rustpython_vm::{AsObject, PyObjectRef, PyResult, VirtualMachine, scope::Scope};
use std::env;
use std::io::IsTerminal;

View File

@@ -3539,9 +3539,9 @@
}
},
"node_modules/node-forge": {
"version": "1.3.2",
"resolved": "https://registry.npmjs.org/node-forge/-/node-forge-1.3.2.tgz",
"integrity": "sha512-6xKiQ+cph9KImrRh0VsjH2d8/GXA4FIMlgU4B757iI1ApvcyA9VlouP0yZJha01V+huImO+kKMU7ih+2+E14fw==",
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/node-forge/-/node-forge-1.4.0.tgz",
"integrity": "sha512-LarFH0+6VfriEhqMMcLX2F7SwSXeWwnEAJEsYm5QKWchiVYVvJyV9v7UDvUv+w5HO23ZpQTXDv/GxdDdMyOuoQ==",
"dev": true,
"license": "(BSD-3-Clause OR GPL-2.0)",
"engines": {
@@ -3823,9 +3823,9 @@
"license": "ISC"
},
"node_modules/picomatch": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz",
"integrity": "sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==",
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz",
"integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==",
"dev": true,
"license": "MIT",
"engines": {
@@ -4029,16 +4029,6 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/randombytes": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/randombytes/-/randombytes-2.1.0.tgz",
"integrity": "sha512-vYl3iOX+4CKUWuxGi9Ukhie6fsqXqS9FE2Zaic4tNFD2N2QQaXOMFbuKK4QmDHC0JO6B1Zp41J0LpT0oR68amQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"safe-buffer": "^5.1.0"
}
},
"node_modules/range-parser": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.0.tgz",
@@ -4384,16 +4374,6 @@
"node": ">= 0.6"
}
},
"node_modules/serialize-javascript": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/serialize-javascript/-/serialize-javascript-6.0.2.tgz",
"integrity": "sha512-Saa1xPByTTq2gdeFZYLLo+RFE35NHZkAbqZeWNd3BpzppeVisAqpDjcp8dyf6uIvEqJRd46jemmyA4iFIeVk8g==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"randombytes": "^2.1.0"
}
},
"node_modules/serve": {
"version": "14.2.5",
"resolved": "https://registry.npmjs.org/serve/-/serve-14.2.5.tgz",
@@ -5024,16 +5004,15 @@
}
},
"node_modules/terser-webpack-plugin": {
"version": "5.3.16",
"resolved": "https://registry.npmjs.org/terser-webpack-plugin/-/terser-webpack-plugin-5.3.16.tgz",
"integrity": "sha512-h9oBFCWrq78NyWWVcSwZarJkZ01c2AyGrzs1crmHZO3QUg9D61Wu4NPjBy69n7JqylFF5y+CsUZYmYEIZ3mR+Q==",
"version": "5.4.0",
"resolved": "https://registry.npmjs.org/terser-webpack-plugin/-/terser-webpack-plugin-5.4.0.tgz",
"integrity": "sha512-Bn5vxm48flOIfkdl5CaD2+1CiUVbonWQ3KQPyP7/EuIl9Gbzq/gQFOzaMFUEgVjB1396tcK0SG8XcNJ/2kDH8g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@jridgewell/trace-mapping": "^0.3.25",
"jest-worker": "^27.4.5",
"schema-utils": "^4.3.0",
"serialize-javascript": "^6.0.2",
"terser": "^5.31.1"
},
"engines": {