* Bytecode parity phase 3
Compiler changes:
- Emit TO_BOOL in and/or short-circuit evaluation (COPY+TO_BOOL+JUMP)
- Add module-level __conditional_annotations__ cell (PEP 649)
- Only set conditional annotations for AnnAssign, not function params
- Skip __classdict__ cell when future annotations are active
- Convert list literals to tuples in for-loop iterables
- Fix cell variable ordering: parameters first, then alphabetical
- Fix RESUME DEPTH1 flag for yield-from/await
- Don't propagate __classdict__/__conditional_annotations__ freevar
through regular functions — only annotation/type-param scopes
- Inline string compilation path
* Skip test_thread_safety in _test_multiprocessing
SIGSEGV in _finalizer_registry dict access under aggressive GC
and thread switching. Root cause is dict thread-safety in VM.
* Skip list→tuple optimization for async for; propagate future_annotations to nested scopes
* fix: flush stdout on interpreter shutdown matching CPython behavior
When stdout flush fails during shutdown, report the error via
run_unraisable and exit with code 120 (matching CPython's
Py_FinalizeEx). Skip flushing already-closed or None streams.
Stderr flush errors remain silently ignored per CPython behavior.
Fixes#5521
Signed-off-by: majiayu000 <1835304752@qq.com>
* refactor: replace magic number 120 with named constant EXITCODE_FLUSH_FAILURE
Address review feedback on PR #7503: improve readability by extracting
the CPython-compat exit code into a named constant.
Signed-off-by: majiayu000 <1835304752@qq.com>
---------
Signed-off-by: majiayu000 <1835304752@qq.com>
* Bytecode parity
Compiler changes:
- Remove PUSH_NULL from decorator cal
ls, use CALL 0
- Collect __static_attributes__ from self.xxx = patterns
- Sort __static_attributes__ alphabetically
- Move __classdict__ init before __doc__ in class prologue
- Fold unary negative constants
- Fold constant list/set literals (3+ elements)
- Use BUILD_MAP 0 + MAP_ADD for 16+ dict pairs
- Always run peephole optimizer for s
uperinstructions
- Emit RETURN_GENERATOR for generator
functions
- Add is_generator flag to SymbolTabl
e
* Fix formatting and collapsible_if clippy warnings in compile.rs
* Fix clippy, fold_unary_negative chaining, and generator line tracing
- Replace irrefutable if-let with let for ExceptHandler
- Remove folded UNARY_NEGATIVE instead of replacing with NOP,
enabling chained negation folding
- Initialize prev_line to def line for generators/coroutines
to suppress spurious LINE events from preamble instructions
- Remove expectedFailure markers for now-passing tests
* Fix JIT StoreFastStoreFast, format, and remove expectedFailure markers
- Add StoreFastStoreFast handling in JIT instructions
- Fix cargo fmt in frame.rs
- Remove 11 expectedFailure markers for async jump tests in
test_sys_settrace that now pass
* Fix peephole optimizer: use NOP replacement instead of remove()
Using remove() shifts instruction indices and corrupts subsequent
references, causing "pop stackref but null found" panics at runtime.
Replace folded/combined instructions with NOP instead, which are
cleaned up by the existing remove_nops pass.
* Revert peephole_optimize to use remove() for chaining support
NOP replacement broke chaining of peephole optimizations (e.g.
LOAD_CONST+TO_BOOL then LOAD_CONST+UNARY_NOT for 'not True').
The remove() approach is used by upstream and works correctly here;
fold_unary_negative keeps NOP replacement since it doesn't need chaining.
* Fix StoreFastStoreFast to handle NULL from LoadFastAndClear
StoreFast uses pop_value_opt() to allow NULL values from
LoadFastAndClear in inlined comprehension cleanup paths.
StoreFastStoreFast must do the same, otherwise the peephole
optimizer's fusion of two StoreFast instructions panics when
restoring unbound locals after an inlined comprehension.
* Handle EINTR retry in os.write() (PEP 475)
Add EINTR retry loop to os.write(), matching the existing
pattern in os.read() and os.readinto(). Remove the
expectedFailure marker from test_write in _test_eintr.py.
* Add atomic snapshot for dict/dict_keys in extract_elements
Add fast paths for dict and dict_keys types in
extract_elements_with, matching _list_extend() in CPython
Objects/listobject.c. Each branch takes an atomic snapshot
under a single read lock, preventing race conditions from
concurrent dict mutation without the GIL.
Remove expectedFailure from test_thread_safety.
* Emit TO_BOOL before conditional jumps, fix class/module prologue
- Emit TO_BOOL before POP_JUMP_IF_TRUE/FALSE in the general case
of compile_jump_if (Compare expressions excluded since they
already produce a bool)
- Module-level __doc__: use STORE_NAME instead of STORE_GLOBAL
- Class body __module__: use LOAD_NAME instead of LOAD_GLOBAL
- Class body: store __firstlineno__ before __doc__
* Emit MAKE_CELL and COPY_FREE_VARS before RESUME
Emit MAKE_CELL for each cell variable and COPY_FREE_VARS N for
free variables at the start of each code object, before RESUME.
These instructions are no-ops in the VM but align the bytecode
with CPython 3.14's output.
* Emit __static_attributes__ at end of class bodies
Store a tuple of attribute names (currently always empty) as
__static_attributes__ in the class namespace, matching CPython
3.14's class body epilogue. Attribute name collection from
self.xxx accesses is a follow-up task.
* Remove expectedFailure from DictProxyTests iter tests
test_iter_keys, test_iter_values, test_iter_items now pass
because class bodies emit __static_attributes__ and
__firstlineno__, matching the expected dict key set.
* Use 1-based stack indexing for LIST_EXTEND, SET_UPDATE, etc.
Switch LIST_APPEND, LIST_EXTEND, SET_ADD, SET_UPDATE, MAP_ADD
from 0-based to 1-based stack depth argument, matching CPython's
PEEK(oparg) convention. Adjust the VM to subtract 1 before
calling nth_value.
* Use plain LOAD_ATTR + PUSH_NULL for calls on imported names
When the call target is an attribute of an imported name (e.g.,
logging.getLogger()), use plain LOAD_ATTR (method_flag=0) with
a separate PUSH_NULL instead of method-mode LOAD_ATTR. This
matches CPython 3.14's behavior which avoids the method call
optimization for module attribute access.
* Duplicate return-None epilogue for fall-through blocks
When the last block in a code object is exactly LOAD_CONST None +
RETURN_VALUE (the implicit return), duplicate these instructions
into blocks that would otherwise fall through to it. This matches
CPython 3.14's behavior of giving each code path its own explicit
return instruction.
* Run cargo fmt on ir.rs
* Remove expectedFailure from test_intrinsic_1 in test_dis
* Emit TO_BOOL before conditional jumps for all expressions including Compare
* Add __classdict__ cell for classes with function definitions
Set needs_classdict=true for class scopes that contain function
definitions (def/async def), matching CPython 3.14's behavior for
PEP 649 deferred annotation support. Also restore the Compare
expression check in compile_jump_if to skip TO_BOOL for comparison
operations.
* Emit __classdictcell__ store in class body epilogue
Store the __classdict__ cell reference as __classdictcell__ in
the class namespace when the class has __classdict__ as a cell
variable. Uses LOAD_DEREF (RustPython separates cell vars from
fast locals unlike CPython's unified array).
* Always run DCE to remove dead code after terminal instructions
Run basic dead code elimination (truncating instructions after
RETURN_VALUE/RAISE/JUMP within blocks) at all optimization
levels, not just optimize > 0. CPython always removes this dead
code during assembly.
* Restrict LOAD_ATTR plain mode to module/class scope imports
Only use plain LOAD_ATTR + PUSH_NULL for imports at module or
class scope. Function-local imports use method call mode LOAD_ATTR,
matching CPython 3.14's behavior.
* Eliminate unreachable blocks after jump normalization
Split DCE into two phases: (1) within-block truncation after
terminal instructions (always runs), (2) whole-block elimination
for blocks only reachable via fall-through from terminal blocks
(runs after normalize_jumps when dead jump instructions exist).
* Fold BUILD_TUPLE 0 into LOAD_CONST empty tuple
Convert BUILD_TUPLE with size 0 to LOAD_CONST () during constant
folding, matching CPython's optimization for empty tuple literals.
* Handle __classcell__ and __classdictcell__ in type.__new__
- Remove __classcell__ from class dict after setting the cell value
- Add __classdictcell__ handling: set cell to class namespace dict,
then remove from class dict
- Register __classdictcell__ identifier
- Use LoadClosure instead of LoadDeref for __classdictcell__ emission
- Reorder MakeFunctionFlag bits to match CPython
- Run ruff format on scripts
* Revert __classdict__ cell and __classdictcell__ changes
The __classdict__ cell addition (for classes with function defs)
and __classdictcell__ store caused cell initialization failures
in importlib. These require deeper VM changes to properly support
the cell variable lifecycle. Reverted for stability.
* Fix unreachable block elimination with fixpoint reachability
Use fixpoint iteration to properly determine block reachability:
only mark jump targets of already-reachable blocks, preventing
orphaned blocks from falsely marking their targets as reachable.
Also add a final DCE pass after assembly NOP removal to catch
dead code created by normalize_jumps.
* Check enclosing scopes for IMPORTED flag in LOAD_ATTR mode
When deciding whether to use plain LOAD_ATTR for attribute calls,
check if the name is imported in any enclosing scope (not just
the current scope). This handles the common pattern where a module
is imported at module level but used inside functions.
* Add __classdict__ cell for classes with function definitions
Set needs_classdict=true when a class scope contains function
definitions (def/async def), matching CPython 3.14 which always
creates a __classdict__ cell for PEP 649 support in such classes.
* Store __classdictcell__ in class body epilogue
Store the __classdict__ cell reference as __classdictcell__ in
the class namespace using LoadClosure (which loads the cell
object itself, not the value inside). This matches CPython 3.14's
class body epilogue.
* Fix clippy collapsible_if warnings and cargo fmt
* Revert __classdict__ and __classdictcell__ changes (cause import failures)
* Revert type.__new__ __classcell__ removal and __classdictcell__ handling
Revert the class cell cleanup changes from e6975f973 that cause
import failures when frozen module bytecode is stale. The original
behavior (not removing __classcell__ from class dict) is restored.
* Re-add __classdict__ cell and __classdictcell__ store
Restore the __classdict__ cell for classes with function
definitions and __classdictcell__ store in class body epilogue.
Previous failure was caused by stale .pyc cache files containing
bytecode from an intermediate MakeFunctionFlag reorder attempt,
not by these changes themselves.
* Reorder MakeFunctionFlag to match CPython's SET_FUNCTION_ATTRIBUTE
Reorder discriminants: Defaults=0, KwOnlyDefaults=1, Annotations=2,
Closure=3, Annotate=4, TypeParams=5. This aligns the oparg values
with CPython 3.14's convention.
Note: after this change, stale .pyc cache files must be deleted
(find . -name '*.pyc' -delete) to avoid bytecode mismatch errors.
* Use CPython-compatible power-of-two encoding for SET_FUNCTION_ATTRIBUTE
Override From/TryFrom for MakeFunctionFlag to use power-of-two
values (1,2,4,8,16,32) matching CPython's SET_FUNCTION_ATTRIBUTE
oparg encoding, instead of sequential discriminants (0,1,2,3,4,5).
* Remove expectedFailure from test_elim_jump_after_return1 and test_no_jump_over_return_out_of_finally_block
* Remove __classcell__ and __classdictcell__ from class dict in type.__new__
* Remove expectedFailure from test___classcell___expected_behaviour, cargo fmt
* Handle MakeCell and CopyFreeVars as no-ops in JIT
These prologue instructions are handled at frame creation time
by the VM. The JIT operates on already-initialized frames, so
these can be safely skipped during compilation.
* Remove expectedFailure from test_load_fast_known_simple
* Restore expectedFailure for test_load_fast_known_simple
The test expects LOAD_FAST_BORROW_LOAD_FAST_BORROW superinstruction
which RustPython does not emit yet.
* Disallow instantiation of sys.getwindowsversion type
Add slot_new to PyWindowsVersion that raises TypeError,
matching sys.flags behavior.
* Remove incorrect WSAHOS errno constant
WSAHOS was hardcoded as an alias for WSAHOST_NOT_FOUND, but
CPython guards it with #ifdef WSAHOS which doesn't exist in
modern Windows SDK headers.
* Fix mmap resize to raise OSError instead of SystemError
* Fix CreateProcess with empty environment on Windows
Empty env dict produced a single null terminator, but
CreateProcessW requires a double null for a valid empty
environment block.
* Revert mmap resize error to SystemError and fix errno.rs formatting
mmap resize raises SystemError (not OSError) when mremap is unavailable,
matching CPython behavior. test_mmap catches SystemError to skip unsupported
resize operations.
* Fix named mmap resize to raise OSError and unmark test_sleep expectedFailure
Named mmap resize on Windows should raise OSError (not SystemError).
Remove expectedFailure mark from TimeEINTRTest.test_sleep as it now passes.
* Use expectedFailureIf for TimeEINTRTest.test_sleep on Linux
test_sleep passes on macOS but fails on Linux due to timing.
* Remove expectedFailure for TimeEINTRTest.test_sleep
test_sleep now passes on all platforms.
* Add GetDescriptor for PyBoundMethod (return self)
CPython's method_descr_get always returns the bound method unchanged.
This preserves the original binding when __get__ is called on an
already-bound method (e.g. a.meth.__get__(b, B) still returns a).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add constructor validation for PyBoundMethod
Reject non-callable functions and None instances, matching CPython's
method_new which checks PyCallable_Check(func) and instance != Py_None.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix PyBoundMethod __reduce__ to propagate errors
Previously swallowed errors from get_attr with .ok(), silently
returning None. Now propagates errors matching CPython's method_reduce.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Detect list mutation during sort even when list length is unchanged
* Use mutation counter instead of capacity check for sort mutation detection
The capacity heuristic missed mutations when `clear()` reset capacity to
0 via `mem::take`. An AtomicU32 counter on PyList, incremented in
`borrow_vec_mut()`, reliably detects all mutations during sort.
* Hold write guard during sort mutation counter reads
* Fix list mutation counter race in `borrow_vec_mut`
* Fix allow_threads and EINTR handling
- Wrap Windows SemLock acquire wait with allow_threads
- Retry nanosleep on EINTR with remaining time instead of
returning early
- Remove expectedFailure for test_sleep in _test_eintr.py
* Remove expectedFailureIfWindows for testHashComparisonOfMethods
- Add CO_NESTED flag (0x10) for nested function scopes
- Emit LOAD_SMALL_INT for integers 0..=255 instead of LOAD_CONST
- Eliminate dead constant expression statements (no side effects)
- Ensure None in co_consts for functions with no other constants
- Add code.__replace__() for copy.replace() support
- Mark test_co_lnotab and test_invalid_bytecode as expectedFailure
When -O flag removes assert statements, any nested scopes
(generators, comprehensions, lambdas) inside the assert
expression still have symbol tables in the sub_tables list.
Without consuming them, the next_sub_table index gets
misaligned, causing later scopes to use wrong symbol tables.
Walk the skipped assert expression with an AST visitor to
find and consume nested scope symbol tables, keeping the
index aligned with AST traversal order.
* Preserve imaginary zero signs when adding real values to complex numbers
* Refactor complex_add with match expression
* Correct complex real subtract op
* Remove unnecessary vm arugment
Simplify classmethod.__get__ to always create a PyBoundMethod binding
the callable to the class, matching CPython's cm_descr_get which simply
calls PyMethod_New(cm->cm_callable, type).
The previous implementation incorrectly tried to call __get__ on the
wrapped callable, which broke when a bound method was passed to
classmethod() (e.g. classmethod(A().foo)).
Use single-record reading (recv_one_tls_record) for all SSL
reads, not just handshake. This prevents rustls from eagerly
consuming close_notify alongside application data, which left
the TCP buffer empty and caused select()-based servers to miss
readability and time out.
Also fix recv_one_tls_record to return Eof (not WantRead) when
peek returns empty bytes, since empty peek means the peer has
closed the TCP connection.
* gc: add CollectResult, stats fields, get_referrers, and fix count reset
- Add CollectResult struct with collected/uncollectable/candidates/duration
- Add candidates and duration fields to GcStats and gc.get_stats()
- Pass CollectResult to gc.callbacks info dict
- Reset generation counts for all collected generations (0..=N)
- Return 0 for third value in gc.get_threshold() (3.13+)
- Implement gc.get_referrers() by scanning all tracked objects
- Add DEBUG_COLLECTABLE output for collectable objects
- Update test_gc.py to expect candidates/duration in stats
* Update test_gc from v3.14.3
* Update test_gc.py from CPython v3.15.0a5
Taken from v3.15 (not v3.14.3) because get_stats() candidates/duration
fields were added in 3.13+ and the corresponding test assertions only
exist in 3.15.
* Fix gc_state build on wasm32: skip Instant timing
* Add candidates/duration to gc callback info, mark v3.15 test failures
* Fix gc.get_referrers to exclude executing frames, fix Future cancelled exc leak
- get_referrers: skip frame objects on the execution stack, since
they are not GC-tracked in CPython (_PyInterpreterFrame)
- _asyncio Future/Task make_cancelled_error_impl: clear the stored
cancelled exception after returning it, matching the Python
_make_cancelled_error behavior
* Fix gc.get_threshold to return actual gen2 threshold value
* Fix inconsistent GC count reset in early-return paths
Use the same reset_end formula in unreachable-empty early returns
as in the main collection path and collecting-empty path.
* Accept keyword arguments in socket.__init__
Use a FromArgs struct instead of a positional-only tuple so that
family, type, proto, and fileno can be passed as keyword arguments.
* Disable comp_inlined in symbol table to match compiler
The compiler does not yet implement PEP 709 inlined comprehensions
(is_inlined_comprehension_context always returns false), but the
symbol table was marking comprehensions as inlined. This mismatch
could cause comprehension-local symbols to be merged into the parent
scope while the compiler still looks them up in a separate scope.
---------
Co-authored-by: CPython Developers <>
* Fix atexit unraisable exception message format
Match PyErr_FormatUnraisable behavior: use
"Exception ignored in atexit callback {func!r}" as err_msg
and pass None as object instead of the callback function.
* Fix atexit unregister deadlock with reentrant __eq__
Release the lock during equality comparison in unregister so
that __eq__ can safely call atexit.unregister or atexit._clear.
Store callbacks in LIFO order (insert at front) and use
identity-based search after comparison to handle list mutations,
matching atexitmodule.c behavior.
Also pass None as err_msg when func.repr() fails, matching
CPython's PyErr_FormatUnraisable fallback.
* Use patched parking_lot_core with fork-safe HASHTABLE reset
parking_lot_core's global HASHTABLE retains stale ThreadData after
fork(), causing segfaults when contended locks enter park(). Use the
patched version from youknowone/parking_lot (rustpython branch) which
registers a pthread_atfork handler to reset the hash table.
Unskip test_asyncio TestFork. Add Manager+fork integration test.
* Unskip fork-related flaky tests after parking_lot fix
With parking_lot_core's HASHTABLE now properly reset via
pthread_atfork, fork-related segfaults and connection errors
in multiprocessing tests should be resolved.
Remove skip/expectedFailure markers from:
- test_concurrent_futures/test_wait.py (6 tests)
- test_concurrent_futures/test_process_pool.py (1 test)
- test_multiprocessing_fork/test_manager.py (all WithManagerTest*)
- test_multiprocessing_fork/test_misc.py (5 tests)
- test_multiprocessing_fork/test_threads.py (2 tests)
- _test_multiprocessing.py (2 shared_memory tests)
Keep test_repr_rlock skipped (flaky thread start latency,
not fork-related).
* apply more allow_threads
* Simplify STW thread state transitions
- Fix park_detached_threads: successful CAS no longer sets
all_suspended=false, avoiding unnecessary polling rounds
- Replace park_timeout(50µs) with park() in wait_while_suspended
- Remove redundant self-suspension in attach_thread and detach_thread;
the STW controller handles DETACHED→SUSPENDED via park_detached_threads
- Add double-check under mutex before condvar wait to prevent lost wakes
- Remove dead stats_detach_wait_yields field and add_detach_wait_yields
* Representable for ThreadHandle
* Set ThreadHandle state to Running in parent thread after spawn
Like CPython's ThreadHandle_start, set RUNNING state in the parent
thread immediately after spawn() succeeds, rather than in the child.
This eliminates a race where join() could see Starting state if called
before the child thread executes.
Also reverts the macOS skip for test_start_new_thread_failed since the
root cause is fixed.
* Set ThreadHandle state to Running in parent thread after spawn
* Add debug_assert for thread state in start_the_world
* Unskip now-passing test_get_event_loop_thread and test_start_new_thread_at_finalization
* Wrap IO locks and file ops in allow_threads
Add lock_wrapped to ThreadMutex for detaching thread state
while waiting on contended locks. Use it for buffered and
text IO locks. Wrap FileIO read/write in allow_threads via
crt_fd to prevent STW hangs on blocking file operations.
* Use std::sync for thread start/ready events
Replace parking_lot Mutex/Condvar with std::sync (pthread-based)
for started_event and handle_ready_event. This prevents hangs
in forked children where parking_lot's global HASHTABLE may be
corrupted.
* Suspend Python threads before fork()
Add stop-the-world thread suspension around fork() to prevent
deadlocks from locks held by dead parent threads in the child.
- Thread states: DETACHED / ATTACHED / SUSPENDED with atomic CAS
transitions matching _PyThreadState_{Attach,Detach,Suspend}
- stop_the_world / start_the_world: park all non-requester threads
before fork, resume after (parent) or reset (child)
- allow_threads (Py_BEGIN/END_ALLOW_THREADS): detach around blocking
syscalls (os.read/write, waitpid, Lock.acquire, time.sleep) so
stop_the_world can force-park via CAS
- Acquire/release import lock around fork lifecycle
- zero_reinit_after_fork: generic lock reset for parking_lot types
- gc_clear_raw: detach dict instead of clearing entries
- Lock-free double-check for descriptor cache reads (no read-side
seqlock); write-side seqlock retained for writer serialization
- fork() returns PyResult, checks PythonFinalizationError, calls
sys.audit
