Commit Graph

6 Commits

Author SHA1 Message Date
Jeong, YunWon
55737ede63 Rewrite _tokenize with 2-phase model
Replace per-line reparsing with single-pass tokenization:
- Read all lines via readline, parse once, yield tokens
- Fix token type values (COMMENT=65, NL=66, OP=55)
- Fix NEWLINE/NL end positions and implicit newline handling
- Fix DEDENT positions via look-ahead to next non-DEDENT token
- Handle FSTRING_MIDDLE brace unescaping ({{ → {, }} → })
- Emit implicit NL before ENDMARKER when source lacks trailing newline
- Raise IndentationError from lexer errors
- Remove 13 expectedFailure marks for now-passing tests
2026-03-10 11:28:16 +09:00
Shahar Naveh
c904a91bef Update test_tabnanny.py from 3.14.3 (#7083) 2026-02-11 17:41:57 +09:00
CPython Developers
db347b344d Update tabnanny from v3.14.2-288-g06f9c8ca1c 2026-02-02 11:51:53 +09:00
Shahar Naveh
582e25b11b Update tabnanny.py from 3.13.5 (#6021) 2025-07-23 12:27:56 +09:00
Jeong YunWon
cc202e1e90 Mark failing tests for test_tabnanny 2022-08-06 05:27:03 +09:00
Jeong Yunwon
216fcd8ba5 Add tabnanny module and its test from Python v3.10.4 2022-08-06 05:21:55 +09:00