Commit Graph

74 Commits

Author SHA1 Message Date
harupy
70ce090476 Fix make_tokenizer_located 2022-12-27 21:54:18 +09:00
Jeong YunWon
247e815880 Merge pull request #4359 from yt2b/check_bom
Add BOM check
2022-12-26 16:03:51 +09:00
Jeong YunWon
06a815f8c7 Merge pull request #4358 from harupy/fix-slice-location
Fix `Slice` location
2022-12-26 16:03:09 +09:00
Jeong YunWon
647d542d38 Merge pull request #4356 from andersk/with-tuple-named
Fix parsing of tuple with named expression as context manager
2022-12-26 16:02:17 +09:00
yt2b
452e3a3b36 Fix comment 2022-12-26 09:30:12 +09:00
yt2b
7b830dc004 Add BOM check 2022-12-25 11:15:29 +09:00
harupy
d7f7fd2e21 Fix slice location 2022-12-25 09:37:07 +09:00
Anders Kaseorg
c7d03d9cb4 Fix parsing of tuple with named expression as context manager
Because the ‘with’ item grammar disallows named expressions, CPython
parses ‘with (a := 0, b := 1):’ as a tuple rather than two ‘with’
items.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-24 13:15:53 -08:00
harupy
1589072680 Fix IfExp location 2022-12-25 00:33:02 +09:00
Anders Kaseorg
5c543445c3 Simplify parenthesized context manager parsing with LALRPOP conditions
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-24 23:13:10 +09:00
harupy
083191db47 Fix the location of BinOp 2022-12-21 22:05:05 +09:00
Jim Fasarakis-Hilliard
c01f014b12 Merge pull request #4340 from harupy/fix-locations-of-parethesized-expressions
Fix the start and end locations of `Tuple`
2022-12-18 15:17:40 +02:00
harupy
fdbfe156ca Fix locations of parethesized expressions 2022-12-18 20:53:30 +09:00
Anders Kaseorg
316c4858e2 Remove unnecessary boxing of ASDL product children
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-14 02:30:02 -08:00
Dimitris Fasarakis Hilliard
609ac2754c Fix end location in with statements. 2022-12-13 13:28:17 +02:00
Jim Fasarakis-Hilliard
8d879a5319 Merge pull request #4327 from harupy/fix-end-location-body
Fix end location of compound statements
2022-12-13 12:30:21 +02:00
Charlie Marsh
75a9b4a352 Set ExprContext::Store on parenthesized with expressions 2022-12-12 09:09:15 -05:00
harupy
7a40d9c876 Refactor 2022-12-12 22:36:34 +09:00
harupy
337b0ead4a Update snapshot 2022-12-12 22:26:03 +09:00
harupy
c906e431be Format 2022-12-12 22:18:26 +09:00
harupy
fbb409b755 Resolve conflict 2022-12-12 22:16:46 +09:00
harupy
05e1a7a0aa Address comments
Signed-off-by: harupy <hkawamura0130@gmail.com>
2022-12-12 22:14:05 +09:00
Anders Kaseorg
b7f4ff7283 Parse Python 3.9+ parenthesized context managers
Since the upstream grammar for this is not LR(1), we abuse LALRPOP
macros and the Into/TryInto traits to build a cover grammar that
converts to either tuples or `with` items after additional validation.
It’s annoying and ugly, but something like this is basically our only
option short of switching to a more powerful parser algorithm.

Fixes #4145.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-12 00:47:33 -08:00
Anders Kaseorg
78ca48cd79 Split and simplify some LALRPOP rules
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-11 22:02:08 -08:00
harupy
7e315c16c6 Use method chaining 2022-12-12 10:24:00 +09:00
harupy
91f1fc73ab Fix other compound statements 2022-12-12 01:10:42 +09:00
harupy
9f9ba0286e Fix end location of nodes containing body 2022-12-11 12:35:28 +09:00
harupy
6a0f13eb4f Refactor 2022-12-10 22:01:42 +09:00
harupy
d6d50d4635 Fix 2022-12-10 18:45:36 +09:00
harupy
991c8299f8 Fix plain string 2022-12-10 18:09:26 +09:00
harupy
93f3433e77 Fix the end location of an implicitly-concatenated string 2022-12-10 17:49:57 +09:00
Jeong YunWon
5fed6a35af Merge pull request #4310 from youknowone/fix-clippy
Fix nightly clippy warnings
2022-12-05 12:57:01 +09:00
Jeong YunWon
d935fbc445 Fix nightly clippy warnings 2022-12-05 12:18:16 +09:00
harupy
5c63109f4a add tests 2022-12-04 05:59:51 +09:00
harupy
adec9e94fa Fix location 2022-12-04 05:45:15 +09:00
yt2b
a79fb0195f use bool.then 2022-11-24 09:23:20 +09:00
yt2b
81b7c41f3e restore if expression 2022-11-23 11:30:54 +09:00
yt2b
683a3e5740 Refactor lexer functions 2022-11-22 21:10:19 +09:00
yt2b
592e9b49b7 Add test 2022-11-21 22:36:05 +09:00
Charlie Marsh
3a6729eedd Use match 2022-11-20 23:41:52 -05:00
Charlie Marsh
ebeb0904d4 Use rustc-hash 2022-11-20 15:30:19 -05:00
Charlie Marsh
3de87f9828 Implement some minor performance optimizations 2022-11-20 13:33:20 -05:00
Bongjun Jang
e72f01773d Refactor lexer struct (#4257) 2022-11-19 21:43:34 +09:00
Jeong YunWon
499b34cee0 Apply let-else statements 2022-11-10 15:39:31 +09:00
Jim Fasarakis-Hilliard
27bf82a225 Merge pull request #4266 from charliermarsh/charlie/comments
Implement Tok::Comment
2022-11-07 23:20:15 +02:00
Charlie Marsh
ba4beb56c5 Set comparator start location to beginning of comparison 2022-11-07 12:24:14 -05:00
Charlie Marsh
978829f263 Implement Tok::Comment 2022-11-07 10:33:55 -05:00
dvermd
4f14fc746c improve col_offset in new line and lalr 2022-10-26 21:30:51 +02:00
dvermd
dc595d1276 improve fstring parser
part of: #1671
2022-10-26 21:15:24 +02:00
dvermd
f2bfb78d3c Fix ast types' _fields and use 0-based column 2022-10-27 03:09:38 +09:00