Lines Matching refs:token
14 from . import token
37 <for each input token>:
38 if p.addtoken(...): # parse a token; may raise ParseError
44 A Parser instance contains state pertaining to the current token
46 to parse separate token sequences.
79 tuple, where type is the node type (a token or symbol number),
117 """Add a token; return True iff this is the end of the program."""
118 # Map from token to label
120 # Loop until the token is shifted; may raise exceptions
131 # Shift a token; we're done with it
142 # Done with this token
157 # Done parsing, but another token is input
165 """Turn a token into a label. (Internal)"""
166 if type == token.NAME:
175 raise ParseError("bad token", type, value, context)
179 """Shift a token. (Internal)"""