Skip to main content

Browse the archive

Staging
v0.5.1
https://github.com/python/cpython
09 December 2020, 11:06:31 UTC
Revision 26a9f7467f224897a737a1f6c0614ead9f388554 authored by Jack Jansen on 13 February 1995, 22:47:02 UTC, committed by Jack Jansen on 13 February 1995, 22:47:02 UTC 
Some of Guido's bgen modules aren't think-only anymore
1 parent 6cfab23
Raw File
Tip revision: 26a9f7467f224897a737a1f6c0614ead9f388554 authored by Jack Jansen on 13 February 1995, 22:47:02 UTC
Some of Guido's bgen modules aren't think-only anymore
Tip revision: 26a9f74
token.py 
# Tokens (from "token.h")

ENDMARKER = 0
NAME = 1
NUMBER = 2
STRING = 3
NEWLINE = 4
INDENT = 5
DEDENT = 6
LPAR = 7
RPAR = 8
LSQB = 9
RSQB = 10
COLON = 11
COMMA = 12
SEMI = 13
PLUS = 14
MINUS = 15
STAR = 16
SLASH = 17
VBAR = 18
AMPER = 19
LESS = 20
GREATER = 21
EQUAL = 22
DOT = 23
PERCENT = 24
BACKQUOTE = 25
LBRACE = 26
RBRACE = 27
EQEQUAL = 28
NOTEQUAL = 29
LESSEQUAL = 30
GREATEREQUAL = 31
TILDE = 32
CIRCUMFLEX = 33
LEFTSHIFT = 34
RIGHTSHIFT = 35
OP = 36
ERRORTOKEN = 37

names = dir()
tok_name = {}
for name in names:
	number = eval(name)
	tok_name[number] = name

N_TOKENS = 38				# Number of tokens including ERRORTOKEN

NT_OFFSET = 256				# Start of non-terminal symbols
back to top