Home | Trees | Index | Help |
|
---|
Package dns :: Module tokenizer :: Class Tokenizer |
|
object
--+
|
Tokenizer
A DNS master file format tokenizer.
A token is a (type, value) tuple, where type is an int, and value is a string. The valid types are EOF, EOL, WHITESPACE, IDENTIFIER, QUOTED_STRING, COMMENT, and DELIMITER.Method Summary | |
---|---|
Initialize a tokenizer instance. | |
__iter__(self)
| |
a new object with type S, a subtype of T |
|
(int, string) tuple |
Get the next token. |
string |
Read the next token and raise an exception if it isn't EOL or EOF. |
int |
Read the next token and interpret it as an integer. |
dns.name.Name object |
Read the next token and interpret it as a DNS name. |
string |
Read the next token and interpret it as a string. |
int |
Read the next token and interpret it as a 16-bit unsigned integer. |
int |
Read the next token and interpret it as a 32-bit unsigned integer. |
int |
Read the next token and interpret it as an 8-bit unsigned integer. |
(int, string) |
Return the next item in an iteration. |
int |
Consume input until a non-whitespace character is encountered. |
Unget a token. | |
(string, int) tuple. The first item is the filename of the input, the second is the current line number. |
Return the current location in the input. |
Instance Variable Summary | |
---|---|
dict | delimiters
- The current delimiter dictionary. |
bool | eof
- This variable is true if the tokenizer has encountered EOF. |
file | file
- The file to tokenize |
string | filename
- A filename that will be returned by the where method. |
int | line_number
- The current line number |
int | multiline
- The current multiline level. |
bool | quoting
- This variable is true if the tokenizer is currently reading a quoted
string. |
string | ungotten_char
- The most recently ungotten character, or None. |
(int, string) token tuple | ungotten_token
- The most recently ungotten token, or None. |
Method Details |
---|
__init__(self,
f=<epydoc.imports._DevNull instance at 0x...,
filename=None)
Initialize a tokenizer instance.
|
__new__(S, ...)
|
get(self, want_leading=False, want_comment=False)Get the next token.
|
get_eol(self)Read the next token and raise an exception if it isn't EOL or EOF.
|
get_int(self)Read the next token and interpret it as an integer.
|
get_name(self, origin=None)Read the next token and interpret it as a DNS name.
|
get_string(self, origin=None)Read the next token and interpret it as a string.
|
get_uint16(self)Read the next token and interpret it as a 16-bit unsigned integer.
|
get_uint32(self)Read the next token and interpret it as a 32-bit unsigned integer.
|
get_uint8(self)Read the next token and interpret it as an 8-bit unsigned integer.
|
next(self)Return the next item in an iteration.
|
skip_whitespace(self)Consume input until a non-whitespace character is encountered. The non-whitespace character is then ungotten, and the number of whitespace characters consumed is returned. If the tokenizer is in multiline mode, then newlines are whitespace.
|
unget(self, token)Unget a token. The unget buffer for tokens is only one token large; it is an error to try to unget a token when the unget buffer is not empty.
|
where(self)Return the current location in the input.
|
Instance Variable Details |
---|
delimitersThe current delimiter dictionary.
|
eofThis variable is true if the tokenizer has encountered EOF.
|
fileThe file to tokenize
|
filenameA filename that will be returned by thewhere method.
|
line_numberThe current line number
|
multilineThe current multiline level. This value is increased by one every time a '(' delimiter is read, and decreased by one every time a ')' delimiter is read.
|
quotingThis variable is true if the tokenizer is currently reading a quoted string.
|
ungotten_charThe most recently ungotten character, or None.
|
ungotten_tokenThe most recently ungotten token, or None.
|
Home | Trees | Index | Help |
|
---|
Generated by Epydoc 1.1 on Sun Jun 6 13:48:46 2004 | http://epydoc.sf.net |