Home | Trees | Index | Help |
|
---|
Package DNS :: Module tokenizer :: Class Tokenizer |
|
object
--+
|
Tokenizer
A DNS master file format tokenizer.
A token is a (type, value) tuple, where type is an int, and value is a string. The valid types are EOF, EOL, WHITESPACE, IDENTIFIER, QUOTED_STRING, COMMENT, and DELIMITER.Method Summary | |
---|---|
Initialize a tokenizer instance. | |
__iter__(self)
| |
a new object with type S, a subtype of T |
|
(int, string) tuple |
Get the next token. |
string |
Read a character from input. |
string |
Read the next token and raise an exception if it isn't EOL or EOF. |
int |
Read the next token and interpret it as an integer. |
DNS.name.Name object |
Read the next token and interpret it as a DNS name. |
string |
Read the next token and interpret it as a string. |
int |
Read the next token and interpret it as a 16-bit unsigned integer. |
int |
Read the next token and interpret it as a 32-bit unsigned integer. |
(int, string) |
Return the next item in an iteration. |
int |
Consume input until a non-whitespace character is encountered. |
Unget a token. | |
Unget a character. |
Instance Variable Summary | |
---|---|
dict | delimiters
- The current delimiter dictionary. |
bool | eof
- This variable is true if the tokenizer has encountered EOF. |
file | file
- The file tokenize |
int | multiline
- The current multiline level. |
bool | quoting
- This variable is true if the tokenizer is currently reading a quoted
string. |
string | ungotten_char
- The most recently ungotten character, or None. |
string | ungotten_token
- The most recently ungotten token, or None. |
Method Details |
---|
__init__(self,
f=<epydoc.imports._DevNull instance at 0x...)
Initialize a tokenizer instance.
|
__new__(S, ...)
|
get(self, want_leading=0, want_comment=0)Get the next token.
|
get_char(self)Read a character from input.
|
get_eol(self)Read the next token and raise an exception if it isn't EOL or EOF.
|
get_int(self)Read the next token and interpret it as an integer.
|
get_name(self, origin=None)Read the next token and interpret it as a DNS name.
|
get_string(self, origin=None)Read the next token and interpret it as a string.
|
get_uint16(self)Read the next token and interpret it as a 16-bit unsigned integer.
|
get_uint32(self)Read the next token and interpret it as a 32-bit unsigned integer.
|
next(self)Return the next item in an iteration.
|
skip_whitespace(self)Consume input until a non-whitespace character is encountered. The non-whitespace character is then ungotten, and the number of whitespace characters consumed is returned. If the tokenizer is in multiline mode, then newlines are whitespace.
|
unget(self, token)Unget a token. The unget buffer for tokens is only one token large; it is an error to try to unget a token when the unget buffer is not empty.
|
unget_char(self, c)Unget a character. The unget buffer for characters is only one character large; it is an error to try to unget a character when the unget buffer is not empty.
|
Instance Variable Details |
---|
delimitersThe current delimiter dictionary.
|
eofThis variable is true if the tokenizer has encountered EOF.
|
fileThe file tokenize
|
multilineThe current multiline level. This value is increased by one every time a '(' delimiter is read, and decreased by one every time a ')' delimiter is read.
|
quotingThis variable is true if the tokenizer is currently reading a quoted string.
|
ungotten_charThe most recently ungotten character, or None.
|
ungotten_tokenThe most recently ungotten token, or None.
|
Home | Trees | Index | Help |
|
---|
Generated by Epydoc 1.1 on Thu Jun 5 22:37:31 2003 | http://epydoc.sf.net |