| Home | Trees | Index | Help |
|
|---|
| Package dns :: Module tokenizer :: Class Tokenizer |
|
object --+
|
Tokenizer
A DNS master file format tokenizer.
A token is a (type, value) tuple, where type is an int, and value is a string. The valid types are EOF, EOL, WHITESPACE, IDENTIFIER, QUOTED_STRING, COMMENT, and DELIMITER.| Method Summary | |
|---|---|
Initialize a tokenizer instance. | |
__iter__(self)
| |
| a new object with type S, a subtype of T |
|
| (int, string) tuple |
Get the next token. |
| string |
Read a character from input. |
| string |
Read the next token and raise an exception if it isn't EOL or EOF. |
| int |
Read the next token and interpret it as an integer. |
| dns.name.Name object |
Read the next token and interpret it as a DNS name. |
| string |
Read the next token and interpret it as a string. |
| int |
Read the next token and interpret it as a 16-bit unsigned integer. |
| int |
Read the next token and interpret it as a 32-bit unsigned integer. |
| int |
Read the next token and interpret it as an 8-bit unsigned integer. |
| (int, string) |
Return the next item in an iteration. |
| int |
Consume input until a non-whitespace character is encountered. |
Unget a token. | |
Unget a character. | |
| Instance Variable Summary | |
|---|---|
| dict | delimiters
- The current delimiter dictionary. |
| bool | eof
- This variable is true if the tokenizer has encountered EOF. |
| file | file
- The file tokenize |
| int | multiline
- The current multiline level. |
| bool | quoting
- This variable is true if the tokenizer is currently reading a quoted
string. |
| string | ungotten_char
- The most recently ungotten character, or None. |
| string | ungotten_token
- The most recently ungotten token, or None. |
| Method Details |
|---|
__init__(self,
f=<epydoc.imports._DevNull instance at 0x...)
Initialize a tokenizer instance.
|
__new__(S, ...)
|
get(self, want_leading=False, want_comment=False)Get the next token.
|
get_char(self)Read a character from input.
|
get_eol(self)Read the next token and raise an exception if it isn't EOL or EOF.
|
get_int(self)Read the next token and interpret it as an integer.
|
get_name(self, origin=None)Read the next token and interpret it as a DNS name.
|
get_string(self, origin=None)Read the next token and interpret it as a string.
|
get_uint16(self)Read the next token and interpret it as a 16-bit unsigned integer.
|
get_uint32(self)Read the next token and interpret it as a 32-bit unsigned integer.
|
get_uint8(self)Read the next token and interpret it as an 8-bit unsigned integer.
|
next(self)Return the next item in an iteration.
|
skip_whitespace(self)Consume input until a non-whitespace character is encountered. The non-whitespace character is then ungotten, and the number of whitespace characters consumed is returned. If the tokenizer is in multiline mode, then newlines are whitespace.
|
unget(self, token)Unget a token. The unget buffer for tokens is only one token large; it is an error to try to unget a token when the unget buffer is not empty.
|
unget_char(self, c)Unget a character. The unget buffer for characters is only one character large; it is an error to try to unget a character when the unget buffer is not empty.
|
| Instance Variable Details |
|---|
delimitersThe current delimiter dictionary.
|
eofThis variable is true if the tokenizer has encountered EOF.
|
fileThe file tokenize
|
multilineThe current multiline level. This value is increased by one every time a '(' delimiter is read, and decreased by one every time a ')' delimiter is read.
|
quotingThis variable is true if the tokenizer is currently reading a quoted string.
|
ungotten_charThe most recently ungotten character, or None.
|
ungotten_tokenThe most recently ungotten token, or None.
|
| Home | Trees | Index | Help |
|
|---|
| Generated by Epydoc 1.1 on Tue Jun 24 12:10:12 2003 | http://epydoc.sf.net |