In file ../include/EST_Token.h:

class EST_Token

This class is similar to EST_String but also maintains the original punctuation and whitespace found around the token.

Public Methods

	EST_Token ()
	EST_Token (const EST_String p)
void	init ()
	Basic access to fields void set_token (const EST_String &p) set token from a string void set_token (const char* p) void set_whitespace (const EST_String &p) set whitespace of token void set_whitespace (const char* p) void set_punctuation (const EST_String &p) set (post) punctuation of token void set_punctuation (const char* p) void set_prepunctuation (const EST_String &p) set prepunction void set_prepunctuation (const char* p) const EST_String& whitespace () const EST_String& punctuation () const EST_String& prepunctuation () Access token as a string const EST_String& S () const Access token as a string const EST_String& String () const Access token as a string operator EST_String () const For automatic coercion to EST_String Access token as a int Access token as a long Access token as a float Access token as a double
void	set_row (int r)
void	set_col (int c)
void	set_filepos (int c) Set file position in original EST_TokenStream
EST_String	lstring () Return lower case version of token name
EST_String	ustring () Return upper case version of token name
int	row (void) const Line number in original EST_TokenStream
int	col (void) const Line position in original EST_TokenStream
int	filepos (void) const file position in original EST_TokenStream
const EST_String	pos_description () const A string describing current position, suitable for error messages
EST_Token&	operator = (const EST_Token &a)
EST_Token&	operator = (const EST_String &a)
int	operator == (const EST_String &a)
int	operator != (const EST_String &a)
int	operator == (const char* a)
int	operator != (const char* a)

Public

Documentation

This class is similar to EST_String but also maintains the original punctuation and whitespace found around the token.

EST_Token's primary use is with EST_TokenStream class which allows easy tokenizing of ascii files.

A token consists of four parts, any of which may be empty: a name, the actual token, preceding whitespace, preceding punctuation, the name and succeeding punctuation.

EST_Token()

EST_Token(const EST_String p)

void init()

Basic access to fields

void set_token(const EST_String &p)

set token from a string

void set_token(const char* p)

void set_whitespace(const EST_String &p)

set whitespace of token

void set_whitespace(const char* p)

void set_punctuation(const EST_String &p)

set (post) punctuation of token

void set_punctuation(const char* p)

void set_prepunctuation(const EST_String &p)

set prepunction

void set_prepunctuation(const char* p)

const EST_String& whitespace()

const EST_String& punctuation()

const EST_String& prepunctuation()

Access token as a string

const EST_String& S() const

Access token as a string

const EST_String& String() const

Access token as a string

operator EST_String() const

For automatic coercion to EST_String

Access token as a int

Access token as a long

Access token as a float

Access token as a double

void set_quoted(int q)

Note that this token was quoted (or not)

int quoted() const

TRUE is token was quoted

void set_row(int r)

void set_col(int c)

void set_filepos(int c)

Set file position in original EST_TokenStream

EST_String lstring()

Return lower case version of token name

EST_String ustring()

Return upper case version of token name

int row(void) const

Line number in original EST_TokenStream

int col(void) const

Line position in original EST_TokenStream

int filepos(void) const

file position in original EST_TokenStream

const EST_String pos_description() const

A string describing current position, suitable for error messages

EST_Token& operator = (const EST_Token &a)

EST_Token& operator = (const EST_String &a)

int operator == (const EST_String &a)

int operator != (const EST_String &a)

int operator == (const char* a)

int operator != (const char* a)

This class has no child classes.

Friends:

ostream& operator << (ostream& s, const EST_Token &p)

Author:

Alan W Black (awb@cstr.ed.ac.uk): April 1996

Alphabetic index HTML hierarchy of classes or Java

This page is part of the Edinburgh Speech Tools Library documentation
Copyright University of Edinburgh 1997
Contact: speech_tools@cstr.ed.ac.uk