Detailed Description

A TokenReader reads sequential tokens from a file or string.

It can get or peek the next token and it will track line numbers for informative diagnosis output.

The token stream is meant to be XML compliant, i.e., there are two dedicated token types that mark the begin and the end of XML elements while all other token types represent atomic data such as integers or strings that form the XML character data; see the documentation of the class Token for details.

The TokenReader maintains a current position in the stream and implements searching within nested elements and for sequential access of the tokens within each element; e.g. ReadBegin(const std::string&) will search for the element with the specified name.

Reading from the Tokenreader by a particular method encodes the type of the requested data, e.g. ReadInteger() to read an integer token. If the token at the current position does not match the requiested type, an exception is thrown. There are alose Get() and Peek() methods to retrieve and inspect the token at the current position.

For convenience, the TokenReader also implements reading all contents of an alement and larger chunks of data formtated as CDATA markup within an element; e.g. ReadVerbatim(const std::string&, std::string&) reads the contents of the specified element en block as one string.

Definition at line 63 of file cfl_tokenreader.h.

#include <cfl_tokenreader.h>

Classes
struct	LState
	State on entry of respective level. More...

Public Types
enum	Mode { File , Stdin , String }
	Mode of operation: read from file, stdin or string. More...

Public Member Functions
	TokenReader (Mode mode, const std::string &rInString="")
	TokenReader constructor. More...

	TokenReader (const std::string &rFilename)
	Creates a TokenReader for reading a file. More...

	~TokenReader (void)
	Destruct. More...

void	Rewind (void)
	Rewind stream. More...

std::istream *	Streamp (void)
	Access C++ stream. More...

Mode	SourceMode (void) const
	Access stream mode. More...

std::string	FileName (void) const
	Access the filename. More...

bool	Peek (Token &token)
	Peek next token. More...

bool	Get (Token &token)
	Get next token. More...

bool	ExistsBegin (const std::string &rLabel)
	Search for specified element. More...

void	ReadBegin (const std::string &rLabel)
	Open a section by specified label. More...

void	ReadBegin (const std::string &rLabel, Token &rToken)
	Open a section by specified label. More...

void	ReadEnd (const std::string &rLabel)
	Close the current section by matching the previous ReadBegin(). More...

void	SeekBegin (const std::string &rLabel)
	Find specified begin label. More...

void	SeekBegin (const std::string &rLabel, Token &rToken)
	Find specified begin label. More...

bool	Eos (const std::string &rLabel)
	Peek a token and check whether it ends the specified section. More...

long int	ReadInteger (void)
	Read integer token. More...

double	ReadFloat (void)
	Read float token. More...

std::string	ReadString (void)
	Read string token. More...

std::string	ReadOption (void)
	Read option token. More...

void	ReadBinary (std::string &rData)
	Read binary token. More...

void	ReadText (const std::string &rLabel, std::string &rText)
	Read plain text. More...

void	ReadVerbatim (const std::string &rLabel, std::string &rText)
	Read verbatim text. More...

void	ReadCharacterData (std::string &rData)
	Read plain text. More...

void	ReadSection (std::string &rSectionString)
	Read XML section. More...

bool	operator>> (Token &token)
	Operator for get. More...

int	Line (void) const
	Return number of lines read. More...

int	Level (void) const
	Return current level of section nesting. More...

bool	Recover (int level)
	Recover by skipping tokens until returning to the specified level of section nesting. More...

bool	Reset (int level=-1)
	Reset to the begining of the specified level of section nesting. More...

std::string	FileLine (void) const
	Return "filename:line". More...

Private Attributes
Mode	mMode
	input mode More...

std::istream *	mpStream
	istream object pointer More...

std::ifstream	mFStream
	actual stream object, file input More...

std::istringstream *	mpSStream
	actual stream object on heap, string input More...

std::string	mFileName
	Filename. More...

int	mLineCount
	Line counter. More...

long int	mFilePos
	file position More...

bool	mFaudesComments
	flag to ignore faudes comments marked by '' More...

int	mLevel
	Level (of nested sections) More...

std::vector< LState >	mLevelState

Token	mPeekToken
	peek buffer More...

Member Enumeration Documentation

◆ Mode

enum faudes::TokenReader::Mode

Mode of operation: read from file, stdin or string.

Enumerator
File
Stdin
String

Definition at line 69 of file cfl_tokenreader.h.

Constructor & Destructor Documentation

◆ TokenReader() [1/2]

faudes::TokenReader::TokenReader	(	Mode	mode,
		const std::string &	rInString = `""`
	)

TokenReader constructor.

Parameters

mode	select source: File, Stdin or String
rInString	string to read from or filename, respectively.

Exceptions

Exception

faudes::Exception ios error opening file (id 1)
faudes::Exception invalid mode (Stdin not implemented) (id 1)

Definition at line 32 of file cfl_tokenreader.cpp.

◆ TokenReader() [2/2]

faudes::TokenReader::TokenReader ( const std::string & rFilename )

Creates a TokenReader for reading a file.

This is a convenience wrapper for TokenReader(Mode, const std::string&).

Parameters

rFilename file to read

Exceptions

Exception

faudes::Exception ios error opening file (id 1)

Definition at line 68 of file cfl_tokenreader.cpp.

◆ ~TokenReader()

faudes::TokenReader::~TokenReader ( void )

Destruct.

Definition at line 89 of file cfl_tokenreader.cpp.

Member Function Documentation

◆ Eos()

bool faudes::TokenReader::Eos ( const std::string & rLabel )

Peek a token and check whether it ends the specified section.

This function is meant for scanning a section with a while- construct.

ReadBegin("MySec"); 
while(!Eos("MySec")) { 
  ... 
}; 
ReadEnd("MySec");

Parameters

rLabel Token label to specify section

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
unexpected eof or section mismatch (id 51)

Returns: True at end of section

Definition at line 439 of file cfl_tokenreader.cpp.

◆ ExistsBegin()

bool faudes::TokenReader::ExistsBegin ( const std::string & rLabel )

Search for specified element.

This function searches for the specified section on the current level. It skips any sections on the levels below, and it will wrap to the begin of the current section once. In the case of success, it returns true, else false. In contrast to other token i/o methodes, this method will not throw any execptions.

In the case of success, the next token is the begin-tag of the specified element, which can be read with ReadBegin().

Parameters

rLabel Token label to specify section

Returns: True if sectiob exists

Definition at line 314 of file cfl_tokenreader.cpp.

◆ FileLine()

std::string faudes::TokenReader::FileLine ( void ) const

Return "filename:line".

Definition at line 667 of file cfl_tokenreader.cpp.

◆ FileName()

std::string faudes::TokenReader::FileName ( void ) const

Access the filename.

Returns the name of the attached file, if any. For string mode and console mode dummy values are returned.

Returns: filename

Definition at line 125 of file cfl_tokenreader.cpp.

◆ Get()

bool faudes::TokenReader::Get ( Token & token )

Get next token.

Same as Pekk() except that the token is removed from the buffer.

Parameters

token Reference to token

Exceptions

Exception faudes exception ios (id 1)

Returns: true for a valid token, false for eof or other stream errors

Definition at line 151 of file cfl_tokenreader.cpp.

◆ Level()

int faudes::TokenReader::Level ( void ) const

inline

Return current level of section nesting.

Returns: Number of lines read

Definition at line 493 of file cfl_tokenreader.h.

◆ Line()

int faudes::TokenReader::Line ( void ) const

Return number of lines read.

Returns: Number of lines read

Definition at line 662 of file cfl_tokenreader.cpp.

◆ operator>>()

bool faudes::TokenReader::operator>> ( Token & token )

inline

Operator for get.

Definition at line 475 of file cfl_tokenreader.h.

◆ Peek()

bool faudes::TokenReader::Peek ( Token & token )

Peek next token.

Copies the next token to the provided reference and returns true on success. The token remains in an internal buffer.

Technical note: we should have used a const-ref as return in orde to avoid the copy. However, this will require a tedious rewrite.

Parameters

token reference to token

Returns: true for a valid token, false for eof or other stream errors

Exceptions

Exception

faudes::Exception ios error reading file (id 1)

Definition at line 130 of file cfl_tokenreader.cpp.

◆ ReadBegin() [1/2]

void faudes::TokenReader::ReadBegin ( const std::string & rLabel )

Open a section by specified label.

This function searches for the section on this level, it skips any sections on levels below this level, and it will wrap once to the begin of the current section. In the case of success, the matching begin token is the last token read. After processing the section, a matching ReadEnd(label) must be called. If the specified element does not exist, an exception is thrown.

Parameters

rLabel Token label to specify section

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section begin label not found (id 51)

Definition at line 249 of file cfl_tokenreader.cpp.

◆ ReadBegin() [2/2]

void faudes::TokenReader::ReadBegin	(	const std::string &	rLabel,
		Token &	rToken
	)

Open a section by specified label.

This wrapper ReadBegin(const std::string&) will return the actual begin tag in its second argument, e.g., to inspect XML attributes.

Parameters

rLabel	Token label to specify section
rToken	Begin tag as found in token stream.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section begin label not found (id 51)

Definition at line 255 of file cfl_tokenreader.cpp.

◆ ReadBinary()

void faudes::TokenReader::ReadBinary ( std::string & rData )

Read binary token.

Reads the next token and interprets it as an base64 encoded binary array.

Parameters

rData Buffer to read data

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Definition at line 504 of file cfl_tokenreader.cpp.

◆ ReadCharacterData()

void faudes::TokenReader::ReadCharacterData ( std::string & rData )

Read plain text.

Read all text until and excluding the next markup tag. This method does no interpretation/substitution at all. It is meant to implemet carbon copy of text sections.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Stream mismatch (id 50)

Parameters

rData Buffer to read characterdata

Definition at line 610 of file cfl_tokenreader.cpp.

◆ ReadEnd()

void faudes::TokenReader::ReadEnd ( const std::string & rLabel )

Close the current section by matching the previous ReadBegin().

Reads all tokens up to and including end of current section.

Parameters

rLabel Token label to specify section

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section end label not found (id 51)

Definition at line 364 of file cfl_tokenreader.cpp.

◆ ReadFloat()

double faudes::TokenReader::ReadFloat ( void )

Read float token.

Reads the next token and interprets it as a float.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Returns: float value of token

Definition at line 467 of file cfl_tokenreader.cpp.

◆ ReadInteger()

long int faudes::TokenReader::ReadInteger ( void )

Read integer token.

Reads the next token and interprets it as an non-negative integer.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Returns: value of index token

Definition at line 455 of file cfl_tokenreader.cpp.

◆ ReadOption()

std::string faudes::TokenReader::ReadOption ( void )

Read option token.

Reads the next token and interprets it as an option.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Returns: value of name token

Definition at line 492 of file cfl_tokenreader.cpp.

◆ ReadSection()

void faudes::TokenReader::ReadSection ( std::string & rSectionString )

Read XML section.

Reads the current element, including all character data and markup, until and excluding the matching end tag. This method does no interpretation whatsoever. The result is a string that represents the respective section in plain XML format and can be used for expernal post-processing.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Stream mismatch (id 50)

Parameters

rSectionString Buffer to read section

Definition at line 634 of file cfl_tokenreader.cpp.

◆ ReadString()

std::string faudes::TokenReader::ReadString ( void )

Read string token.

Reads the next token and interprets it as a string.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Returns: value of name token

Definition at line 479 of file cfl_tokenreader.cpp.

◆ ReadText()

void faudes::TokenReader::ReadText	(	const std::string &	rLabel,
		std::string &	rText
	)

Read plain text.

Interpret the specified section as plain charater data section, read the character data and interpret relevant entities. Leading and trailing whitespaces are ignored, other formating is maintained.

This method facilitates the input of paragraphs of plain ASCII text with no other markup as the relevant entities (i.e. no HTML or RTF).

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Stream mismatch (id 50)

Parameters

rLabel	Buffer to read Text
rText	Name of section to read text from

Definition at line 517 of file cfl_tokenreader.cpp.

◆ ReadVerbatim()

void faudes::TokenReader::ReadVerbatim	(	const std::string &	rLabel,
		std::string &	rText
	)

Read verbatim text.

Interpret the section as plain charater data section, read the character data from either one faudes string token or from consecutive CDATA markups. Leading and trailing whitespaces are ignored, other formating is maintained.

This method facilitates the input of paragraphs of plain ASCII text with excessive use of special characters, e.g., program fragments.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Stream mismatch (id 50)

Parameters

rLabel	Buffer to read Text
rText	Name of section to read text from

Definition at line 568 of file cfl_tokenreader.cpp.

◆ Recover()

bool faudes::TokenReader::Recover ( int level )

Recover by skipping tokens until returning to the specified level of section nesting.

Returns: True on success

Definition at line 397 of file cfl_tokenreader.cpp.

◆ Reset()

bool faudes::TokenReader::Reset ( int level = -1 )

Reset to the begining of the specified level of section nesting.

Parameters

level target level, defaults to current level

Returns: True on success

Definition at line 412 of file cfl_tokenreader.cpp.

◆ Rewind()

void faudes::TokenReader::Rewind ( void )

Rewind stream.

Reset the internal state to its initial value, i.e., the current position is the beginning of the stream. This is not functional in console mode.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)

Definition at line 99 of file cfl_tokenreader.cpp.

◆ SeekBegin() [1/2]

void faudes::TokenReader::SeekBegin ( const std::string & rLabel )

Find specified begin label.

This function searches for the section on this level and any descending level. It does not read the specified section tag, but stops just one token before (and in this regard matches the behaviour of ExistsBegin()).

Technical note: Former versions of libFAUDES also read the actual begin token and required a matching call of SeekEnd(). As of version 2.18a, this is not supported anymore. The previous behaviour was rarely needed and can be mimiqued by an ordinary ReadEnd() with a subsequent Recover(level).

Parameters

rLabel Label to specify section

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section begin not found (id 51)

Definition at line 207 of file cfl_tokenreader.cpp.

◆ SeekBegin() [2/2]

void faudes::TokenReader::SeekBegin	(	const std::string &	rLabel,
		Token &	rToken
	)

Find specified begin label.

This version SeekBegin(const std::string&) will return the actual begin tag in its second argument.

Parameters

rLabel	Token label to specify section
rToken	Begin tag as found in token stream.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section begin label not found (id 51)

Definition at line 213 of file cfl_tokenreader.cpp.

◆ SourceMode()

Mode faudes::TokenReader::SourceMode ( void ) const

inline

Access stream mode.

Returns mode from construction, i.e. file, string or console.

Returns: Mode

Definition at line 136 of file cfl_tokenreader.h.

◆ Streamp()

std::istream * faudes::TokenReader::Streamp ( void )

Access C++ stream.

This method provides direc access to the underlying C++ stream. After any such access, the TokenReader must be Rewind() to reset its internal state.

Returns: pointer to C++ input stream

Definition at line 94 of file cfl_tokenreader.cpp.

Member Data Documentation

◆ mFaudesComments

bool faudes::TokenReader::mFaudesComments

private

flag to ignore faudes comments marked by ''

Definition at line 542 of file cfl_tokenreader.h.

◆ mFileName

std::string faudes::TokenReader::mFileName

private

Filename.

Definition at line 533 of file cfl_tokenreader.h.

◆ mFilePos

long int faudes::TokenReader::mFilePos

private

file position

Definition at line 539 of file cfl_tokenreader.h.

◆ mFStream

std::ifstream faudes::TokenReader::mFStream

private

actual stream object, file input

Definition at line 527 of file cfl_tokenreader.h.

◆ mLevel

int faudes::TokenReader::mLevel

private

Level (of nested sections)

Definition at line 545 of file cfl_tokenreader.h.

◆ mLevelState

std::vector<LState> faudes::TokenReader::mLevelState

private

Definition at line 555 of file cfl_tokenreader.h.

◆ mLineCount

int faudes::TokenReader::mLineCount

private

Line counter.

Definition at line 536 of file cfl_tokenreader.h.

◆ mMode

Mode faudes::TokenReader::mMode

private

input mode

Definition at line 521 of file cfl_tokenreader.h.

◆ mPeekToken

Token faudes::TokenReader::mPeekToken

private

peek buffer

Definition at line 558 of file cfl_tokenreader.h.

◆ mpSStream

std::istringstream* faudes::TokenReader::mpSStream

private

actual stream object on heap, string input

Definition at line 530 of file cfl_tokenreader.h.

◆ mpStream

std::istream* faudes::TokenReader::mpStream

private

istream object pointer

Definition at line 524 of file cfl_tokenreader.h.

The documentation for this class was generated from the following files:

Detailed Description

Classes

Public Types

Public Member Functions

Private Attributes

Member Enumeration Documentation

◆ Mode

Constructor & Destructor Documentation

◆ TokenReader() [1/2]

◆ TokenReader() [2/2]

◆ ~TokenReader()

Member Function Documentation

◆ Eos()

◆ ExistsBegin()

◆ FileLine()

◆ FileName()

◆ Get()

◆ Level()

◆ Line()

◆ operator>>()

◆ Peek()

◆ ReadBegin() [1/2]

◆ ReadBegin() [2/2]

◆ ReadBinary()

◆ ReadCharacterData()

◆ ReadEnd()

◆ ReadFloat()

◆ ReadInteger()

◆ ReadOption()

◆ ReadSection()

◆ ReadString()

◆ ReadText()

◆ ReadVerbatim()

◆ Recover()

◆ Reset()

◆ Rewind()

◆ SeekBegin() [1/2]

◆ SeekBegin() [2/2]

◆ SourceMode()

◆ Streamp()

Member Data Documentation

◆ mFaudesComments

◆ mFileName

◆ mFilePos

◆ mFStream

◆ mLevel

◆ mLevelState

◆ mLineCount

◆ mMode

◆ mPeekToken

◆ mpSStream

◆ mpStream