ezTokenType Struct Reference

Describes which kind of token an ezToken is. More...

#include <Tokenizer.h>

Public Types
enum	Enum {   Unknown, Whitespace, Identifier, NonIdentifier,   Newline, LineComment, BlockComment, String1,   String2, Integer, Float, RawString1,   RawString1Prefix, RawString1Postfix, EndOfFile, ENUM_COUNT }

Static Public Attributes
static const char *	EnumNames [ENUM_COUNT]

Detailed Description

Describes which kind of token an ezToken is.

Member Enumeration Documentation

Enum

enum ezTokenType::Enum

Enumerator
Unknown	for internal use
Whitespace	The token is a space or tab.
Identifier	a series of alphanumerics or underscores
NonIdentifier	Everything else.
Newline	Either ' ' or '\r '.
LineComment	A comment that starts with two slashes and ends at the next newline (or end of file)
BlockComment	A comment that starts with a slash and a star, and ends at the next star/slash combination (or end of file)
String1	A string enclosed in ".
String2	A string enclosed in '.
Integer	An integer number.
Float	A floating point number.
RawString1	A raw c++11 string enclosed in ". Contents do not contain the enclosing " or the start / end marker.
RawString1Prefix	The prefix part of a C++11 string. E.g: R"foo(.
RawString1Postfix	The postfix part of a C++11 string. E.g: )foo".
EndOfFile	End-of-file marker.

Member Data Documentation

EnumNames

const char * ezTokenType::EnumNames

static

Initial value:

= {
  "Unknown",
  "Whitespace",
  "Identifier",
  "NonIdentifier",
  "Newline",
  "LineComment",
  "BlockComment",
  "String1",
  "String2",
  "Integer",
  "Float",
  "RawString1",
  "RawString1Prefix",
  "RawString1Postfix",
  "EndOfFile"}

---

The documentation for this struct was generated from the following files:

• /home/runner/work/ezEngine/ezEngine/Code/Engine/Foundation/CodeUtils/Tokenizer.h
• /home/runner/work/ezEngine/ezEngine/Code/Engine/Foundation/CodeUtils/Implementation/Tokenizer.cpp