hoshi-lang dev
Yet another programming language
Loading...
Searching...
No Matches
Classes | Public Member Functions | Public Attributes | Private Attributes | List of all members
lexer Class Reference

#include <lexer.hpp>

Collaboration diagram for lexer:
Collaboration graph
[legend]

Classes

struct  Comment
 
struct  lexerState
 
struct  token
 

Public Member Functions

void getCh ()
 
 lexer (std::wstringstream ss)
 
void saveState ()
 
void returnState ()
 
void dropState ()
 
token scan ()
 
token alphaStart ()
 
token operatorStart ()
 
token strStart ()
 
token digitStart ()
 
token minusStart ()
 
token plusStart ()
 
token asteriskStart ()
 
token slashStart ()
 
token percentSignStart ()
 
token equalStart ()
 
token notStart ()
 
token lessStart ()
 
token greaterStart ()
 
token semicolonStart ()
 
token colonStart ()
 
token commaStart ()
 
token dotStart ()
 
token sharpStart ()
 
token leftParenthesesStart ()
 
token rightParenthesesStart ()
 
token leftBracketStart ()
 
token rightBracketStart ()
 
token leftBracesStart ()
 
token rightBracesStart ()
 
token andStart ()
 
token orStart ()
 
token xorStart ()
 
token binaryNotStart ()
 

Public Attributes

vec< Comment > comments
 
vec< lexerState > states
 
token curToken
 
wchar curCh
 
int64_t line
 
int64_t col
 

Private Attributes

std::wstringstream stream
 

Detailed Description

Definition at line 14 of file lexer.hpp.

Constructor & Destructor Documentation

◆ lexer()

lexer ( std::wstringstream  ss)
explicit

Definition at line 8 of file lexer.cpp.

References lexer::getCh().

Here is the call graph for this function:

Member Function Documentation

◆ alphaStart()

lexer::token alphaStart ( )

Definition at line 96 of file lexer.cpp.

References lexer::token::boolean, lexer::col, lexer::curCh, lexer::getCh(), lexer::token::identifier, lexer::token::kAlias, lexer::token::kAlwaysInline, lexer::token::kAs, lexer::token::kBreak, lexer::token::kCallable, lexer::token::kCast, lexer::token::kCatch, lexer::token::kConcept, lexer::token::kConstructor, lexer::token::kContinue, lexer::token::kDataField, lexer::token::kDataStruct, lexer::token::kDecltype, lexer::token::kDynCast, lexer::token::kElif, lexer::token::kElse, lexer::token::kEnum, lexer::token::kExport, lexer::token::kFinalizer, lexer::token::kFinally, lexer::token::kFor, lexer::token::kForEach, lexer::token::kFrom, lexer::token::kFunc, lexer::token::kGenerator, lexer::token::kIf, lexer::token::kImpl, lexer::token::kImport, lexer::token::kIn, lexer::token::kInterface, lexer::token::kInterfaceOf, lexer::token::kIntrinsic, lexer::token::kLet, lexer::token::kNew, lexer::token::kNoFFI, lexer::token::kNull, lexer::token::kReturn, lexer::token::kSatisfy, lexer::token::kStatic, lexer::token::kStruct, lexer::token::kThrow, lexer::token::kTry, lexer::token::kTypeId, lexer::token::kUse, lexer::token::kWeak, lexer::token::kWhile, lexer::token::kYield, lexer::line, lexer::operatorStart(), and lexer::scan().

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ andStart()

lexer::token andStart ( )

Definition at line 562 of file lexer.cpp.

References lexer::token::binaryAnd, lexer::col, lexer::curCh, lexer::getCh(), lexer::line, and lexer::token::logicAnd.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ asteriskStart()

lexer::token asteriskStart ( )

Definition at line 372 of file lexer.cpp.

References lexer::token::asterisk, lexer::col, lexer::curCh, lexer::getCh(), lexer::line, and lexer::token::multiplicationAssignment.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ binaryNotStart()

lexer::token binaryNotStart ( )

Definition at line 594 of file lexer.cpp.

References lexer::token::binaryNot, lexer::col, lexer::getCh(), and lexer::line.

Here is the call graph for this function:

◆ colonStart()

lexer::token colonStart ( )

Definition at line 482 of file lexer.cpp.

References lexer::col, lexer::token::colon, lexer::curCh, lexer::token::directAssignSign, lexer::getCh(), and lexer::line.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ commaStart()

lexer::token commaStart ( )

Definition at line 492 of file lexer.cpp.

References lexer::col, lexer::token::comma, lexer::getCh(), and lexer::line.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ digitStart()

lexer::token digitStart ( )

Definition at line 235 of file lexer.cpp.

References lexer::col, lexer::curCh, lexer::token::decimal, lexer::getCh(), lexer::token::integer, lexer::line, lexer::token::shortInt, lexer::token::unsignedInt, and yoi::yoi_assert().

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ dotStart()

lexer::token dotStart ( )

Definition at line 498 of file lexer.cpp.

References lexer::col, lexer::curCh, lexer::token::dot, lexer::getCh(), lexer::token::kThreeDots, lexer::line, and lexer::stream.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ dropState()

void dropState ( )

Definition at line 557 of file lexer.cpp.

References lexer::states.

Referenced by yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), and lexer::returnState().

Here is the caller graph for this function:

◆ equalStart()

lexer::token equalStart ( )

Definition at line 430 of file lexer.cpp.

References lexer::token::assignSign, lexer::col, lexer::curCh, lexer::token::equal, lexer::getCh(), and lexer::line.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ getCh()

void getCh ( )

Definition at line 12 of file lexer.cpp.

References lexer::col, lexer::curCh, lexer::line, yoi::panic(), and lexer::stream.

Referenced by lexer::alphaStart(), lexer::andStart(), lexer::asteriskStart(), lexer::binaryNotStart(), lexer::colonStart(), lexer::commaStart(), lexer::digitStart(), lexer::dotStart(), lexer::equalStart(), lexer::greaterStart(), lexer::leftBracesStart(), lexer::leftBracketStart(), lexer::leftParenthesesStart(), lexer::lessStart(), lexer::lexer(), lexer::minusStart(), lexer::notStart(), lexer::orStart(), lexer::percentSignStart(), lexer::plusStart(), lexer::rightBracesStart(), lexer::rightBracketStart(), lexer::rightParenthesesStart(), lexer::scan(), lexer::semicolonStart(), lexer::sharpStart(), lexer::slashStart(), lexer::strStart(), and lexer::xorStart().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ greaterStart()

lexer::token greaterStart ( )

Definition at line 463 of file lexer.cpp.

References lexer::token::binaryShiftRight, lexer::col, lexer::curCh, lexer::getCh(), lexer::token::greaterEqual, lexer::token::greaterThan, and lexer::line.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ leftBracesStart()

lexer::token leftBracesStart ( )

Definition at line 533 of file lexer.cpp.

References lexer::col, lexer::getCh(), lexer::token::leftBraces, and lexer::line.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ leftBracketStart()

lexer::token leftBracketStart ( )

Definition at line 521 of file lexer.cpp.

References lexer::col, lexer::getCh(), lexer::token::leftBracket, and lexer::line.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ leftParenthesesStart()

lexer::token leftParenthesesStart ( )

Definition at line 509 of file lexer.cpp.

References lexer::col, lexer::getCh(), lexer::token::leftParentheses, and lexer::line.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ lessStart()

lexer::token lessStart ( )

Definition at line 450 of file lexer.cpp.

References lexer::token::binaryShiftLeft, lexer::col, lexer::curCh, lexer::getCh(), lexer::token::lessEqual, lexer::token::lessThan, and lexer::line.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ minusStart()

lexer::token minusStart ( )

Definition at line 343 of file lexer.cpp.

References lexer::col, lexer::curCh, lexer::token::decrementSign, lexer::getCh(), lexer::line, lexer::token::minus, lexer::token::subtractionAssignment, and lexer::token::toSign.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ notStart()

lexer::token notStart ( )

Definition at line 440 of file lexer.cpp.

References lexer::col, lexer::curCh, lexer::getCh(), lexer::line, lexer::token::logicNot, and lexer::token::notEqual.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ operatorStart()

lexer::token operatorStart ( )

Definition at line 648 of file lexer.cpp.

References lexer::token::additionAssignment, lexer::token::assignSign, lexer::token::asterisk, lexer::token::binaryAnd, lexer::token::binaryNot, lexer::token::binaryOr, lexer::token::binaryShiftLeft, lexer::token::binaryShiftRight, lexer::token::binaryXor, lexer::col, lexer::curToken, lexer::token::decrementSign, lexer::token::directAssignSign, lexer::token::divisionAssignment, lexer::token::equal, lexer::token::greaterThan, lexer::token::identifier, lexer::token::incrementSign, lexer::token::kind, lexer::token::leftBracket, lexer::token::leftParentheses, lexer::token::lessThan, lexer::line, lexer::token::minus, lexer::token::multiplicationAssignment, lexer::token::notEqual, yoi::panic(), lexer::token::percentSign, lexer::token::plus, lexer::token::reminderAssignment, lexer::token::rightBracket, lexer::token::rightParentheses, lexer::scan(), lexer::token::slash, lexer::token::subtractionAssignment, and yoi::yoi_assert().

Referenced by lexer::alphaStart().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ orStart()

lexer::token orStart ( )

Definition at line 572 of file lexer.cpp.

References lexer::token::binaryOr, lexer::col, lexer::curCh, lexer::getCh(), lexer::line, and lexer::token::logicOr.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ percentSignStart()

lexer::token percentSignStart ( )

Definition at line 420 of file lexer.cpp.

References lexer::col, lexer::curCh, lexer::getCh(), lexer::line, lexer::token::percentSign, and lexer::token::reminderAssignment.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ plusStart()

lexer::token plusStart ( )

Definition at line 359 of file lexer.cpp.

References lexer::token::additionAssignment, lexer::col, lexer::curCh, lexer::getCh(), lexer::token::incrementSign, lexer::line, and lexer::token::plus.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ returnState()

void returnState ( )

Definition at line 549 of file lexer.cpp.

References lexer::lexerState::col, lexer::col, lexer::lexerState::curCh, lexer::curCh, lexer::lexerState::curToken, lexer::curToken, lexer::dropState(), lexer::lexerState::line, lexer::line, lexer::lexerState::pos, lexer::states, and lexer::stream.

Referenced by yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), and yoi::parse().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ rightBracesStart()

lexer::token rightBracesStart ( )

Definition at line 539 of file lexer.cpp.

References lexer::col, lexer::getCh(), lexer::line, and lexer::token::rightBraces.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ rightBracketStart()

lexer::token rightBracketStart ( )

Definition at line 527 of file lexer.cpp.

References lexer::col, lexer::getCh(), lexer::line, and lexer::token::rightBracket.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ rightParenthesesStart()

lexer::token rightParenthesesStart ( )

Definition at line 515 of file lexer.cpp.

References lexer::col, lexer::getCh(), lexer::line, and lexer::token::rightParentheses.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ saveState()

void saveState ( )

Definition at line 545 of file lexer.cpp.

References lexer::col, lexer::curCh, lexer::curToken, lexer::line, lexer::states, and lexer::stream.

Referenced by yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), and yoi::parse().

Here is the caller graph for this function:

◆ scan()

lexer::token scan ( )

Definition at line 29 of file lexer.cpp.

References lexer::alphaStart(), lexer::andStart(), lexer::asteriskStart(), lexer::col, lexer::colonStart(), lexer::commaStart(), lexer::curCh, lexer::curToken, lexer::digitStart(), lexer::dotStart(), lexer::token::eof, lexer::equalStart(), lexer::getCh(), lexer::greaterStart(), lexer::leftBracesStart(), lexer::leftBracketStart(), lexer::leftParenthesesStart(), lexer::lessStart(), lexer::line, lexer::minusStart(), lexer::notStart(), lexer::orStart(), yoi::panic(), lexer::percentSignStart(), lexer::plusStart(), lexer::rightBracesStart(), lexer::rightBracketStart(), lexer::rightParenthesesStart(), lexer::semicolonStart(), lexer::sharpStart(), lexer::slashStart(), lexer::strStart(), and lexer::xorStart().

Referenced by lexer::alphaStart(), visitor::checkMarcoSatisfaction(), compilerContext::compileModule(), compilerContext::initializeSharedObjects(), main(), lexer::operatorStart(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), ProjectIndex::parseAndIndex(), DocumentStore::reparseDocument(), and lexer::slashStart().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ semicolonStart()

lexer::token semicolonStart ( )

Definition at line 476 of file lexer.cpp.

References lexer::col, lexer::getCh(), lexer::line, and lexer::token::semicolon.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ sharpStart()

lexer::token sharpStart ( )

Definition at line 588 of file lexer.cpp.

References lexer::col, lexer::getCh(), lexer::line, and lexer::token::sharp.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ slashStart()

lexer::token slashStart ( )

Definition at line 382 of file lexer.cpp.

References lexer::col, lexer::comments, lexer::curCh, lexer::token::divisionAssignment, lexer::getCh(), lexer::line, lexer::scan(), and lexer::token::slash.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ strStart()

lexer::token strStart ( )

Definition at line 214 of file lexer.cpp.

References lexer::token::character, lexer::col, lexer::curCh, lexer::getCh(), lexer::line, yoi::panic(), yoi::parseString(), and lexer::token::string.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ xorStart()

lexer::token xorStart ( )

Definition at line 582 of file lexer.cpp.

References lexer::token::binaryXor, lexer::col, lexer::getCh(), and lexer::line.

Referenced by lexer::scan().

Here is the call graph for this function:
Here is the caller graph for this function:

Member Data Documentation

◆ col

int64_t col

Definition at line 176 of file lexer.hpp.

Referenced by lexer::alphaStart(), lexer::andStart(), lexer::asteriskStart(), lexer::binaryNotStart(), lexer::colonStart(), lexer::commaStart(), lexer::digitStart(), lexer::dotStart(), lexer::equalStart(), lexer::getCh(), lexer::greaterStart(), lexer::leftBracesStart(), lexer::leftBracketStart(), lexer::leftParenthesesStart(), lexer::lessStart(), lexer::minusStart(), lexer::notStart(), lexer::operatorStart(), lexer::orStart(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), lexer::percentSignStart(), lexer::plusStart(), lexer::returnState(), lexer::rightBracesStart(), lexer::rightBracketStart(), lexer::rightParenthesesStart(), lexer::saveState(), lexer::scan(), lexer::semicolonStart(), lexer::sharpStart(), lexer::slashStart(), lexer::strStart(), and lexer::xorStart().

◆ comments

vec<Comment> comments

Definition at line 23 of file lexer.hpp.

Referenced by lexer::slashStart().

◆ curCh

wchar curCh

Definition at line 174 of file lexer.hpp.

Referenced by lexer::alphaStart(), lexer::andStart(), lexer::asteriskStart(), lexer::colonStart(), lexer::digitStart(), lexer::dotStart(), lexer::equalStart(), lexer::getCh(), lexer::greaterStart(), lexer::lessStart(), lexer::minusStart(), lexer::notStart(), lexer::orStart(), lexer::percentSignStart(), lexer::plusStart(), lexer::returnState(), lexer::saveState(), lexer::scan(), lexer::slashStart(), and lexer::strStart().

◆ curToken

token curToken

Definition at line 173 of file lexer.hpp.

Referenced by lexer::operatorStart(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), lexer::returnState(), lexer::saveState(), and lexer::scan().

◆ line

int64_t line

Definition at line 176 of file lexer.hpp.

Referenced by lexer::alphaStart(), lexer::andStart(), lexer::asteriskStart(), lexer::binaryNotStart(), lexer::colonStart(), lexer::commaStart(), lexer::digitStart(), lexer::dotStart(), lexer::equalStart(), lexer::getCh(), lexer::greaterStart(), lexer::leftBracesStart(), lexer::leftBracketStart(), lexer::leftParenthesesStart(), lexer::lessStart(), lexer::minusStart(), lexer::notStart(), lexer::operatorStart(), lexer::orStart(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), yoi::parse(), lexer::percentSignStart(), lexer::plusStart(), lexer::returnState(), lexer::rightBracesStart(), lexer::rightBracketStart(), lexer::rightParenthesesStart(), lexer::saveState(), lexer::scan(), lexer::semicolonStart(), lexer::sharpStart(), lexer::slashStart(), lexer::strStart(), and lexer::xorStart().

◆ states

vec<lexerState> states

Definition at line 171 of file lexer.hpp.

Referenced by lexer::dropState(), lexer::returnState(), and lexer::saveState().

◆ stream

std::wstringstream stream
private

Definition at line 15 of file lexer.hpp.

Referenced by lexer::dotStart(), lexer::getCh(), lexer::returnState(), and lexer::saveState().

The documentation for this class was generated from the following files: