Interpreter in Go - 4
In Chapter 1.3 of [Ball’s Writing an Interpreter in Go][1], we encounter one design decision of his Monkey programming language. Here, the lexer has a NextToken() method that looks like this:
func (l *Lexer) NextToken() token.Token {
var tok token.Token
switch l.ch {
// [...]
default:
if isLetter(l.ch) {
tok.Literal = l.readIdentifier()
return tok
} else {
tok = newToken(token.ILLEGAL, l.ch)
}
}
// [...]
}
This means the lexer itself does not do backtracking. The meaning of a character at any point cannot be ambiguous. You cannot say, for example, that ‘+’ is the ‘plus’ token unless it is in the middle of a variable name. I don’t know many programming languages that support such behavior – so it is probably an acceptable design decision. You know what they say, “Keep it Simple, Smartypants”. I’m not sure if there are other notable constraints introduced by the design at this point, but it is something that tickles my overly analytical brain.