Lexical analysis is a process of recognizing the input source program. To recognize tokens lexical analyzers perform following steps-
- Lexical analyzers store the input in input in the input buffer.
- The token is read from the input buffer and regular expressions are built for the corresponding token.
- From these regular expressions, finite automata are built. The finite automata are usually in the non-deterministic form. That means Non-deterministic. Finite Automata (NFA) is built.
- For each state of NFA, a function is designed and each input along the transitional edges corresponds to input parameters of these functions.
- The set of such functions ultimately creates a lexical analyzer program.