This is a guide that should teach you how to perform basic lexing in a functional programing language.
Everything was written in Elixir, but you should be able to follow it if you are using any functional programming language.
Or even an imperative programming language like Python, C or Java.\
The first question you should ask yourself is, what is lexing? It's the same as tokenising, scanning or lexical analysis.
That might not help you if you haven't heard of these either. Put simply, lexing is the process of breaking down a string
into meaningful units, indepdendent of context. What a lexer will do is make the following happen: