lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


On Mon, Feb 3, 2014 at 7:36 AM, francesco
<francescoarmandoporta@gmail.com> wrote:
> For example with this string (line):
>
>     #include <sys/types.h>
>
> I would like to have a collection (table) like this:
>
>     1, line = 1, type = control, value = #
>     2, line = 1, type = word, value = include
>     3, line = 1, type = control, value = >
>     etc.


this is called a tokenizer or lexer (although sometimes a lexer has a
little more knowledge about the specific language, and produces a
syntax tree instead of a token stream).  check
http://en.wikipedia.org/wiki/Lexical_analysis for some discussions on
this domain.

-- 
Javier