TokenizerInterface
A lossless tokenizer for TypoScript syntax.
tokenize() creates a stream of LineInterface objects from a TypoScript string, each line contains the important streams or tokens of a single line.
There are two tokenizer implementations:
- LossyTokenizer: This one skip all invalid lines and comments and everything that is not needed for AST building.
- LosslessTokenizer: This one creates a stream of lines useful for backend template module to elaborate on details and failures in TypoScript.
The tokenizer does not parse conditions or includes itself (no file / db lookups), this is part of the IncludeTree parser.
Tags
Table of Contents
Methods
Methods
tokenize()
public
tokenize(string $source) : LineStream
Parameters
- $source : string