Creating an Extension
*********************

.[perex]
An extension is a reusable class that can define custom tags, filters, functions, providers, etc.

We create extensions when we want to reuse our Latte customizations in different projects or share them with others.
It is also useful to create an extension for each web project that will contain all the specific tags and filters you want to use in the project templates.


Extension Class
===============

Extension is a class inheriting from [api:Latte\Extension]. It is registered with Latte using `addExtension()` (or via [configuration file |application:configuration#Latte]):

```php
$latte = new Latte\Engine;
$latte->addExtension(new MyLatteExtension);
```

If you register multiple extensions and they define identically named tags, filters, or functions, the last added extension wins. This also implies that your extensions can override native tags/filters/functions.

Whenever you make a change to a class and auto-refresh is not turned off, Latte will automatically recompile your templates.

A class can implement any of the following methods:

```php
abstract class Extension
{
	/**
	 * Initializes before template is compiler.
	 */
	public function beforeCompile(Engine $engine): void;

	/**
	 * Returns a list of parsers for Latte tags.
	 * @return array<string, callable>
	 */
	public function getTags(): array;

	/**
	 * Returns a list of compiler passes.
	 * @return array<string, callable>
	 */
	public function getPasses(): array;

	/**
	 * Returns a list of |filters.
	 * @return array<string, callable>
	 */
	public function getFilters(): array;

	/**
	 * Returns a list of functions used in templates.
	 * @return array<string, callable>
	 */
	public function getFunctions(): array;

	/**
	 * Returns a list of providers.
	 * @return array<mixed>
	 */
	public function getProviders(): array;

	/**
	 * Returns a value to distinguish multiple versions of the template.
	 */
	public function getCacheKey(Engine $engine): mixed;

	/**
	 * Initializes before template is rendered.
	 */
	public function beforeRender(Template $template): void;
}
```

For an idea of what the extension looks like, take a look at the built-in "CoreExtension":https://github.com/nette/latte/blob/master/src/Latte/Essential/CoreExtension.php.


beforeCompile(Latte\Engine $engine): void .[method]
---------------------------------------------------

Called before the template is compiled. The method can be used for compilation-related initializations, for example.


getTags(): array .[method]
--------------------------

Called when the template is compiled. Returns an associative array *tag name => callable*, which are [tag parsing functions|#Tag Parsing Function].

```php
public function getTags(): array
{
	return [
		'foo' => [FooNode::class, 'create'],
		'bar' => [BarNode::class, 'create'],
		'n:baz' => [NBazNode::class, 'create'],
		// ...
	];
}
```

The `n:baz` tag represents a pure n:attribute, i.e. it is a tag that can only be written as an attribute.

In the case of the `foo` and `bar` tags, Latte will automatically recognize whether they are pairs, and if so, they can be written automatically using n:attributes, including variants with the `n:inner-foo` and `n:tag-foo` prefixes.

The order of execution of such n:attributes is determined by their order in the array returned by `getTags()`. Thus, `n:foo` is always executed before `n:bar`, even if the attributes are listed in reverse order in the HTML tag as `<div n:bar="..." n:foo="...">`.

If you need to determine the order of n:attributes across multiple extensions, use the `order()` helper method, where the `before` xor `after` parameter determines which tags are ordered before or after the tag.

```php
public function getTags(): array
{
	return [
		'foo' => self::order([FooNode::class, 'create'], before: 'bar')]
		'bar' => self::order([BarNode::class, 'create'], after: ['block', 'snippet'])]
	];
}
```


getPasses(): array .[method]
----------------------------

It is called when the template is compiled. Returns an associative array *name pass => callable*, which are functions representing so-called [#compiler passes] that traverse and modify the AST.

Again, the `order()` helper method can be used. The value of the `before` or `after` parameters can be `*` with the meaning before/after all.

```php
public function getPasses(): array
{
	return [
		'optimize' => [Passes::class, 'optimizePass'],
		'sandbox' => self::order([$this, 'sandboxPass'], before: '*'),
		// ...
	];
}
```


beforeRender(Latte\Engine $engine): void .[method]
--------------------------------------------------

It is called before each template rendering. The method can be used, for example, to initialize variables used during rendering.


getFilters(): array .[method]
-----------------------------

It is called before the template is rendered. Returns [filters|extending-latte#filters] as an associative array *filter name => callable*.

```php
public function getFilters(): array
{
	return [
		'batch' => [$this, 'batchFilter'],
		'trim' => [$this, 'trimFilter'],
		// ...
	];
}
```


getFunctions(): array .[method]
-------------------------------

It is called before the template is rendered. Returns [functions|extending-latte#functions] as an associative array *function name => callable*.

```php
public function getFunctions(): array
{
	return [
		'clamp' => [$this, 'clampFunction'],
		'divisibleBy' => [$this, 'divisibleByFunction'],
		// ...
	];
}
```


getProviders(): array .[method]
-------------------------------

It is called before the template is rendered. Returns an array of providers, which are usually objects that use tags at runtime. They are accessed via `$this->global->...`.

```php
public function getProviders(): array
{
	return [
		'myFoo' => $this->foo,
		'myBar' => $this->bar,
		// ...
	];
}
```


getCacheKey(Latte\Engine $engine): mixed .[method]
--------------------------------------------------

It is called before the template is rendered. The return value becomes part of the key whose hash is contained in the name of the compiled template file. Thus, for different return values, Latte will generate different cache files.


How Does Latte Work?
====================

To understand how to define custom tags or compiler passes, it is essential to understand how Latte works under the hood.

Template compilation in Latte simplistically works like this:

- First, the **lexer** tokenizes the template source code into small pieces (tokens) for easier processing
- Then, the **parser** converts the stream of tokens into a meaningful tree of nodes (the Abstract Syntax Tree, AST)
- Finally, the compiler **generates** a PHP class from the AST that renders the template and caches it.

Actually, the compilation is a bit more complicated. Latte **has two** lexers and parsers: one for the HTML template and one for the PHP-like code inside the tags. Also, the parsing doesn't run after tokenization, but the lexer and parser run in parallel in two "threads" and coordinate. It's rocket science :-)

Furthermore, all tags have their own parsing routines. When the parser encounters a tag, it calls its parsing function (it returns [Extension::getTags()|#getTags]).
Their job is to parse the tag arguments and, in the case of paired tags, the inner content. It returns a *node* that becomes part of the AST. See [#Tag parsing function] for details.

When the parser finishes its work, we have a complete AST representing the template. The root node is `Latte\Compiler\Nodes\TemplateNode`. The individual nodes inside the tree then represent not only the tags, but also the HTML elements, their attributes, any expressions used inside the tags, etc.

After this, the so-called [#Compiler passes] come into play, which are functions (returned by [Extension::getPasses()|#getPasses]) that modify the AST.

The whole process, from loading the template content, through parsing, to generating the resulting file, can be sequenced with this code, which you can experiment with and dump the intermediate results:

```php
$latte = new Latte\Engine;
$source = $latte->getLoader()->getContent($file);
$ast = $latte->parse($source);
$latte->applyPasses($ast);
$code = $latte->generate($ast, $file);
```


Example of AST
--------------

To get a better idea of the AST, we add a sample. This is the source template:

```latte
{foreach $category->getItems() as $item}
	<li>{$item->name|upper}</li>
	{else}
	no items found
{/foreach}
```

And this is its representation in the form of AST:

/--pre
Latte\Compiler\Nodes\<b>TemplateNode</b>(
   Latte\Compiler\Nodes\<b>FragmentNode</b>(
      - Latte\Essential\Nodes\<b>ForeachNode</b>(
           expression: Latte\Compiler\Nodes\Php\Expression\<b>MethodCallNode</b>(
              object: Latte\Compiler\Nodes\Php\Expression\<b>VariableNode</b>('$category')
              name: Latte\Compiler\Nodes\Php\<b>IdentifierNode</b>('getItems')
           )
           value: Latte\Compiler\Nodes\Php\Expression\<b>VariableNode</b>('$item')
           content: Latte\Compiler\Nodes\<b>FragmentNode</b>(
              - Latte\Compiler\Nodes\<b>TextNode</b>('  ')
              - Latte\Compiler\Nodes\<b>Html\ElementNode</b>('li')(
                   content: Latte\Essential\Nodes\<b>PrintNode</b>(
                      expression: Latte\Compiler\Nodes\Php\Expression\<b>PropertyFetchNode</b>(
                         object: Latte\Compiler\Nodes\Php\Expression\<b>VariableNode</b>('$item')
                         name: Latte\Compiler\Nodes\Php\<b>IdentifierNode</b>('name')
                      )
                      modifier: Latte\Compiler\Nodes\Php\<b>ModifierNode</b>(
                         filters:
                            - Latte\Compiler\Nodes\Php\<b>FilterNode</b>('upper')
                      )
                   )
                )
            )
            else: Latte\Compiler\Nodes\<b>FragmentNode</b>(
               - Latte\Compiler\Nodes\<b>TextNode</b>('no items found')
            )
        )
   )
)
\--


Custom Tags
===========

Three steps are needed to define a new tag:

- defining [#tag parsing function] (responsible for parsing the tag into a node)
- creating a node class (responsible for [#generating PHP code] and [#AST traversing])
- registering the tag using [Extension::getTags()|#getTags]


Tag Parsing Function
--------------------

Parsing of tags is handled by its parsing function (the one returned by [Extension::getTags()|#getTags]). Its job is to parse and check any arguments inside the tag (it uses TagParser to do this).
Furthermore, if the tag is a pair, it will ask TemplateParser to parse and return the inner content.
The function creates and returns a node, which is usually a child of `Latte\Compiler\Nodes\StatementNode`, and this becomes part of the AST.

We create a class for each node, which we'll do now, and elegantly place the parsing function into it as a static factory. As an example, let's try creating the familiar `{foreach}` tag:

```php
use Latte\Compiler\Nodes\StatementNode;

class ForeachNode extends StatementNode
{
	// a parsing function that just creates a node for now
	public static function create(Latte\Compiler\Tag $tag): self
	{
		$node = $tag->node = new self;
		return $node;
	}

	public function print(Latte\Compiler\PrintContext $context): string
	{
		// code will be added later
	}

	public function &getIterator(): \Generator
	{
		// code will be added later
	}
}
```

The parsing function `create()` is passed an object [api:Latte\Compiler\Tag], which carries basic information about the tag (whether it is a classic tag or n:attribute, what line it is on, etc.) and mainly accesses the [api:Latte\Compiler\TagParser] in `$tag->parser`.

If the tag must have arguments, check for their existence by calling `$tag->expectArguments()`. The methods of the `$tag->parser` object are available for parsing them:

- `parseExpression(): ExpressionNode` for a PHP-like expression (e.g. `10 + 3`)
- `parseUnquotedStringOrExpression(): ExpressionNode` for an expression or unquoted-string
- `parseArguments(): ArrayNode` content of the array (e.g. `10, true, foo => bar`)
- `parseModifier(): ModifierNode` for a modifier (e.g. `|upper|truncate:10`)
- `parseType(): expressionNode` for typehint (e.g. `int|string` or `Foo\Bar[]`)

and a low-level [api:Latte\Compiler\TokenStream] operating directly with tokens:

- `$tag->parser->stream->consume(...): Token`
- `$tag->parser->stream->tryConsume(...): ?Token`

Latte extends the PHP syntax in small ways, for example by adding modifiers, shortened ternary operators, or allowing simple alphanumeric strings to be written without quotes. This is why we use the term *PHP-like* instead of PHP. Thus, the `parseExpression()` method parses `foo` as `'foo'`, for example.
In addition, *unquoted-string* is a special case of a string that also does not need to be quoted, but at the same time does not need to be alphanumeric. For example, it is the path to a file in the `{include ../file.latte}` tag. The `parseUnquotedStringOrExpression()` method is used to parse it.

.[note]
Studying the node classes that are part of Latte is the best way to learn all the nitty-gritty details of the parsing process.

Let's go back to the `{foreach}` tag. In it, we expect arguments of the form `expression + 'as' + second expression`, which we parse as follows:

```php
use Latte\Compiler\Nodes\StatementNode;
use Latte\Compiler\Nodes\Php\ExpressionNode;
use Latte\Compiler\Nodes\AreaNode;

class ForeachNode extends StatementNode
{
	public ExpressionNode $expression;
	public ExpressionNode $value;

	public static function create(Latte\Compiler\Tag $tag): self
	{
		$tag->expectArguments();
		$node = $tag->node = new self;
		$node->expression = $tag->parser->parseExpression();
		$tag->parser->stream->consume('as');
		$node->value = $parser->parseExpression();
		return $node;
	}
}
```

The expressions we have written into the variables `$expression` and `$value` represent subnodes.

.[tip]
Define variables with subnodes as **public** so that they can be modified in [further processing steps |#Compiler Passes] if necessary. It is also necessary to **make them available** for [traversing |#AST Traversing].

For paired tags, like ours, the method must also let TemplateParser parse the inner contents of the tag. This is handled by `yield`, which returns a pair ''[inner content, end tag]''. We store the inner content in the `$node->content` variable.

```php
public AreaNode $content;

public static function create(Latte\Compiler\Tag $tag): \Generator
{
	// ...
	[$node->content, $endTag] = yield;
	return $node;
}
```

The `yield` keyword causes the `create()` method to terminate, returning control back to the TemplateParser, which continues parsing the content until it hits the end tag. It then passes control back to `create()`, which continues from where it left off. Using the `yield`, method automatically returns `Generator`.

You can also pass an array of tag names to `yield` for which you want to stop parsing if they occur before the end tag. This helps us implement the `{foreach}...{else}...{/foreach}` construct. If `{else}` occurs, we parse the content after it into `$node->elseContent`:

```php
public AreaNode $content;
public ?AreaNode $elseContent = null;

public static function create(Latte\Compiler\Tag $tag): \Generator
{
	// ...
	[$node->content, $nextTag] = yield ['else'];
	if ($nextTag?->name === 'else') {
		[$node->elseContent] = yield;
	}

	return $node;
}
```

Returning node completes the tag parsing.


Generating PHP Code
-------------------

Each node must implement the `print()` method. Returns PHP code that renders the given part of the template (runtime code). It is passed an object [api:Latte\Compiler\PrintContext] as a parameter, which has a useful `format()` method that simplifies the assembly of the resulting code.

The `format(string $mask, ...$args)` method accepts the following placeholders in the mask:
- `%node` prints Node
- `%dump` exports the value to PHP
- `%raw` inserts the text directly without any transformation
- `%args` prints ArrayNode as arguments to the function call
- `%line` prints a comment with a line number
- `%escape(...)` escapes the content
- `%modify(...)` applies a modifier
- `%modifyContent(...)` applies a modifier to blocks


Our `print()` function might look like this (we neglect the `else` branch for simplicity):

```php
public function print(Latte\Compiler\PrintContext $context): string
{
	return $context->format(
		<<<'XX'
			foreach (%node as %node) %line {
				%node
			}

			XX,
		$this->expression,
		$this->value,
		$this->position,
		$this->content,
	);
}
```

The `$this->position` variable is already defined by the [api:Latte\Compiler\Node] class and is set by the parser. It contains an [api:Latte\Compiler\Position] object with the position of the tag in the source code in the form of a row and column number.

Runtime code may use auxiliary variables. To avoid collision with variables used by the template itself, it is convention to prefix them with `$ʟ__` characters.

It can also use arbitrary values at runtime, which are passed to the template in the form of providers using the [Extension::getProviders()|#getProviders] method. It accesses them using `$this->global->...`.


AST Traversing
--------------

In order to traverse the AST tree in depth, it is necessary to implement the `getIterator()` method. This will provide access to subnodes:

```php
public function &getIterator(): \Generator
{
	yield $this->expression;
	yield $this->value;
	yield $this->content;
	if ($this->elseContent) {
		yield $this->elseContent;
	}
}
```

Note that `getIterator()` returns a reference. This is what allows node visitors to replace individual nodes with other nodes.

.[warning]
If a node has subnodes, it is necessary to implement this method and make all subnodes available. Otherwise, a security hole could be created. For example, sandbox mode would not be able to control subnodes and ensure that unallowed constructs are not called in them.

Since the `yield` keyword must be present in the method body even if it has no child nodes, write it as follows:

```php
public function &getIterator(): \Generator
{
	if (false) {
		yield;
	}
}
```


AuxiliaryNode
-------------

If you are creating a new tag for Latte, it is advisable to create a dedicated node class for it, which will represent it in the AST tree (see the `ForeachNode` class in the example above). In some cases, you might find the trivial helper node class [AuxiliaryNode|api:Latte\Compiler\Nodes\Php\Expression\AuxiliaryNode] useful, which allows you to pass the body of the `print()` method and the list of nodes made accessible by the `getIterator()` method as constructor parameters:

```php
// Latte\Compiler\Nodes\Php\Expression\AuxiliaryNode
// or Latte\Compiler\Nodes\AuxiliaryNode

$node = new AuxiliaryNode(
	// body of the print() method:
	fn(PrintContext $context, $argNode) => $context->format('myFunc(%node)', $argNode),
	// nodes accessed via getIterator() and also passed into the print() method:
	[$argNode],
);
```


Compiler Passes
===============

Compiler Passes are functions that modify ASTs or collect information in them. They are returned by the [Extension::getPasses()|#getPasses] method.


Node Traverser
--------------

The most common way to work with the AST is by using a [api:Latte\Compiler\NodeTraverser]:

```php
use Latte\Compiler\Node;
use Latte\Compiler\NodeTraverser;

$ast = (new NodeTraverser)->traverse(
	$ast,
	enter: fn(Node $node) => ...,
	leave: fn(Node $node) => ...,
);
```

The *enter* function (ie. visitor) is called when a node is first encountered, before its subnodes are processed. The *leave* function is called after all subnodes have been visited.
A common pattern is that *enter* is used to collect some information and then *leave* performs modifications based on that. At the time when *leave* is called, all the code inside the node will have already been visited and necessary information collected.

How to modify AST? The easiest way is to simply change the properties of the nodes. The second way is to replace the node entirely by returning a new node. Example: the following code will change all integers in the AST to strings (e.g. 42 will be changed to `'42'`).

```php
use Latte\Compiler\Nodes\Php;

$ast = (new NodeTraverser)->traverse(
	$ast,
	leave: function (Node $node) {
		if ($node instanceof Php\Scalar\IntegerNode) {
            return new Php\Scalar\StringNode((string) $node->value);
        }
	},
);
```

An AST can easily contain thousands of nodes, and traversing over all of them may be slow. In some cases, it is possible to avoid a full traversal.

If you are looking for all `Html\ElementNode` in a tree, you know that once you've seen `Php\ExpressionNode`, there is no point in also checking all it's child nodes, because HTML cannot be inside in expressions. In this case, you can instruct the traverser to not recurse into the class node:

```php
$ast = (new NodeTraverser)->traverse(
	$ast,
	enter: function (Node $node) {
		if ($node instanceof Php\ExpressionNode) {
			return NodeTraverser::DontTraverseChildren;
        }
        // ...
	},
);
```

If you are only looking for one specific node, it is also possible to abort the traversal entirely after finding it.

```php
$ast = (new NodeTraverser)->traverse(
	$ast,
	enter: function (Node $node) {
		if ($node instanceof Nodes\ParametersNode) {
			return NodeTraverser::StopTraversal;
        }
        // ...
	},
);
```


Node Helpers
------------

Class [api:Latte\Compiler\NodeHelpers] provides some methods which can find AST nodes that either satisfy a certain callback etc. A couple of examples are shown:

```php
use Latte\Compiler\NodeHelpers;

// finds all HTML element nodes
$elements = NodeHelpers::find($ast, fn(Node $node) => $node instanceof Nodes\Html\ElementNode);

// finds first text node
$text = NodeHelpers::findFirst($ast, fn(Node $node) => $node instanceof Nodes\TextNode);

// converts PHP value node to real value
$value = NodeHelpers::toValue($node);

// converts static textual node to string
$text = NodeHelpers::toText($node);
```