# Format Modules

Format modules tell *Lx* how to recognize and work with different kinds of files.

Each format module may provide file detection, syntax highlighting, comments, outlines, text operations, opening hints, and other capabilities. Syntax is still the right name for its text-highlighting rules; it is one part of the selected format rather than the identity of the whole module.

The format tells *Lx* things such as:

- Which files belong to this type
- Which text should receive a color
- How comments are written
- What should appear in the outline
- Which formatting choices are available
- Which characters can surround selected text
- Whether the file needs special handling when it opens

This guide explains the complete system. You do not need to be an experienced programmer. Each part starts with what it does, then shows the exact setting *Lx* understands.

## Using an existing format

*Lx* normally chooses for you. It looks at the file name, its ending, and sometimes the beginning of the file.

If it chooses incorrectly, open the command palette and run:

```text
doc:set-format
```

Choose the format you want. The document updates immediately.

Changing the format changes how *Lx* reads and displays the document. It does not rewrite your text.

## Where format modules live

Built-in format modules live in `data/formats`.

Each format module is a small Lua description. It calls `format.register` once to give *Lx* its detection rules and capabilities.

Here is a small example:

```lua
local format = require("core.format")

format.register({
	name = "Example",
	shortname = "EX",
	extensions = { "ex" },
	comment = "//",

	patterns = {
		{ pattern = "%s*//.-\n", type = "comment", line_start = true },
		{ pattern = "-?%d+%.?%d*", type = "number" },
		{ pattern = "[%a_][%w_]*", type = "symbol" },
	},

	symbols = {
		["true"] = "literal",
		["false"] = "literal",
	},
})
```

You can copy this shape and change one part at a time.

## Adding an open handler from a plugin

A format may prefer a mode without importing the plugin that renders it:

```lua
default_mode = "preview"
```

The optional plugin registers the handler separately:

```lua
local open_handler = require("core.open_handler")

open_handler.register({
	name = "font",
	formats = { "otf", "ttf" },
	modes = { "preview" },
	create_view = function(request)
		return FontPreview(request.doc, request.options)
	end,
})
```

The format describes what the file is and which mode it prefers. The handler describes how to open it. If the preview handler is unavailable, *Lx* uses the ordinary read-only view.

Bundled handler names include `markdown`, `ansiart`, `font`, `wav`, `office`, `als`, `image`, `bytes`, and `hex`. A user may select one for a detected format through `config.format_handlers`.

## Naming the format

Use these entries to give the format a clear name:

- `name` is the full name shown to you, such as `Markdown Document`
- `shortname` is a compact name, such as `Markdown`
- `author` records who created the format
- `created` records when it was created
- `default_mode` may prefer `edit`, `view`, or `preview`

Only the name is important to everyday use. The mode is optional, while the author and date are notes for people maintaining the file.

## Choosing which files belong to it

*Lx* can recognize a format in several ways.

### File endings

Use `extensions` without the dot:

```lua
extensions = { "md", "markdown" }
```

This matches `notes.md` and `readme.markdown`. Letter case does not matter.

You may use one string when there is only one ending:

```lua
extensions = "lua"
```

### Exact file names

Use `filenames` for special names:

```lua
filenames = { "makefile", ".gitignore" }
```

The whole file name must match. Letter case does not matter.

### Filename patterns

Use `filename_patterns` when a family of final filenames belongs to the format:

```lua
filename_patterns = { "^license.*%.txt$" }
```

These are case-insensitive Lua patterns matched against the final filename only. They rank below exact filenames but above generic file endings, so a specific family such as `license*.txt` can beat an ordinary `.txt` format.

### File paths

Use `files` when the location or full name follows a pattern:

```lua
files = { "/%.config/.*%.conf$" }
```

These are Lua patterns. They are explained later in this guide.

### The beginning of a file

Use `headers` when the first part of the file identifies it:

```lua
headers = "^#!.-python"
```

You may provide one pattern or a list of patterns.

Three extra choices control how strongly the header matters:

- `require_header_match = true` accepts the file-name match only when the header also matches
- `require_file_match = true` prevents the header from selecting files whose names do not match this format
- `force_header_check = true` asks *Lx* to compare header matches again and prefer a stronger result

### Which match wins

*Lx* checks exact file names first, filename patterns second, file endings third, and file-path patterns fourth.

A required header can reject an otherwise correct-looking file name. A required file match stops a header from claiming unrelated filenames. A forced header check can let a more suitable format win afterward.

### The usual ending for a new file

Use `default_extension` for new, unsaved documents:

```lua
default_extension = "md"
```

If you leave it out, *Lx* uses the first value in `extensions`. If neither exists, *Lx* does not add an ending automatically.

## Looking inside a file before opening it

Some formats cannot be recognized safely from their name alone. A format may use `detect` to inspect the opening data and *Lx*'s basic byte probe.

The probe keeps its findings separate:

- `bom` describes a UTF BOM when present
- `utf8`, `utf16`, and `binary` each report `status` as `valid`, `invalid`, or `unsure`
- `utf8.invalid_offset` points to the first malformed byte when known
- `utf16.endianness` is `le` or `be` when UTF-16 is valid
- `binary.nul_count` and `binary.control_count` provide the basic binary evidence

The detector reads these under `ctx.probe`. An `unsure` status means the bounded probe did not have enough bytes to make that one finding final; it does not mean invalid data.

The detector receives a small description of the file. It can:

- Accept the format
- Reject it
- Suggest extra information, such as a text encoding
- Add a hint without becoming the selected format

This is useful for older DOS text. A detector can suggest `CP437`, while another format still supplies the syntax colors.

That means the chosen format and the chosen text encoding do not always come from the same place.

A format can keep looking for another match while carrying useful text information:

```lua
return {
	action = "continue",
	codepage = "cp437",
	force_text_open = true,
}
```

`action` controls detection and is not saved as information about the document. Public values such as `codepage` and `force_text_open` remain available to the opening path.

This is an advanced feature. Most format modules do not need it.

## Starting from another syntax

Closely related syntaxes often share rules. `inherit` lets one syntax borrow selected parts from another.

```lua
inherit = {
	syntax = "module:example",
	keys = { "patterns", "symbols" },
}
```

The source can be written in two ways:

- `module:example` means the syntax from the format module named `format.example`
- `.ex` means the syntax registered for the `ex` file ending

`keys` lists only the parts you want to borrow.

You may write `keys` as a normal list, or as names set to `true`:

```lua
keys = {
	patterns = true,
	symbols = true,
}
```

You may also borrow from several sources:

```lua
inherit = {
	{ syntax = "module:html", keys = { "patterns" } },
	{ syntax = ".css", keys = { "symbols" } },
}
```

Your own pattern rules stay first. Borrowed pattern rules follow as fallbacks. The same applies to outline rules.

Other tables are combined more deeply. Your own value wins when both sides provide the same entry.

Remove an inherited entry with:

```lua
some_key = format.REMOVE
```

## Coloring text with patterns

The `patterns` list contains the main coloring rules.

Each rule answers two questions:

1. What text should *Lx* find?
2. What kind of text is it?

```lua
patterns = {
	{ pattern = "%d+", type = "number" },
	{ pattern = "#.-\n", type = "comment" },
}
```

`pattern` describes the text. `type` names the color role.

*Lx* uses Lua patterns, not regular expressions. They look similar, but they are not the same language.

### Useful Lua pattern pieces

These are enough for many format modules:

- `%d` means a digit
- `%a` means a letter
- `%w` means a letter or digit
- `%s` means spacing
- `.` means any character
- `+` means one or more
- `*` means zero or more
- `-` means as few as possible
- `?` means optional
- `^` means the beginning
- `$` means the end
- `[abc]` means one character from the brackets
- `[^abc]` means anything except those characters
- `%(` means a real opening parenthesis

For example, `-?%d+%.?%d*` finds an optional minus sign, digits, and an optional decimal part.

### One pattern or an opening and closing pair

A rule may use one pattern:

```lua
{ pattern = "%d+", type = "number" }
```

It may also use an opening and closing pair:

```lua
{ pattern = { '"', '"', "\\" }, type = "string" }
```

The pair contains:

1. The opening pattern
2. The closing pattern
3. An optional escape pattern

Pair rules can continue across lines. *Lx* remembers that the opening part is still active until it finds the close.

### Coloring the whole match or separate pieces

A single `type` colors the whole match:

```lua
{ pattern = "%d+", type = "number" }
```

A list of types colors captured pieces separately:

```lua
{
	pattern = "(%d+)(:)(%s*)(off)",
	type = { "number", "operator", "normal", "error" },
}
```

Each pair of plain parentheses captures one piece. The first type colors the first capture, the second type colors the second capture, and so on.

Text outside the captures keeps the normal text color.

## Choosing between overlapping rules

Several rules can begin at the same character. `match.strategy` tells *Lx* which one wins.

```lua
match = {
	strategy = "longest",
}
```

The available choices are:

- `first` uses the first suitable rule in the list
- `longest` uses the rule that covers the most text
- `priority_first` uses the highest priority, then the earliest rule
- `priority_longest` uses the highest priority, then the longest match, then the earliest rule

`first` is the default.

Add a priority to an individual rule like this:

```lua
{ pattern = "important", type = "keyword", priority = 10 }
```

A larger number is stronger. Zero is the default. Negative priorities are allowed.

When lengths or priorities are equal, the earlier rule wins.

For a simple syntax, place specific rules first and broad fallback rules later. You often do not need a special strategy.

## Extra checks for a pattern rule

A rule can ask *Lx* to check the text around a possible match.

- `line_start = true` tests the rule only from the start of a line
- `next_line` requires the following line to match a pattern
- `line_prefix` requires everything before the match to fit a pattern
- `line_not_prefix` requires the line prefix not to fit a pattern
- `preceded_by` requires the text immediately before the match
- `not_preceded_by` rejects that preceding text
- `followed_by` requires the text immediately after the match
- `not_followed_by` rejects that following text
- `caseless = true` ignores letter case for literal letters in the pattern
- `priority` gives the rule strength when a priority strategy is used

These checks help two similar pieces of text receive different colors without making one enormous pattern.

### Stop a broad match early

`stop_on` cuts a broad one-part match before an inner terminator:

```lua
{
	pattern = "[^\n]+",
	type = "string",
	stop_on = "#",
}
```

`terminators` is another name for `stop_on`.

This works only with single-string patterns. It does not apply to opening and closing pair rules.

If the terminator appears at the first character, *Lx* discards that candidate so another rule can try.

## Coloring the beginning of a line first

`line_patterns` is a separate list checked once at the start of each line.

Use it when a line's opening structure decides how the rest should be read.

```lua
line_patterns = {
	{
		pattern = "([%a_][%w_]*)()%s*=()",
		type = { "symbol", "operator" },
		consume_to = 2,
	},
}
```

*Lx* follows this order:

1. Check `line_patterns`
2. Produce the winning line colors
3. Continue with normal `patterns`

Line rules use the same match strategy and priorities as normal rules.

By default, normal patterns may examine the same line again.

Use `consume_line = true` when the line rule should own the complete line.

Use `consume_to = N` when normal rules should resume at a saved position inside the match.

An empty capture written as `()` saves a position. `consume_to` counts only these position captures, not ordinary text captures.

In the example above, the second `()` tells *Lx* where normal coloring should continue.

Position captures can also divide a line rule into separately colored spans. Keep the number and order of those spans aligned with the entries in `type`.

*Lx* warns once and skips a bad line rule when:

- `consume_to` points to a missing position
- Saved positions are out of order
- The number of colored spans does not fit the type list

Empty spans are simply ignored.

`line_patterns` is deliberately limited:

- It does not use opening and closing pair patterns
- It does not hand text to another embedded syntax
- It does not remember state across lines

Use normal pair rules when ownership must continue onto later lines.

## Handing part of a file to another syntax

A pair rule can give its inner text to another syntax with `syntax`:

```lua
{
	pattern = { "<style>", "</style>" },
	type = "normal",
	syntax = ".css",
}
```

This is how one file can contain another kind of text, such as CSS inside HTML.

The value may be:

- A loaded syntax
- An extension such as `.js`
- A module name such as `module:js`

Other string forms are invalid. *Lx* reports them instead of silently guessing.

`syntax` works only on opening and closing pair rules.

`delim_type` can give the opening and closing delimiters their own color while the inside uses the embedded syntax.

### Giving a multiline pair a row background

Use `line_bg` on a pair rule when every covered row should use a named background color:

```lua
{
	pattern = { opening_fence, closing_fence },
	syntax = ".cpp",
	line_bg = "code",
}
```

The format module supplies only the syntax color name. The active style supplies the actual background through `style.syntax.code.bg`.

The rows inside the pair receive the background. The opening and closing rows do not. Empty and wrapped rows inside the pair are included. *Lx* caches the name while highlighting and joins adjacent visible rows into one rectangle while drawing.

You can turn off all backgrounds requested by format modules in your settings:

```lua
config.syntax_line_backgrounds = false
```

## Giving exact words special colors

Use `symbols` when complete words should receive a known color:

```lua
symbols = {
	["true"] = "literal",
	["false"] = "literal",
	["return"] = "keyword",
}
```

This is an exact lookup. `true` does not match the `true` part of `untrue`.

The normal word shape comes from `config.symbol_pattern`:

```lua
"[%a_][%w_]*"
```

That means a letter or underscore first, followed by letters, digits, or underscores.

### Adding letter-case versions automatically

Symbol expansion saves you from writing several letter-case versions by hand.

```lua
symbols = {
	_expansion_flags = "rlu",
	["Error"] = "error",
	["Debug"] = { "warning", "rluc" },
}
```

The letters in the flag string mean:

- `r` keeps the word exactly as written
- `l` adds a lowercase version
- `u` adds an uppercase version
- `c` adds a version with the first letter capitalized

`_expansion_flags` supplies the usual choice for the table. The default is `r`.

A symbol may provide its own flags with `{ "type", "flags" }`.

*Lx* expands these versions once when the format loads. An invalid flag is reported and safe defaults are used.

Borrowed symbol tables are combined after expansion. Your child syntax does not expand the parent's words a second time.

When two expanded entries produce the same word, the later entry wins.

## Recoloring words inside a larger match

Sometimes a broad match has one base color, but a few words inside it need special colors.

Use `post_symbols` on the pattern:

```lua
patterns = {
	{
		pattern = "%f[%S]%d+:%s*[^#\n]+",
		type = "string",
		post_symbols = "state_symbols",
	},
}

state_symbols = {
	["on"] = "success",
	["off"] = "error",
}
```

`post_symbols = true` uses the syntax's table named `post_symbols`.

`post_symbols = "state_symbols"` uses the named table instead.

The lookup still uses complete words. It does not match pieces inside larger words.

## Color names

The `type` value is a color role. *Lx* looks for that role in `style.syntax`.

Common roles include:

- `normal`
- `comment`
- `keyword`
- `keyword2`
- `number`
- `literal`
- `string`
- `operator`
- `function`

You may use a custom role:

```lua
{ pattern = "%u[%w_]*", type = "typename" }
```

Then a style can provide its color:

```lua
style.syntax.typename = { 220, 180, 120 }
```

If the current style does not define that role, *Lx* uses the normal text color.

## Describing comments

The `comment` entry tells line-comment commands what to insert:

```lua
comment = "//"
```

Some syntaxes use a table when they need more than one line-comment form.

### Block comments

Use `block_comment` for comments surrounding a larger section:

```lua
block_comment = {
	block_open = "/*",
	block_close = "*/",
}
```

Available choices are:

- `block_open` is the required opening text
- `block_close` is the required closing text
- `block_line_prefix` is added to each inner line
- `open_on_own_line = true` puts the opening text on a separate line
- `close_on_own_line = true` puts the closing text on a separate line
- `close_line_prefix` is placed before a close on its own line
- `strip_line_prefix` is a Lua pattern removed from inner lines when uncommenting
- `allow_nested = true` permits wrapping text that already contains the same markers

For a documentation-style block, `block_line_prefix = " * "` can place a star before every inner line.

### Guard comments

Guard comments wrap text with stronger boundary markers:

```lua
guard_comment = {
	guard_open = "#if 0",
	guard_close = "#endif",
}
```

Available choices are:

- `guard_open` is the opening marker or list of accepted markers
- `guard_close` is an optional closing marker
- `guard_body_prefix` is added to inner lines
- `guard_body_prefix = "auto"` uses one normal indentation step
- `guard_align_to_indent = true` follows the selected text's indentation
- `allow_nested = true` allows an existing guard inside the new one

When `guard_open` is a list, *Lx* inserts the first value and recognizes every value in the list.

### Custom comment behavior

`comment_hook` lets a format provide its own comment action when ordinary markers are not enough.

Markdown uses this kind of hook to cycle headings through several levels.

This is an advanced Lua function. The comment command passes the requested action and text to it, and the function returns the changed text.

## Filling the document outline

`outline_patterns` finds items for outline navigation.

```lua
outline_patterns = {
	{ pattern = "^#%s+(.+)", type = "symbol", outline_level = 1 },
}
```

The first non-empty text capture becomes the outline label. Position captures written as `()` are ignored because they contain a location rather than readable text. When a pattern has no non-empty text capture, *Lx* displays the complete source line as before.

`outline_level` describes the label's nesting level.

Outline rules use the same general pattern ideas as highlighting rules.

`outline_hook` can replace normal outline matching when the format needs custom Lua behavior.

## Providing formatting commands

`format_commands` connects general editor commands to the current format.

```lua
format_commands = {
	bold = apply_markdown_format,
	italic = apply_markdown_format,
	code = apply_markdown_format,
}
```

The available editor commands include:

- `format:bold`
- `format:italic`
- `format:underline`
- `format:strikethrough`
- `format:code`

A command appears only when the current format supplies its matching function.

The function receives the open document and the format name. It changes all suitable selections and reports whether anything changed.

Markdown uses this to keep combined formatting in one stable order. It can also remove an existing wrapper before rebuilding the result.

See [Formatting and surrounding text](format_shortcuts.md) for how you use these commands in the editor.

## Surrounding selected text

`surrounding_pairs` tells *Lx* what to place around selected text when you type a supported character.

The shortest form uses the typed character on the left and the supplied text on the right:

```lua
surrounding_pairs = {
	["("] = ")",
	["["] = "]",
}
```

Typing `(` around selected `hello` produces `(hello)`.

The full form gives you more control:

```lua
surrounding_pairs = {
	["~"] = {
		open = "~",
		close = "~",
		min_repeat = 2,
		max_repeat = 2,
		behavior = "stop",
	},
}
```

The entries mean:

- The table key is the character you type or pass as a preset
- `open` is placed on the left; it defaults to the table key
- `close` is required and is placed on the right
- `min_repeat` is the first valid number of markers; it defaults to one
- `max_repeat` is the largest allowed number; leaving it out means no limit
- `behavior` decides what happens after the maximum
- `resolve` can choose the opening and closing text from the current selection

The three behaviors are:

- `stop` leaves the text at the maximum
- `cycle` returns to `min_repeat`
- `unwrap` removes all matching markers

`behavior` needs a `max_repeat`. `max_repeat` cannot be smaller than `min_repeat`.

Markdown uses `min_repeat = 2` and `max_repeat = 2` for `~`. This prevents invalid single or triple tilde wrappers.

A resolver is a Lua function. It receives:

- The typed or preset value
- The selected text
- The document and current syntax
- The line and selection columns

It returns the opening and closing text to use. This is useful when the correct markers depend on the selection.

Auto Surround works only when every active selection contains text on one line. The whole operation is applied together, so one rejected selection does not leave the others half-changed.

See [Auto Surround](auto_surround.md) for the user commands and settings.

## Per-file viewing choices

A format can place initial document choices in `defaults`:

```lua
defaults = {
	word_wrap = false,
}
```

This lets the format start with a useful choice, such as disabling wrapping for line-based records.

The user can still change wrapping afterward.

## Keeping rules predictable

These habits make a syntax easier to understand:

1. Put narrow, specific rules before broad rules.
2. Use exact symbols for known complete words.
3. Use pair rules for text that can cross lines.
4. Use `line_patterns` only when the start of a line needs a special first pass.
5. Add priorities only when list order is not enough.
6. Test similar-looking text that should not match.
7. Keep comments and examples beside unusual Lua patterns.

*Lx* reports malformed patterns, invalid embedded syntax names, bad surrounding-pair settings, and invalid line-rule positions. The message names the syntax and the failing part whenever possible.

## A fuller example

This example brings several parts together:

```lua
local format = require("core.format")

format.register({
	name = "Example Document",
	shortname = "Example",
	extensions = { "ex", "example" },
	filenames = { "examplefile" },
	default_extension = "ex",
	comment = "//",

	match = {
		strategy = "priority_longest",
	},

	patterns = {
		{ pattern = "%s*//.-\n", type = "comment", line_start = true, priority = 10 },
		{ pattern = { '"', '"', "\\" }, type = "string" },
		{ pattern = "-?%d+%.?%d*", type = "number" },
		{ pattern = "[%a_][%w_]*", type = "symbol" },
	},

	symbols = {
		_expansion_flags = "rl",
		["TRUE"] = "literal",
		["FALSE"] = "literal",
	},

	outline_patterns = {
		{ pattern = "^section%s+([%w_]+)", type = "symbol", outline_level = 1 },
	},

	block_comment = {
		block_open = "/*",
		block_close = "*/",
	},

	surrounding_pairs = {
		["("] = ")",
		["*"] = { close = "*", max_repeat = 3, behavior = "cycle" },
	},

	defaults = {
		word_wrap = false,
	},
})
```

Read it from top to bottom:

- The names describe the format
- File endings and the exact filename identify matching files
- The match strategy settles overlapping color rules
- Patterns find comments, strings, numbers, and words
- Symbols recolor known words without caring about letter case
- Outline rules find named sections
- Comment metadata powers block-comment commands
- Surrounding pairs wrap selected text
- The file starts with long-line wrapping turned off

You can begin with the minimal example and add only the parts your format actually needs.
