9. Supported filters

Here are listed officially supported filters with the list of parameters they accept. Depending on compile options, some of these filters might be unavailable. The list of available filter classes, including the ones added by the addons built into the binary, is reported in haproxy -vv.

A filter is declared on a proxy in one of two modes:

  • The legacy mode relies on the “filter” directive. Each “filter” line declares an independent filter configuration and the declaration order defines the evaluation order. However, the filters implicitly declared by other directives (the decompression options, the compression options, the “cache-store” rules and “use-fcgi-app”) are automatically ordered, independently on each direction, and an explicitly declared filter is never moved: it keeps its declared position and the implicitly declared ones are ordered inside the segments it delimits (see the default orders below). This mode is kept for backward compatibility.

  • The new mode relies on the “filter-config” directive. Each declaration is a filter instance, attached to its filter class. The evaluation order is defined by the class placement and can be adjusted per proxy with the “filter-sequence” directive. The instances are disabled by default and are activated with the “enabled” token or the “filter-enable” directive.

For each mode, three orders exist: a global one, used for the side-independent operations (configuration checks, initialization and destruction), and a request and a response one, used for the evaluation order of the filters on the corresponding channel. The request and the response orders are independent, so a filter may be evaluated at a different position depending on the direction, and the global order may be changed without affecting them. The resulting orders of a proxy may be inspected with the “show filter classes” CLI command, and the “show filter instances” one reports the resulting order of the instances.

The default orders are informational only: they describe the current placement of the classes and of the implicitly declared filters, and they may be changed by the build options or by a future version. Only the placements explicitly requested by a filter, and the position of an implicitly declared legacy filter relative to the explicitly declared ones, should be relied upon.

In the new mode, the orders follow the class placement, which can be adjusted per proxy with the “filter-sequence” directive. The built-in classes are placed, when no explicit placement is requested, as follows:

  • global order: trace > decomp-req > decomp-res > cache-store > spoe > lua > http-comp-req > http-comp-res > bwlim-in > bwlim-out > fcgi

  • request order: trace > decomp-req > lua > spoe > http-comp-res > decomp-res > http-comp-req > bwlim-in > fcgi

  • response order: fcgi > trace > decomp-res > cache-store > spoe > lua > http-comp-res > decomp-req > bwlim-out

In the legacy mode, the automatically ordered filters are the implicit core filters: the decompression filters, the cache filters, the compression filters and the fcgi filter. They are placed, when implicitly declared, as follows, where “any other filter” stands for the explicitly declared filters and the other legacy filters in their declaration order:

  • global order: decomp-req > decomp-res > cache-store > any other filter > http-comp-req > http-comp-res > fcgi

  • request order: decomp-req > any other filter > http-comp-res > decomp-res > http-comp-req > fcgi

  • response order: fcgi > decomp-res > cache-store > any other filter > decomp-req > http-comp-res

The mode is selected by the configuration itself: as soon as a legacy “filter” directive is used in a proxy, the proxy works in legacy mode and the “filter-*” directives must not be used in the same proxy, including when they are inherited from a defaults section. Conversely, a legacy “filter” directive is rejected in a proxy inheriting filter instances from its defaults section. Filters implicitly declared by other directives (for instance by the “compression” options, the “cache-store” rules or “use-fcgi-app”) do not select a mode: a proxy using only such filters works in the new mode, unless the implied filters are only known as legacy filter configurations, without any filter class: a proxy implying only such filters works in the legacy mode, and a proxy implying filters of both kinds uses the new mode, where the class-less ones are ignored. All the filters documented in this section declare their classes, so they are always visible to the new mode. The “filter-api” global option changes this last rule: when it is set to “downgrade-to-legacy”, the proxies using only implicitly declared filters work in the legacy mode, whatever they imply, so the filters not supporting the new mode keep working. Explicit filter declarations always win, in both modes.

All the filters documented below support the new mode. The <class> name to use with “filter-config” may differ from the legacy filter name; it is given in the documentation of each filter. The “filter-config” syntax is documented below the legacy one.

The Lua filters are not documented here: they are registered by Lua scripts with core.register_filter() and are declared with the “lua” filter class. See the Lua API documentation for their declaration syntax.

The filter instances of a proxy may be inspected with the “show filter classes” and “show filter instances” CLI commands (see the management manual).

See also

“filter”, “filter-config”, “filter-enable”, “filter-disable” and

"filter-sequence".

9.1. Trace

filter trace [name <name>] [quiet] [random-parsing] [random-forwarding]

[max-fwd <max>] [hexdump]

filter-config trace id <name> [enabled] [quiet] [random-parsing]

[random-forwarding] [max-fwd <max>] [hexdump]

Arguments:

<name> is an arbitrary name that will be reported in messages. If no name is provided, “TRACE” is used. With “filter-config”, the instance id is mandatory and is used as the name.

<quiet> inhibits trace messages.

<random-parsing> enables the random parsing of data. By default, this filter parses all available data. With this parameter, it only parses a random amount of the available data.

<random-forwarding> enables the random forwarding of parsed data. By default, this filter forwards all previously parsed data. With this parameter, it only forwards a random amount of the parsed data.

<max> is the maximum amount of data that can be forwarded at a time. “max-fwd” option can be combined with the random forwarding. <max> must be an positive integer. 0 means there is no limit.

<hexdump> dumps all forwarded data to the server and the client.

This filter can be used as a base to develop new filters. It defines all callbacks and prints a message on the standard error stream (stderr) with useful information for all of them. It may be useful to debug the activity of other filters or, quite simply, HAProxy’s activity.

Using <random-parsing> and/or <random-forwarding> parameters is a good way to test the behavior of a filter that parses data exchanged between a client and a server by adding some latencies in the processing.

With the new mode, the filter class is “trace”. Several instances may be declared in the same proxy, each one with its own id.

9.2. HTTP compression

filter comp-req

Enables filter that explicitly tries to compress HTTP requests according to “compression” settings. Implicitly sets “compression direction request”.

filter comp-res

Enables filter that explicitly tries to compress HTTP responses according to “compression” settings. Implicitly sets “compression direction response”

filter compression (deprecated)

Alias for backward compatibility purposes that is functionally equivalent to enabling both “comp-req” and “comp-res” filter. “compression” keyword must be used to configure appropriate behavior:

filter-config http-comp-req
filter-config http-comp-res

Explicitly declare the request or the response compression filter with the new mode. These classes do not accept an id. Like their legacy counterparts “filter comp-req” and “filter comp-res”, they are configured with the “compression” directives. Declaring “filter-config http-comp-req” implicitly sets “compression direction request” and declaring “filter-config http-comp-res” implicitly sets “compression direction response”.

Note that the legacy mode rules above about implicit declarations do not apply in the new mode. The evaluation order of these filters relative to the other ones is defined by the class placement and can be adjusted with the “filter-sequence” directive.

The HTTP compression has been moved in a filter in HAProxy 1.7. “compression” keyword must still be used to enable and configure the HTTP compression. And when no other filter is used, it is enough. When used with the cache or the fcgi-app enabled, it is also enough. In this case, the compression is always done after the response is stored in the cache. But it is mandatory to explicitly use a filter line to enable the HTTP compression when at least one filter other than the cache or the fcgi-app is used for the same listener/frontend/backend. This is important to know the filters evaluation order.

See also

“compression”, section 9.5 about the cache filter and section 9.6

about the fcgi-app filter.

9.3. Decompression

filter decomp-req

Enables filter that explicitly tries to decompress requests according to “decompression” settings (setting algo is mandatory). It is only allowed on proxies with frontend capability. It supports both TCP and HTTP compressed streams. The filter runs during the stream forwarding phase, thus it will not be possible to perform TCP/HTTP rules on the decompressed stream (ie: req.body won’t work).

filter decomp-res

Enables filter that explicitly tries to decompress responses according to “decompression” settings (setting algo is mandatory). It is only allowed on proxies with backend capability. It supports both TCP and HTTP compressed streams. The filter runs during the stream forwarding phase, thus it will not be possible to perform TCP/HTTP rules on the decompressed stream (ie: res.body won’t work).

filter-config decomp-req

Explicitly declare the request decompression filter with the new mode. This class does not accept an id. Like its legacy counterparts “filter decomp-req”, it is configured with the “decompression” directives.

filter-config decomp-res

Explicitly declare the response decompression filter with the new mode. This class does not accept an id. Like its legacy counterparts “filter decomp-res”, it is configured with the “decompression” directives.

9.4. Stream Processing Offload Engine (SPOE)

filter spoe [engine <name>] config <file>

Enables the SPOE filter

Arguments:

<name> is the engine name that will be used to find the right scope in the configuration file. If not provided, all the file will be parsed.

<file> is the path of the engine configuration file. This file can contain configuration of several engines. In this case, each part must be placed in its own scope.

filter-config spoe id <id> [enabled] [engine <name>] config <file>

Declare a SPOE filter with the new mode. The instance id is mandatory, as the class allows several instances in the same proxy, but it is only used by the “filter-enable”, “filter-disable” and “filter-sequence” directives. The “engine” and “config” parameters are the same as for the legacy syntax.

The Stream Processing Offload Engine (SPOE) is a filter communicating with external components. It allows the offload of some specifics processing on the streams in tiered applications. These external components and information exchanged with them are configured in dedicated files, for the main part. It also requires dedicated backends, defined in HAProxy configuration.

SPOE communicates with external components using an in-house binary protocol, the Stream Processing Offload Protocol (SPOP).

When the SPOE is used on a stream, a dedicated stream is spawned to handle the communication with the external component. The main stream is the parent stream of this “SPOE” stream. It means it is possible to retrieve variables of the main stream from the “SPOE” stream. See section 2.8 about variables for details.

For all information about the SPOE configuration and the SPOP specification, see “doc/SPOE.txt”.

9.5. Cache

filter cache <name>

Enables the cache filter

Arguments:

<name> is name of the cache section this filter will use.

The cache uses a filter to store cacheable responses. The HTTP rules “cache-store” and “cache-use” must be used to define how and when to use a cache. By default the corresponding filter is implicitly defined. And when no other filters than fcgi-app or compression are used, it is enough. In such case, the compression filter is always evaluated after the cache filter. But it is mandatory to explicitly use a filter line to use a cache when at least one filter other than the compression or the fcgi-app is used for the same listener/frontend/backend. This is important to know the filters evaluation order.

filter-config cache-store id <name> [enabled]

Declare a cache filter with the new mode. The instance id is mandatory and is the name of the cache section, like the <name> parameter of the legacy syntax. The rules above about implicit declarations only apply to the legacy mode. In the new mode, a cache filter instance is implicitly created by the “cache-store” and “cache-use” rules when needed, and its evaluation order can be adjusted with the “filter-sequence” directive.

See also

section 9.2 about the compression filter, section 9.6 about the

fcgi-app filter and section 6 about cache.

9.6. Fcgi-app

filter fcgi-app <name>

Enables the fcgi filter

Arguments:

<name> is name of the fcgi-app section this filter will use.

The FastCGI application uses a filter to evaluate all custom parameters on the request path, and to process the headers on the response path. the <name> must reference an existing fcgi-app section. The directive “use-fcgi-app” should be used to define the application to use. By default the corresponding filter is implicitly defined. And when no other filters than cache or compression are used, it is enough. But it is mandatory to explicitly use a filter line to a fcgi-app when at least one filter other than the compression or the cache is used for the same backend. This is important to know the filters evaluation order.

filter-config fcgi id <name> [enabled]

Declare a fcgi-app filter with the new mode. The instance id is mandatory and is the name of the fcgi-app section, like the <name> parameter of the legacy syntax. The rules above about implicit declarations only apply to the legacy mode. In the new mode, a filter instance is implicitly created by “use-fcgi-app” when needed, and its evaluation order can be adjusted with the “filter-sequence” directive.

See also

“use-fcgi-app”, section 9.2 about the compression filter, section 9.5

about the cache filter and section 10 about FastCGI application.

9.7. OpenTracing

The OpenTracing filter was removed in version 3.5. A replacement filter based on OpenTelemetry is available since 3.4 with complete build instructions currently at:

https://github.com/haproxytech/haproxy-opentelemetry/

9.8. Bandwidth limitation

filter bwlim-in <name> default-limit <size> default-period <time> [min-size <sz>]
filter bwlim-out <name> default-limit <size> default-period <time> [min-size <sz>]
filter bwlim-in <name> limit <size> key <pattern> [table <table>] [min-size <sz>]
filter bwlim-out <name> limit <size> key <pattern> [table <table>] [min-size <sz>]

filter-config bwlim-in id <name> [enabled] default-limit <size> default-period <time> [min-size <sz>]

filter-config bwlim-out id <name> [enabled] default-limit <size> default-period <time> [min-size <sz>]

filter-config bwlim-in id <name> [enabled] limit <size> key <pattern> [table <table>] [min-size <sz>]

filter-config bwlim-out id <name> [enabled] limit <size> key <pattern> [table <table>] [min-size <sz>]

With the new mode, the instance id is mandatory and is the filter name used by the “set-bandwidth-limit” actions, like the <name> parameter of the legacy syntax. The other parameters are the same.

Arguments:

<name> is the filter name that will be used by ‘set-bandwidth-limit’ actions to reference a specific bandwidth limitation filter.

<size> is max number of bytes that can be forwarded over the period. The value must be specified for per-stream and shared bandwidth limitation filters. It follows the HAProxy size format and is expressed in bytes.

<pattern> is a sample expression rule as described in section 7.3. It describes what elements will be analyzed, extracted, combined, and used to select which table entry to update the counters. It must be specified for shared bandwidth limitation filters only.

<table> is an optional table to be used instead of the default one, which is the stick-table declared in the current proxy. It can be specified for shared bandwidth limitation filters only.

<time> is the default time period used to evaluate the bandwidth limitation rate. It can be specified for per-stream bandwidth limitation filters only. It follows the HAProxy time format and is expressed in milliseconds.

<min-size> is the optional minimum number of bytes forwarded at a time by a stream excluding the last packet that may be smaller. This value can be specified for per-stream and shared bandwidth limitation filters. It follows the HAProxy size format and is expressed in bytes.

Bandwidth limitation filters should be used to restrict the data forwarding speed at the stream level. By extension, such filters limit the network bandwidth consumed by a resource. Several bandwidth limitation filters can be used. For instance, it is possible to define a limit per source address to be sure a client will never consume all the network bandwidth, thereby penalizing other clients, and another one per stream to be able to fairly handle several connections for a given client.

The definition order of these filters is important. If several bandwidth filters are enabled on a stream, the filtering will be applied in their definition order. It is also important to understand the definition order of the other filters have an influence. For instance, depending on the HTTP compression filter is defined before or after a bandwidth limitation filter, the limit will be applied on the compressed payload or not. The same is true for the cache filter.

There are two kinds of bandwidth limitation filters. The first one enforces a default limit and is applied per stream. The second one uses a stickiness table to enforce a limit equally divided between all streams sharing the same entry in the table.

In addition, for a given filter, depending on the filter keyword used, the limitation can be applied on incoming data, received from the client and forwarded to a server, or on outgoing data, received from a server and sent to the client. To apply a limit on incoming data, “bwlim-in” keyword must be used. To apply it on outgoing data, “bwlim-out” keyword must be used. In both cases, the bandwidth limitation is applied on forwarded data, at the stream level.

The bandwidth limitation is applied at the stream level and not at the connection level. For multiplexed protocols (H2, H3 and FastCGI), the streams of the same connection may have different limits.

For a per-stream bandwidth limitation filter, default period and limit must be defined. As their names suggest, they are the default values used to setup the bandwidth limitation rate for a stream. However, for this kind of filter and only this one, it is possible to redefine these values using sample expressions when the filter is enabled with a TCP/HTTP “set-bandwidth-limit” action.

For a shared bandwidth limitation filter, depending on whether it is applied on incoming or outgoing data, the stickiness table used must store the corresponding bytes rate information. “bytes_in_rate(<period>)” counter must be stored to limit incoming data and “bytes_out_rate(<period>)” counter must be used to limit outgoing data.

Finally, it is possible to set the minimum number of bytes that a bandwidth limitation filter can forward at a time for a given stream. It should be used to not forward too small amount of data, to reduce the CPU usage. It must carefully be defined. Too small, a value can increase the CPU usage. Too high, it can increase the latency. It is also highly linked to the defined bandwidth limit. If it is too close to the bandwidth limit, some pauses may be experienced to not exceed the limit because too many bytes will be consumed at a time. It is highly dependent on the filter configuration. A good idea is to start with something around 2 TCP MSS, typically 2896 bytes, and tune it after some experimentations.

frontend http
    bind *:80
    mode http

    # If this filter is enabled, the stream will share the download limit
    # of 10m/s with all other streams with the same source address.
    filter bwlim-out limit-by-src key src table limit-by-src limit 10m

    # If this filter is enabled, the stream will be limited to download at 1m/s,
    # independently of all other streams.
    filter bwlim-out limit-by-strm default-limit 1m default-period 1s

    # Limit all streams to 1m/s (the default limit) and those accessing the
    # internal API to 100k/s. Limit each source address to 10m/s. The shared
    # limit is applied first. Both are limiting the download rate.
    http-request set-bandwidth-limit limit-by-strm
    http-request set-bandwidth-limit limit-by-strm limit 100k if { path_beg /internal }
    http-request set-bandwidth-limit limit-by-src
    ...

backend limit-by-src
    # The stickiness table used by <limit-by-src> filter
    stick-table type ip size 1m expire 3600s store bytes_out_rate(1s)

See also

“tcp-request content set-bandwidth-limit”,

"tcp-response content set-bandwidth-limit",
"http-request set-bandwidth-limit" and
"http-response set-bandwidth-limit".