Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ To make alterations to the deployment you can use the `bosh deploy [-o operator-
## Documentation

- [External Certificates](/docs/external_certs.md) - Using HAProxy with additional external certificates
- [Explicit Dual-Stack Bindings](docs/dual-stack.md)
- [Mutual TLS](/docs/mutual_tls.md) - Mutual TLS configuration
- [Rate Limiting](/docs/rate_limiting.md) - Client IP based rate limiting
- [Keepalived](/docs/keepalived.md) - Keepalived integration for high availability
Expand Down
126 changes: 126 additions & 0 deletions docs/dual-stack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Dual-Stack (IPv4 + IPv6) Configuration

Use `ha_proxy.bindings` to configure the sockets used by every generated
listener. This includes HTTP, HTTPS, WSS, HTTP health checks, CF TCP routing,
generic TCP frontends, and TCP health checks. The stats listener continues to
use `ha_proxy.stats_bind`.

## Explicit bindings

Each entry has an `address`. Addresses are IP literals, not hostnames, CIDRs,
ports, bracketed IPv6 literals, or zone IDs. Input order is retained in the
generated `bind` directives.

### IPv4 only

```yaml
ha_proxy:
bindings:
- address: 0.0.0.0
```

IPv4 is inherently IPv4-only, so it has no `ipv6_mode` option:

```
bind 0.0.0.0:443 ssl crt /var/vcap/jobs/haproxy/config/ssl
```

### IPv6 wildcard

The IPv6 wildcard must explicitly state how it handles IPv4:

```yaml
ha_proxy:
bindings:
- address: "::"
ipv6_mode: v6only
```

This produces an IPv6-only socket:

```
bind [::]:443 v6only ssl crt /var/vcap/jobs/haproxy/config/ssl
```

For a single dual-stack socket, use `v4v6` instead:

```yaml
ha_proxy:
bindings:
- address: "::"
ipv6_mode: v4v6
```

```
bind [::]:443 v4v6 ssl crt /var/vcap/jobs/haproxy/config/ssl
```

Only wildcard `::` accepts `ipv6_mode`, because it can be an IPv4 wildcard as
well. A specific IPv6 address cannot act as an IPv4 wildcard and therefore has
no mode:

```yaml
ha_proxy:
bindings:
- address: "fd00::12"
```

### Separate IPv4 and IPv6 sockets

Use an IPv4 wildcard and an IPv6-only wildcard when separate sockets are
required:

```yaml
ha_proxy:
bindings:
- address: 0.0.0.0
- address: "::"
ipv6_mode: v6only
```

All listener-specific options, including TLS, ALPN, and PROXY protocol, are
rendered on every generated bind directive.

## Validation and compatibility

`bindings` is a non-empty array of maps. Duplicate normalized addresses are
rejected, as are wildcard/specific combinations in the same address family.
An IPv6 wildcard with `ipv6_mode: v4v6` cannot be combined with an IPv4
binding. `bindings` cannot be used with a non-empty legacy `binding_ip` or
with `v4v6: true`; set those legacy values to their defaults (`""` and
`false`) before migrating.

When `bindings` is absent, the existing `binding_ip` and `v4v6` behavior is
unchanged, including its historical `if_p("ha_proxy.v4v6")` handling. Because
`v4v6` has a false default, that legacy presence-based block runs when
`v4v6` is false or omitted. As a result, legacy `binding_ip: "::"` currently
renders `v4v6` regardless of whether `v4v6` is true, false, or omitted. This
compatibility quirk is intentionally preserved. Use `bindings` with
`ipv6_mode: v6only` or `ipv6_mode: v4v6` to explicitly select the IPv6/IPv4
behavior.

## Migrating from `frontend_config`

If `frontend_config` was used to add a second bind directive, replace the
manual configuration with explicit bindings. This avoids duplicating TLS and
other listener options and applies the same sockets to all generated listeners.

Before:

```yaml
ha_proxy:
binding_ip: 0.0.0.0
disable_http: true
frontend_config: |
bind :::443 ssl crt /var/vcap/jobs/haproxy/config/ssl
```

After:

```yaml
ha_proxy:
bindings:
- address: 0.0.0.0
- address: "::"
ipv6_mode: v6only
```
15 changes: 13 additions & 2 deletions jobs/haproxy/spec
Original file line number Diff line number Diff line change
Expand Up @@ -582,11 +582,22 @@ properties:
ha_proxy.enable_additional_health_check_proxy:
description: "Enable the additional health check listener with use of the PROXY protocol"
ha_proxy.binding_ip:
description: "If there are multiple ethernet interfaces, specify which one to bind. Set to `::` to bind to all IPv6 interfaces (no IPv4). IPv6 must be enabled on the HAProxy VM in the deployment manifest."
description: "If there are multiple ethernet interfaces, specify which one to bind. Set to `::` to bind to all IPv6 interfaces. For compatibility, the legacy renderer adds `v4v6` whenever this is `::`, even if ha_proxy.v4v6 is false or omitted; this presence-based behavior is intentionally preserved. Use ha_proxy.bindings to explicitly select `v6only` or `v4v6`. IPv6 must be enabled on the HAProxy VM in the deployment manifest."
default: ""
ha_proxy.v4v6:
description: "Boolean, disabled by default. Enables binding to all IPv4 and IPv6 interfaces. Only applies if `ha_proxy.binding_ip` is set to `::`."
description: "Legacy compatibility property, disabled by default. The legacy renderer still adds `v4v6` when ha_proxy.binding_ip is `::` even if this is false or omitted, because of its historical presence-based behavior. This behavior is intentionally preserved; use ha_proxy.bindings with ipv6_mode to explicitly select `v6only` or `v4v6`."
default: false
ha_proxy.bindings:
description: |
Optional explicit list of bind sockets for all generated listeners. Each entry
requires address. IPv4 addresses have no mode. The IPv6 wildcard :: requires
ipv6_mode, either v6only or v4v6; specific IPv6 addresses have no mode.
Cannot be combined with ha_proxy.v4v6: true or a legacy ha_proxy.binding_ip
other than nil or an empty string.
example:
- address: 0.0.0.0
- address: "::"
ipv6_mode: v6only

ha_proxy.cidr_blacklist:
description: "List of CIDRs to block for http(s). Format is string array of CIDRs or single string of base64 encoded gzip."
Expand Down
123 changes: 112 additions & 11 deletions jobs/haproxy/templates/haproxy.config.erb
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,18 @@
end
end

def render_bind(bindings, binding_ip, port, extra_options)
if bindings
bindings.map do |binding|
opts = [binding[:ipv6_mode], *Array(extra_options)].compact.reject(&:empty?).join(' ')
" bind #{binding[:address]}:#{port} #{opts}".rstrip
end.join("\n")
else
options = [*Array(extra_options)].reject(&:empty?).join(' ')
" bind #{binding_ip}:#{port} #{options}".rstrip
end
end

if properties.ha_proxy.config_mode == "auto"
if properties.ha_proxy.raw_config -%>
<%= p("ha_proxy.raw_config") %>
Expand Down Expand Up @@ -173,7 +185,7 @@ if ![true, false, "true", "false", "mtls_only"].include?(disable_domain_fronting
end

# }}}
# IPv4 and IPv6 binding (v4v6) Option {{{
# Legacy IPv4 and IPv6 binding (v4v6) Option {{{
v4v6 = ""
if_p("ha_proxy.v4v6") do
if p("ha_proxy.binding_ip") == "::"
Expand All @@ -182,6 +194,95 @@ if_p("ha_proxy.v4v6") do
end
# }}}

# Explicit Bindings Option {{{
require 'ipaddr'
bindings = nil
if_p("ha_proxy.bindings") do |configured_bindings|
unless configured_bindings.is_a?(Array) && !configured_bindings.empty?
abort("ha_proxy.bindings must be a non-empty array of maps")
end
if p("ha_proxy.v4v6") == true
abort("Conflicting configuration: ha_proxy.bindings cannot be combined with ha_proxy.v4v6: true")
end
legacy_binding_ip = p("ha_proxy.binding_ip")
unless legacy_binding_ip.nil? || (legacy_binding_ip.is_a?(String) && legacy_binding_ip.empty?)
abort("Conflicting configuration: ha_proxy.bindings requires ha_proxy.binding_ip to be nil or an empty string")
end

seen = {}
has_ipv4_wildcard = false
has_ipv4_specific = false
has_ipv6_wildcard = false
has_ipv6_specific = false

bindings = configured_bindings.map do |binding|
unless binding.is_a?(Hash)
abort("ha_proxy.bindings: each entry must be a map, got: #{binding.inspect}")
end

keys = binding.keys
unless keys.include?("address") && (keys - %w[address ipv6_mode]).empty?
abort("ha_proxy.bindings: each entry requires address and only permits ipv6_mode where applicable")
end

address = binding["address"]
ipv6_mode_provided = binding.key?("ipv6_mode")
ipv6_mode = binding["ipv6_mode"]

if address.to_s =~ %r{[/%]}
abort("ha_proxy.bindings: address must be a plain IP literal (no CIDR or zone ID): #{address.inspect}")
end

begin
parsed = IPAddr.new(address)
rescue IPAddr::InvalidAddressError
abort("ha_proxy.bindings: invalid IP address: #{address.inspect}")
end

if parsed.ipv6? && parsed.ipv4_mapped?
abort("ha_proxy.bindings: IPv4-mapped IPv6 addresses are not supported: #{address.inspect}")
end

normalized = parsed.to_s
if seen[normalized]
abort("ha_proxy.bindings: duplicate address: #{address}")
end
seen[normalized] = true

if parsed.ipv6?
if normalized == "::"
unless ipv6_mode_provided && %w[v6only v4v6].include?(ipv6_mode)
abort("ha_proxy.bindings: IPv6 wildcard :: requires ipv6_mode v6only or v4v6")
end
has_ipv6_wildcard = true
else
abort("ha_proxy.bindings: specific IPv6 addresses cannot specify ipv6_mode") if ipv6_mode_provided
has_ipv6_specific = true
end
{ address: "[#{normalized}]", ipv6_mode: ipv6_mode }
else
abort("ha_proxy.bindings: IPv4 addresses cannot specify ipv6_mode") if ipv6_mode_provided
if normalized == "0.0.0.0"
has_ipv4_wildcard = true
else
has_ipv4_specific = true
end
{ address: normalized, ipv6_mode: nil }
end
end

if has_ipv4_wildcard && has_ipv4_specific
abort("ha_proxy.bindings: IPv4 wildcard 0.0.0.0 cannot be combined with a specific IPv4 address")
end
if has_ipv6_wildcard && has_ipv6_specific
abort("ha_proxy.bindings: IPv6 wildcard :: cannot be combined with a specific IPv6 address")
end
if bindings.any? { |binding| binding[:ipv6_mode] == "v4v6" } && bindings.any? { |binding| binding[:ipv6_mode].nil? }
abort("ha_proxy.bindings: IPv6 wildcard :: with ipv6_mode v4v6 cannot be combined with an IPv4 binding")
end
end
# }}}

# ALPN Option {{{
default_alpn_config = ""
if p("ha_proxy.enable_http2")
Expand Down Expand Up @@ -398,7 +499,7 @@ listen stats

<% if p("ha_proxy.enable_health_check_http") %>
listen health_check_http_url
bind <%= p("ha_proxy.binding_ip") %>:<%= p("ha_proxy.health_check_port") %> <%= v4v6 %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), p("ha_proxy.health_check_port"), [v4v6]) %>
mode http
option httpclose
monitor-uri /health
Expand All @@ -411,7 +512,7 @@ listen health_check_http_url

<%- if enable_additional_health_check_proxy -%>
listen health_check_http_url_proxy_protocol
bind <%= p("ha_proxy.binding_ip") %>:<%= p("ha_proxy.health_check_port") + 1 %> accept-proxy <%= v4v6 %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), p("ha_proxy.health_check_port") + 1, ["accept-proxy", v4v6]) %>
mode http
option httpclose
monitor-uri /health
Expand Down Expand Up @@ -444,7 +545,7 @@ backend st_tcp_conn_rate
# HTTP Frontend {{{
frontend http-in
mode http
bind <%= p("ha_proxy.binding_ip") %>:80 <%= accept_proxy %> <%= v4v6 %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), 80, [accept_proxy, v4v6]) %>
<%- if properties.ha_proxy.frontend_config -%>
<%= format_indented_multiline_config(p("ha_proxy.frontend_config")) %>
<%- end -%>
Expand Down Expand Up @@ -558,7 +659,7 @@ frontend http-in
# HTTPS Frontend {{{
frontend https-in
mode http
bind <%= p("ha_proxy.binding_ip") %>:443 <%= accept_proxy %> <%= tls_bind_options %> <%= v4v6 %> <%= default_alpn_config %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), 443, [accept_proxy, tls_bind_options, default_alpn_config, v4v6]) %>
# Set this acl when the request is a route service request, used by ha_proxy.forward_true_client_ip_header and ha_proxy.forwarded_client_cert
acl route_service_request hdr(X-Cf-Proxy-Signature) -m found
<%- if disable_domain_fronting -%>
Expand Down Expand Up @@ -761,7 +862,7 @@ frontend https-in
# HTTPS Websockets Frontend {{{
frontend wss-in
mode http
bind <%= p("ha_proxy.binding_ip") %>:4443 <%= accept_proxy %> <%= tls_bind_options %> <%= v4v6 %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), 4443, [accept_proxy, tls_bind_options, v4v6]) %>
<%- if disable_domain_fronting -%>
# Check whether the client is attempting domain fronting.
# Ensure a host header exists to check against
Expand Down Expand Up @@ -1030,7 +1131,7 @@ backend http-routed-backend-<%= prefix_hash %>

frontend cf_tcp_routing
mode tcp
bind <%= p("ha_proxy.binding_ip") %>:<%= p("ha_proxy.tcp_routing.port_range") %> <%= v4v6 %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), p("ha_proxy.tcp_routing.port_range"), [v4v6]) %>
default_backend cf_tcp_routers

backend cf_tcp_routers
Expand Down Expand Up @@ -1061,9 +1162,9 @@ end -%>
frontend tcp-frontend_<%= tcp_proxy["name"]%>
mode tcp
<%- if tcp_proxy["ssl"] -%>
bind <%= p("ha_proxy.binding_ip") %>:<%= tcp_proxy["port"] %> <%= tcp_accept_proxy %> <%= tls_bind_options %> <%= v4v6 %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), tcp_proxy["port"], [tcp_accept_proxy, tls_bind_options, v4v6]) %>
<%- else -%>
bind <%= p("ha_proxy.binding_ip") %>:<%= tcp_proxy["port"] %> <%= tcp_accept_proxy %> <%= v4v6 %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), tcp_proxy["port"], [tcp_accept_proxy, v4v6]) %>
<%- end -%>
<%- if p("ha_proxy.expect_proxy_cidrs", []).size > 0 -%>
tcp-request connection expect-proxy layer4 if { src -f /var/vcap/jobs/haproxy/config/expect_proxy_cidrs.txt }
Expand Down Expand Up @@ -1116,7 +1217,7 @@ backend tcp-<%= tcp_proxy["name"] %>

<%- if tcp_proxy["health_check_http"] -%>
listen health_check_http_tcp-<%= tcp_proxy["name"] %>
bind <%= p("ha_proxy.binding_ip") %>:<%= tcp_proxy["health_check_http"] %> <%= v4v6 %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), tcp_proxy["health_check_http"], [v4v6]) %>
mode http
monitor-uri /health
<%- if p("ha_proxy.accept_proxy") && !p("ha_proxy.disable_health_check_proxy") -%>
Expand All @@ -1127,7 +1228,7 @@ listen health_check_http_tcp-<%= tcp_proxy["name"] %>

<%- if enable_additional_health_check_proxy -%>
listen health_check_http_tcp-<%= tcp_proxy["name"] %>_proxy_protocol
bind <%= p("ha_proxy.binding_ip") %>:<%= tcp_proxy["health_check_http"] + 1 %> accept-proxy <%= v4v6 %>
<%= render_bind(bindings, p("ha_proxy.binding_ip"), tcp_proxy["health_check_http"] + 1, ["accept-proxy", v4v6]) %>
mode http
monitor-uri /health
acl tcp-<%= tcp_proxy["name"] %>-routers_down nbsrv(tcp-<%= tcp_proxy["name"] %>) eq 0
Expand Down
Loading