Skip to content

Commit bd9be14

Browse files
authored
[libbeat] LDAP processor improvements (#47827)
1 parent 061db97 commit bd9be14

21 files changed

+1601
-153
lines changed
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# REQUIRED
2+
# Kind can be one of:
3+
# - breaking-change: a change to previously-documented behavior
4+
# - deprecation: functionality that is being removed in a later release
5+
# - bug-fix: fixes a problem in a previous version
6+
# - enhancement: extends functionality but does not break or fix existing behavior
7+
# - feature: new functionality
8+
# - known-issue: problems that we are aware of in a given version
9+
# - security: impacts on the security of a product or a user’s deployment.
10+
# - upgrade: important information for someone upgrading from a prior version
11+
# - other: does not fit into any of the other categories
12+
kind: enhancement
13+
14+
# REQUIRED for all kinds
15+
# Change summary; a 80ish characters long description of the change.
16+
summary: Add GUID translation, base DN inference, and SSPI authentication to LDAP processor.
17+
18+
# REQUIRED for breaking-change, deprecation, known-issue
19+
# Long description; in case the summary is not enough to describe the change
20+
# this field accommodate a description without length limits.
21+
# description:
22+
23+
# REQUIRED for breaking-change, deprecation, known-issue
24+
# impact:
25+
26+
# REQUIRED for breaking-change, deprecation, known-issue
27+
# action:
28+
29+
# REQUIRED for all kinds
30+
# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
31+
component: all
32+
33+
# AUTOMATED
34+
# OPTIONAL to manually add other PR URLs
35+
# PR URL: A link the PR that added the changeset.
36+
# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
37+
# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
38+
# Please provide it if you are adding a fragment for a different PR.
39+
pr: https://github.com/elastic/beats/pull/47827
40+
41+
# AUTOMATED
42+
# OPTIONAL to manually add other issue URLs
43+
# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
44+
# If not present is automatically filled by the tooling with the issue linked to the PR number.
45+
# issue: https://github.com/owner/repo/1234

docs/reference/auditbeat/processor-translate-guid.md

Lines changed: 36 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -6,27 +6,27 @@ applies_to:
66
stack: ga
77
---
88

9-
# Translate GUID [processor-translate-guid]
9+
# Translate LDAP Attribute [processor-translate-ldap-attribute]
1010

11+
The `translate_ldap_attribute` processor translates LDAP attributes into friendlier values. The typical use case is converting an Active Directory Global Unique Identifier (GUID) into a human-readable name (for example the object's `cn`).
1112

12-
The `translate_ldap_attribute` processor translates an LDAP attributes between eachother. It is typically used to translate AD Global Unique Identifiers (GUID) into their common names.
13+
Every object on an Active Directory or an LDAP server is issued a GUID. Internal processes refer to their GUID’s rather than the object's name and these values sometimes appear in logs.
1314

14-
Every object on an Active Directory or an LDAP server is issued a GUID. Internal processes refer to their GUID’s rather than the object’s name and these values sometimes appear in logs.
15-
16-
If the search attribute is invalid (malformed) or does not map to any object on the domain then this will result in the processor returning an error unless `ignore_failure` is set.
15+
If the search attribute is invalid (malformed) or does not map to any object on the domain the processor returns an error unless `ignore_failure` is set.
1716

1817
The result of this operation is an array of values, given that a single attribute can hold multiple values.
1918

20-
Note: the search attribute is expected to map to a single object. If it doesn’t, no error will be returned, but only results of the first entry will be added to the event.
19+
The search attribute is expected to map to a single object. If multiple entries match, only the first entry's mapped attribute values are returned.
2120

2221
```yaml
2322
processors:
2423
- translate_ldap_attribute:
2524
field: winlog.event_data.ObjectGuid
26-
ldap_address: "ldap://"
27-
ldap_base_dn: "dc=example,dc=com"
2825
ignore_missing: true
2926
ignore_failure: true
27+
# ldap_domain: "example.com" # Optional - override the OS-discovered domain used for SRV/LOGONSERVER hints
28+
# ldap_address: "ldap://ds.example.com:389" # Optional - Beats discovers controllers when omitted
29+
# ldap_base_dn: "dc=example,dc=com" # Optional - otherwise rootDSE and hostname inference are used
3030
```
3131

3232
The `translate_ldap_attribute` processor has the following configuration settings:
@@ -35,18 +35,40 @@ The `translate_ldap_attribute` processor has the following configuration setting
3535
| --- | --- | --- | --- |
3636
| `field` | yes | | Source field containing a GUID. |
3737
| `target_field` | no | | Target field for the mapped attribute value. If not set it will be replaced in place. |
38-
| `ldap_address` | yes | | LDAP server address. eg: `ldap://ds.example.com:389` |
39-
| `ldap_base_dn` | yes | | LDAP base DN. eg: `dc=example,dc=com` |
40-
| `ldap_bind_user` | no | | LDAP user. |
41-
| `ldap_bind_password` | no | | LDAP password. |
38+
| `ldap_domain` | no | | {applies_to}`stack: ga 9.2.4` DNS domain name (for example, `example.com`) used for DNS SRV discovery and to construct FQDNs from `LOGONSERVER`. When omitted Beats inspects OS metadata to infer the domain (Windows: `USERDNSDOMAIN`, `GetComputerNameEx`, TCP/IP + Kerberos registry keys, hostname; Linux/macOS: `/etc/resolv.conf`, `/etc/krb5.conf`, hostname). |
39+
| `ldap_address` | {applies_to}`stack: ga 9.2.4` no<br>{applies_to}`stack: ga 9.0.0` yes | | LDAP server address (for example, `ldap://ds.example.com:389`). When omitted Beats auto-discovers controllers by querying `_ldaps._tcp.<domain>` first, `_ldap._tcp.<domain>` second, and finally the Windows `LOGONSERVER` variable if available. Candidates are tried in order until one succeeds. |
40+
| `ldap_base_dn` | {applies_to}`stack: ga 9.2.4` no<br>{applies_to}`stack: ga 9.0.0` yes | | LDAP base DN (for example, `dc=example,dc=com`). When omitted Beats queries the server's rootDSE for `defaultNamingContext`/`namingContexts`. If the controller does not expose those attributes, client initialization fails and you must configure the value manually. |
41+
| `ldap_bind_user` | no | | LDAP DN/UPN for simple bind. When provided with `ldap_bind_password` Beats performs a standard bind. When set without a password Beats issues an unauthenticated bind using this identity (useful for servers that expect a bind DN even for anonymous operations). |
42+
| `ldap_bind_password` | no | | LDAP password for simple bind. When both the username and password are omitted Beats attempts automatic authentication: on Windows it first tries SSPI with the Beat's service or user identity using the SPN `ldap/<hostname derived from ldap_address>` and falls back to an unauthenticated bind if that fails. Non-Windows platforms immediately use an unauthenticated bind. |
4243
| `ldap_search_attribute` | yes | `objectGUID` | LDAP attribute to search by. |
4344
| `ldap_mapped_attribute` | yes | `cn` | LDAP attribute to map to. |
4445
| `ldap_search_time_limit` | no | 30 | LDAP search time limit in seconds. |
45-
| `ldap_ssl`\* | no | 30 | LDAP TLS/SSL connection settings. |
46+
| `ldap_ssl` | no | {applies_to}`stack: ga 9.2.4` no default<br>{applies_to}`stack: ga 9.0.0` `30` | LDAP TLS/SSL connection settings. Refer to [SSL](/reference/auditbeat/configuration-ssl.md). |
47+
| `ad_guid_translation` | no | `auto` | {applies_to}`stack: ga 9.2.4` Controls GUID binary conversion for Active Directory attributes. `auto` (default) converts when the LDAP search attribute equals `objectGUID` (case-insensitive). Use `always` to force conversion or `never` to disable it. |
4648
| `ignore_missing` | no | false | Ignore errors when the source field is missing. |
4749
| `ignore_failure` | no | false | Ignore all errors produced by the processor. |
4850

49-
\* Also see [SSL](/reference/auditbeat/configuration-ssl.md) for a full description of the `ldap_ssl` options.
51+
## Authentication flow
52+
53+
Beats attempts LDAP authentication in the following order:
54+
55+
1. Simple bind using `ldap_bind_user` and `ldap_bind_password` when both are supplied.
56+
2. Automatic bind when both values are empty. On Windows Beats creates an SSPI (Kerberos/NTLM) client for the SPN `ldap/<hostname derived from ldap_address>`, which works for Local System, domain-joined services, and gMSA accounts. Other platforms do not yet implement automatic authentication.
57+
3. If automatic authentication is unavailable or fails, Beats issues an unauthenticated bind. When `ldap_bind_user` is set without a password that identity is used; otherwise Beats binds anonymously.
58+
59+
Always prefer specifying `ldap_address` as an FQDN (for example `ldap://dc1.example.com:389`) so the SPN built for SSPI matches the controller's service principal and TLS certificates.
60+
61+
## Server auto-discovery
62+
63+
When `ldap_address` is omitted Beats resolves controllers dynamically:
64+
65+
1. **Domain discovery.** Beats determines the DNS domain from `ldap_domain` (if set) or OS metadata. Windows checks `USERDNSDOMAIN`, `GetComputerNameEx`, the TCP/IP and Kerberos registry keys, and the machine's FQDN. Linux/macOS read `/etc/resolv.conf`, `/etc/krb5.conf`, and the hostname suffix. If no domain is available SRV lookups are skipped.
66+
2. **DNS SRV queries.** When a domain is known Beats queries `_ldaps._tcp.<domain>` first and `_ldap._tcp.<domain>` second using the system resolver. Results are sorted by priority/weight per RFC 2782 and converted to `ldaps://host:port` or `ldap://host:port` URLs.
67+
3. **Windows LOGONSERVER fallback.** If SRV queries return no controllers or no domain was discovered, Beats reads the `LOGONSERVER` environment variable. When a domain is known the NetBIOS name is combined with it to build an FQDN so TLS validation and SSPI SPNs remain valid.
68+
69+
Each candidate address is attempted in order (LDAPS before LDAP) until a connection and bind succeed.
70+
71+
When `ldap_base_dn` is empty the client queries the controller's rootDSE for `defaultNamingContext` or the first non-system `namingContexts` entry. If neither is present Beats cannot continue and you must provide `ldap_base_dn` explicitly.
5072

5173
If the searches are slow or you expect a high amount of different key attributes to be found, consider using a cache processor to speed processing:
5274

@@ -78,4 +100,3 @@ processors:
78100
key_field: winlog.event_data.ObjectGuid
79101
value_field: winlog.common_name
80102
```
81-

docs/reference/filebeat/processor-translate-guid.md

Lines changed: 36 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -6,27 +6,27 @@ applies_to:
66
stack: ga
77
---
88

9-
# Translate GUID [processor-translate-guid]
9+
# Translate LDAP Attribute [processor-translate-ldap-attribute]
1010

11+
The `translate_ldap_attribute` processor translates LDAP attributes into friendlier values. The typical use case is converting an Active Directory Global Unique Identifier (GUID) into a human-readable name (for example the object's `cn`).
1112

12-
The `translate_ldap_attribute` processor translates an LDAP attributes between eachother. It is typically used to translate AD Global Unique Identifiers (GUID) into their common names.
13+
Every object on an Active Directory or an LDAP server is issued a GUID. Internal processes refer to their GUID’s rather than the object's name and these values sometimes appear in logs.
1314

14-
Every object on an Active Directory or an LDAP server is issued a GUID. Internal processes refer to their GUID’s rather than the object’s name and these values sometimes appear in logs.
15-
16-
If the search attribute is invalid (malformed) or does not map to any object on the domain then this will result in the processor returning an error unless `ignore_failure` is set.
15+
If the search attribute is invalid (malformed) or does not map to any object on the domain the processor returns an error unless `ignore_failure` is set.
1716

1817
The result of this operation is an array of values, given that a single attribute can hold multiple values.
1918

20-
Note: the search attribute is expected to map to a single object. If it doesn’t, no error will be returned, but only results of the first entry will be added to the event.
19+
The search attribute is expected to map to a single object. If multiple entries match, only the first entry's mapped attribute values are returned.
2120

2221
```yaml
2322
processors:
2423
- translate_ldap_attribute:
2524
field: winlog.event_data.ObjectGuid
26-
ldap_address: "ldap://"
27-
ldap_base_dn: "dc=example,dc=com"
2825
ignore_missing: true
2926
ignore_failure: true
27+
# ldap_domain: "example.com" # Optional - override the OS-discovered domain used for SRV/LOGONSERVER hints
28+
# ldap_address: "ldap://ds.example.com:389" # Optional - Beats discovers controllers when omitted
29+
# ldap_base_dn: "dc=example,dc=com" # Optional - otherwise rootDSE and hostname inference are used
3030
```
3131

3232
The `translate_ldap_attribute` processor has the following configuration settings:
@@ -35,18 +35,40 @@ The `translate_ldap_attribute` processor has the following configuration setting
3535
| --- | --- | --- | --- |
3636
| `field` | yes | | Source field containing a GUID. |
3737
| `target_field` | no | | Target field for the mapped attribute value. If not set it will be replaced in place. |
38-
| `ldap_address` | yes | | LDAP server address. eg: `ldap://ds.example.com:389` |
39-
| `ldap_base_dn` | yes | | LDAP base DN. eg: `dc=example,dc=com` |
40-
| `ldap_bind_user` | no | | LDAP user. |
41-
| `ldap_bind_password` | no | | LDAP password. |
38+
| `ldap_domain` | no | | {applies_to}`stack: ga 9.2.4` DNS domain name (for example, `example.com`) used for DNS SRV discovery and to construct FQDNs from `LOGONSERVER`. When omitted Beats inspects OS metadata to infer the domain (Windows: `USERDNSDOMAIN`, `GetComputerNameEx`, TCP/IP + Kerberos registry keys, hostname; Linux/macOS: `/etc/resolv.conf`, `/etc/krb5.conf`, hostname). |
39+
| `ldap_address` | {applies_to}`stack: ga 9.2.4` no<br>{applies_to}`stack: ga 9.0.0` yes | | LDAP server address (for example, `ldap://ds.example.com:389`). When omitted Beats auto-discovers controllers by querying `_ldaps._tcp.<domain>` first, `_ldap._tcp.<domain>` second, and finally the Windows `LOGONSERVER` variable if available. Candidates are tried in order until one succeeds. |
40+
| `ldap_base_dn` | {applies_to}`stack: ga 9.2.4` no<br>{applies_to}`stack: ga 9.0.0` yes | | LDAP base DN (for example, `dc=example,dc=com`). When omitted Beats queries the server's rootDSE for `defaultNamingContext`/`namingContexts`. If the controller does not expose those attributes, client initialization fails and you must configure the value manually. |
41+
| `ldap_bind_user` | no | | LDAP DN/UPN for simple bind. When provided with `ldap_bind_password` Beats performs a standard bind. When set without a password Beats issues an unauthenticated bind using this identity (useful for servers that expect a bind DN even for anonymous operations). |
42+
| `ldap_bind_password` | no | | LDAP password for simple bind. When both the username and password are omitted Beats attempts automatic authentication: on Windows it first tries SSPI with the Beat's service or user identity using the SPN `ldap/<hostname derived from ldap_address>` and falls back to an unauthenticated bind if that fails. Non-Windows platforms immediately use an unauthenticated bind. |
4243
| `ldap_search_attribute` | yes | `objectGUID` | LDAP attribute to search by. |
4344
| `ldap_mapped_attribute` | yes | `cn` | LDAP attribute to map to. |
4445
| `ldap_search_time_limit` | no | 30 | LDAP search time limit in seconds. |
45-
| `ldap_ssl`\* | no | 30 | LDAP TLS/SSL connection settings. |
46+
| `ldap_ssl` | no | {applies_to}`stack: ga 9.2.4` no default<br>{applies_to}`stack: ga 9.0.0` `30` | LDAP TLS/SSL connection settings. Refer to [SSL](/reference/filebeat/configuration-ssl.md). |
47+
| `ad_guid_translation` | no | `auto` | {applies_to}`stack: ga 9.2.4` Controls GUID binary conversion for Active Directory attributes. `auto` (default) converts when the LDAP search attribute equals `objectGUID` (case-insensitive). Use `always` to force conversion or `never` to disable it. |
4648
| `ignore_missing` | no | false | Ignore errors when the source field is missing. |
4749
| `ignore_failure` | no | false | Ignore all errors produced by the processor. |
4850

49-
\* Also see [SSL](/reference/filebeat/configuration-ssl.md) for a full description of the `ldap_ssl` options.
51+
## Authentication flow
52+
53+
Beats attempts LDAP authentication in the following order:
54+
55+
1. Simple bind using `ldap_bind_user` and `ldap_bind_password` when both are supplied.
56+
2. Automatic bind when both values are empty. On Windows Beats creates an SSPI (Kerberos/NTLM) client for the SPN `ldap/<hostname derived from ldap_address>`, which works for Local System, domain-joined services, and gMSA accounts. Other platforms do not yet implement automatic authentication.
57+
3. If automatic authentication is unavailable or fails, Beats issues an unauthenticated bind. When `ldap_bind_user` is set without a password that identity is used; otherwise Beats binds anonymously.
58+
59+
Always prefer specifying `ldap_address` as an FQDN (for example `ldap://dc1.example.com:389`) so the SPN built for SSPI matches the controller's service principal and TLS certificates.
60+
61+
## Server auto-discovery
62+
63+
When `ldap_address` is omitted Beats resolves controllers dynamically:
64+
65+
1. **Domain discovery.** Beats determines the DNS domain from `ldap_domain` (if set) or OS metadata. Windows checks `USERDNSDOMAIN`, `GetComputerNameEx`, the TCP/IP and Kerberos registry keys, and the machine's FQDN. Linux/macOS read `/etc/resolv.conf`, `/etc/krb5.conf`, and the hostname suffix. If no domain is available SRV lookups are skipped.
66+
2. **DNS SRV queries.** When a domain is known Beats queries `_ldaps._tcp.<domain>` first and `_ldap._tcp.<domain>` second using the system resolver. Results are sorted by priority/weight per RFC 2782 and converted to `ldaps://host:port` or `ldap://host:port` URLs.
67+
3. **Windows LOGONSERVER fallback.** If SRV queries return no controllers or no domain was discovered, Beats reads the `LOGONSERVER` environment variable. When a domain is known the NetBIOS name is combined with it to build an FQDN so TLS validation and SSPI SPNs remain valid.
68+
69+
Each candidate address is attempted in order (LDAPS before LDAP) until a connection and bind succeed.
70+
71+
When `ldap_base_dn` is empty the client queries the controller's rootDSE for `defaultNamingContext` or the first non-system `namingContexts` entry. If neither is present Beats cannot continue and you must provide `ldap_base_dn` explicitly.
5072

5173
If the searches are slow or you expect a high amount of different key attributes to be found, consider using a cache processor to speed processing:
5274

@@ -78,4 +100,3 @@ processors:
78100
key_field: winlog.event_data.ObjectGuid
79101
value_field: winlog.common_name
80102
```
81-

0 commit comments

Comments
 (0)