summaryrefslogtreecommitdiff
path: root/doc/querylanguage.md
diff options
context:
space:
mode:
authorPaul Buetow <paul@buetow.org>2026-07-22 23:52:52 +0300
committerPaul Buetow <paul@buetow.org>2026-07-22 23:52:52 +0300
commit3004a7100e325c006971cc2e8d0f157338c0ce5c (patch)
treeb9d2be78433b2d6e13be6344357d1f81fa9ec44b /doc/querylanguage.md
parent17bf7e042496a4afcbf6ee7a583378adb3ec502d (diff)
docs: DTail fork — documentation, agent guide, example configsHEADmaster
Squashed development of the documentation and example configuration: - AGENTS.md / CLAUDE.md: repository guide describing build/test/benchmark/PGO workflows and the single default read/output path (formerly "turbo"). - doc/ and docs/: query-language reference, log formats, auth-key fast reconnect, journal source reads, performance analyses (dated point-in-time records kept under historical-note disclaimers), and the turbo-vs-normal benchmark report with its result CSVs. - README.md updates; examples/ config + JSON schema aligned with the current Output* server tuning fields (the removed TurboBoost* keys dropped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diffstat (limited to 'doc/querylanguage.md')
-rw-r--r--doc/querylanguage.md40
1 files changed, 39 insertions, 1 deletions
diff --git a/doc/querylanguage.md b/doc/querylanguage.md
index c3e567e..a405f35 100644
--- a/doc/querylanguage.md
+++ b/doc/querylanguage.md
@@ -52,7 +52,7 @@ STRINGOPERATOR := eq|ne|contains|ncontains|lacks|hasprefix|nhasprefix|hassuffix|
ORDERFIELD := FIELD|AGGREGATION(FIELD)
SET := $VARIABLE = FLOAT|STRING|FIELD|FUNCTION(FIELD)
LOGFORMAT := default|generic|generickv|...
-AGGREGATION := count|sum|min|max|avg|last|len
+AGGREGATION := count|sum|min|max|avg|last|len|percentage|percentile
FUNCTION := md5sum|maskdigits
```
@@ -61,3 +61,41 @@ FUNCTION := md5sum|maskdigits
* `rorder` stands for reverse order.
* `lacks` is an alias for `ncontains` (not contains).
* Available fields (variables and barewords) vary from the log format used. Check out the [log format](./logformats.md) documentation for more information.
+* `percentage(field)` returns the selected group's share of the total for that field across all groups. For non-negative inputs, the result is between 0 and 100; with mixed positive and negative values, it can fall outside that range.
+* `percentile(field)` returns the percentile rank of the selected group's value among all grouped values for that field, also expressed as a value between 0 and 100. Equal values share the same rank.
+
+## Selecting the log format and dynamic fields
+
+Two things commonly trip people up when a `$field`-style reference "does not
+resolve" while positional/built-in fields work. Both are by design:
+
+1. **Dynamic `key=value` fields are barewords, not `$`-variables.** A log line
+ like `...|service=web|bytes=100` exposes `service` and `bytes` as *barewords*.
+ Query them as `select service,sum(bytes)` — **not** `$service`/`$bytes`. The
+ `$` prefix is reserved for values DTail sets itself (e.g. `$time`, `$hostname`,
+ `$line`). A `$name` that is not one of those built-ins silently resolves to
+ the empty string, which is exactly what "did not resolve" looks like:
+ everything collapses into a single empty group. To catch this early, the
+ client prints a plan-time warning to stderr for every `$`-variable the
+ selected parser cannot populate, e.g.
+ `warning: $service is not a known variable for log format "default"; did you
+ mean bareword service?`. It is only a warning (resolution behaviour is
+ unchanged), and it is never emitted for barewords, for built-ins like
+ `$empty`, or for variables defined via a `set` clause.
+
+2. **The `from TABLE` clause selects the rich parser.** Although `from TABLE` is
+ written as optional in the grammar above, omitting it (and not passing an
+ explicit `logformat`) downgrades the query to the `generic` log format, which
+ exposes only the common variables (`$line`, `$hostname`, ...) and **no**
+ dynamic `key=value` fields and **no** default-format `$`-variables such as
+ `$time`. To query DTail's own default-format logs (lines containing
+ `MAPREDUCE:STATS`), use `from STATS`, for example:
+
+ ```shell
+ % dmap --files /var/log/dserver/dserver.log \
+ --query 'from STATS select $hostname,max($goroutines),lifetimeConnections group by $hostname'
+ ```
+
+ Alternatively, name the parser explicitly with the `logformat` keyword (e.g.
+ `logformat generickv`), which works regardless of the `from` clause. See the
+ [log formats](./logformats.md) documentation for details.