diff options
| author | Paul Buetow <pbuetow@mimecast.com> | 2021-12-09 10:22:25 +0000 |
|---|---|---|
| committer | Paul Buetow <pbuetow@mimecast.com> | 2021-12-09 10:22:25 +0000 |
| commit | a9372bc8a882b59fcdd3997a56acc2338776f602 (patch) | |
| tree | b10ff367b7def6bc38b64b27dfd138cb5c4ae692 /doc/querylanguage.md | |
| parent | 18d1783378732b6abca0eb89e29636cc81c02db8 (diff) | |
Documenting log formats
Diffstat (limited to 'doc/querylanguage.md')
| -rw-r--r-- | doc/querylanguage.md | 53 |
1 files changed, 6 insertions, 47 deletions
diff --git a/doc/querylanguage.md b/doc/querylanguage.md index c6b9beb..96d0fd1 100644 --- a/doc/querylanguage.md +++ b/doc/querylanguage.md @@ -7,9 +7,7 @@ The query language allows you to run mapreduce queries on log files. This page i For this to work, DTail needs to understand your log format. DTail already understands its own log format. You can have a look at all examples of the [examples](./examples.md) page using `-query` (these would be all examples of the `dmap` command, and some examples using the `dtail` command). -DTail also ships with a generic log format, which only allows you to run very basic queries. Check out the [log formats](./logformats.md) documentation for this. - -To implement your own log format, please also check out the [log formats](./logformats.md) documentation. +DTail also ships with a generic log format, which only allows you to run very basic queries. Check out the [log format](./logformats.md) documentation for this. To implement your own log format, please also check out the log format documentation. ## The complete language @@ -23,6 +21,7 @@ QUERY := [interval SECONDS] [limit NUM] [outfile "FILENAME.csv"] + [logformat LOGFORMAT] SELECT := FIELD|AGGREGATION(FIELD) TABLE := The mapreduce table name, e.g. WRITE in MAPREDUCE:WRITE AGGREGATION := count|sum|min|max|avg|last|len @@ -31,56 +30,16 @@ ARG := This is either a string: "foo bar" a float number: 3.14 a bareword e.g.: responsecode - or a $variable (see below). + a field or a $variable OPERATOR := This is one of ... Floating point operators: == != < <= > >= String operators: - eq ne contains lacks (lacks is the opposite of contains, e.g. - "not contains") + eq ne contains lacks (lacks is the opposite of contains, e.g. "not contains") GROUPFIELD := bareword|$variable ORDERFIELD := This must be a AGGREGATION(FIELD) or FIELD which was specified in select clause already. +LOGFORMAT := The name of the log format implementation. It's 'default' by default. ``` -## Predefined variables - -This is the list of pre-defined variables. Please note that these vary depending on the log format used. - -### Common variables: - -The common variables may exist in all log formats. - -* `$empty` - The empty string `""` -* `$hostname` - The server FQDN -* `$line` - The current log line -* `$server` - Alias for `$hostname` -* `$timeoffset` - Offset of $timezone -* `$timezone` - The current time zone -* `* (special placeholder) - -### DTail default log format: - -These variables may only exist when your logs are in the DTail default log format: - -*Date and time:* - -* `$hour` - The current hour in format HH -* `$minute` - The current minute in format MM -* `$second` - The current second in format SS. -* `$time` - The current time in format YYYYMMDD-HHMMSS - -*Log level/severity:* - -* `$loglevel` - Alias for `$severity` -* `$severity` - The log severity - -*System and Go runtime:* - -* `$caller` - DTail server caller of the logger -* `$cgocalls` - Num of DTail server CGo calls -* `$cpus` - Num of DTail server CPUs used -* `$goroutines` - Num of DTail server Goroutines used -* `$loadavg` - 1 min. average load average -* `$pid` - DTail server process ID -* `$uptime` - DTail server uptime +Note, that the available fields and variables vary from the log format used. There is also a subtle difference between a field and a variable. Check out the [log format](./logformats.md) documentation for more information. |
