summaryrefslogtreecommitdiff
path: root/doc/querylanguage.md
diff options
context:
space:
mode:
authorPaul Buetow <pbuetow@mimecast.com>2021-12-09 10:22:25 +0000
committerPaul Buetow <pbuetow@mimecast.com>2021-12-09 10:22:25 +0000
commita9372bc8a882b59fcdd3997a56acc2338776f602 (patch)
treeb10ff367b7def6bc38b64b27dfd138cb5c4ae692 /doc/querylanguage.md
parent18d1783378732b6abca0eb89e29636cc81c02db8 (diff)
Documenting log formats
Diffstat (limited to 'doc/querylanguage.md')
-rw-r--r--doc/querylanguage.md53
1 files changed, 6 insertions, 47 deletions
diff --git a/doc/querylanguage.md b/doc/querylanguage.md
index c6b9beb..96d0fd1 100644
--- a/doc/querylanguage.md
+++ b/doc/querylanguage.md
@@ -7,9 +7,7 @@ The query language allows you to run mapreduce queries on log files. This page i
For this to work, DTail needs to understand your log format. DTail already understands its own log format. You can have a look at all examples of the [examples](./examples.md) page using `-query` (these would be all examples of the `dmap` command, and some examples using the `dtail` command).
-DTail also ships with a generic log format, which only allows you to run very basic queries. Check out the [log formats](./logformats.md) documentation for this.
-
-To implement your own log format, please also check out the [log formats](./logformats.md) documentation.
+DTail also ships with a generic log format, which only allows you to run very basic queries. Check out the [log format](./logformats.md) documentation for this. To implement your own log format, please also check out the log format documentation.
## The complete language
@@ -23,6 +21,7 @@ QUERY :=
[interval SECONDS]
[limit NUM]
[outfile "FILENAME.csv"]
+ [logformat LOGFORMAT]
SELECT := FIELD|AGGREGATION(FIELD)
TABLE := The mapreduce table name, e.g. WRITE in MAPREDUCE:WRITE
AGGREGATION := count|sum|min|max|avg|last|len
@@ -31,56 +30,16 @@ ARG := This is either
a string: "foo bar"
a float number: 3.14
a bareword e.g.: responsecode
- or a $variable (see below).
+ a field or a $variable
OPERATOR := This is one of ...
Floating point operators:
== != < <= > >=
String operators:
- eq ne contains lacks (lacks is the opposite of contains, e.g.
- "not contains")
+ eq ne contains lacks (lacks is the opposite of contains, e.g. "not contains")
GROUPFIELD := bareword|$variable
ORDERFIELD := This must be a AGGREGATION(FIELD) or FIELD which was specified in
select clause already.
+LOGFORMAT := The name of the log format implementation. It's 'default' by default.
```
-## Predefined variables
-
-This is the list of pre-defined variables. Please note that these vary depending on the log format used.
-
-### Common variables:
-
-The common variables may exist in all log formats.
-
-* `$empty` - The empty string `""`
-* `$hostname` - The server FQDN
-* `$line` - The current log line
-* `$server` - Alias for `$hostname`
-* `$timeoffset` - Offset of $timezone
-* `$timezone` - The current time zone
-* `* (special placeholder)
-
-### DTail default log format:
-
-These variables may only exist when your logs are in the DTail default log format:
-
-*Date and time:*
-
-* `$hour` - The current hour in format HH
-* `$minute` - The current minute in format MM
-* `$second` - The current second in format SS.
-* `$time` - The current time in format YYYYMMDD-HHMMSS
-
-*Log level/severity:*
-
-* `$loglevel` - Alias for `$severity`
-* `$severity` - The log severity
-
-*System and Go runtime:*
-
-* `$caller` - DTail server caller of the logger
-* `$cgocalls` - Num of DTail server CGo calls
-* `$cpus` - Num of DTail server CPUs used
-* `$goroutines` - Num of DTail server Goroutines used
-* `$loadavg` - 1 min. average load average
-* `$pid` - DTail server process ID
-* `$uptime` - DTail server uptime
+Note, that the available fields and variables vary from the log format used. There is also a subtle difference between a field and a variable. Check out the [log format](./logformats.md) documentation for more information.