1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
|
# Bash I/O, Pipelines, and Data Processing
## Built-ins vs External Commands
- Prefer Bash built-ins for light text processing and arithmetic.
- Use external commands (`sed`, `awk`, `grep`, `cut`, `tr`, `bc`) for heavy or complex text processing.
```bash
# Prefer built-in
addition=$(( X + Y ))
# Prefer external for sophisticated transforms
substitution="$(echo "$string" | sed -e 's/^foo/bar/')"
```
## Process Substitution
Use `<(command)` and `>(command)` for treating command output as a file.
```bash
diff -u <(sort file1) <(sort file2)
tar cjf >(bzip2 -c > file.tar.bz2) foo
```
## `while read` with Process Substitution
When iterating over command output and modifying variables in the parent shell, use process substitution as the input source rather than piping into `while`. A pipe creates a subshell, so variable changes are lost.
```bash
# Good: changes to $count survive
local -i count=0
while IFS='' read -r line; do
(( count++ ))
done < <(command)
# Bad: $count is lost because the while runs in a subshell
local -i count=0
command | while IFS='' read -r line; do
(( count++ ))
done
```
### `IFS='' read -r line` for exact line preservation
Use `IFS='' read -r line` when reading lines you intend to preserve exactly, including leading and trailing whitespace. Without `IFS=''`, leading/trailing whitespace is stripped; without `-r`, backslashes are interpreted.
```bash
while IFS='' read -r line; do
echo "$line"
done < file.txt
```
## Here-Documents and Here-Strings
Use here-docs and here-strings for multi-line or inline input.
```bash
# Here-document with variable interpolation
cat <<EOF
Hello $USER
EOF
# Literal here-document (no interpolation)
cat <<'EOF'
$USER is not expanded
EOF
# Here-string
if grep -q foo <<< "$VAR"; then
echo match
fi
```
Use `<<-EOF` to strip leading tabs from the body.
### Prefer here-strings over `echo | command`
For single-line input, prefer `command <<< "$var"` instead of `echo "$var" | command`. It avoids an extra pipe, subshell, and process spawn.
```bash
# Good
tr '[:upper:]' '[:lower:]' <<< "$text"
# Avoid
echo "$text" | tr '[:upper:]' '[:lower:]'
```
## Input Placeholders and Redirection
- `-` as stdin/stdout placeholder for commands like `tar` and `cat`.
- Redirect via file descriptors explicitly (`2>/dev/null`, `1>&2`).
- Remember redirection order matters.
```bash
echo Foo 2>/dev/null 1>&2 # suppresses everything
```
## `/dev/tcp` Networking
Bash supports TCP via pseudo-files:
```bash
cat < /dev/tcp/time.nist.gov/13
exec 5<>/dev/tcp/google.de/80
```
## List Processing: Pipes over Arrays
For simple list processing, prefer pipelines over arrays. Pass data through stdout to the next stage and use stderr for logging.
```bash
main () {
filter_lines |
process_lines |
postprocess_lines |
generate_report
}
```
## Reading Files and Arrays
- **Read a whole file into a variable** without spawning `cat`: `cfg=$(<config.ini)`
- **Read lines into an array** safely with `mapfile` (aka `readarray`): `mapfile -t lines < file`
- **Assign formatted strings without a subshell** using `printf -v`: `printf -v msg 'Hello %s' "$USER"`
## Safe xargs with NULs
Avoid breaking on spaces/newlines by pairing `find -print0` with `xargs -0`:
```bash
find . -type f -name '*.log' -print0 | xargs -0 rm -f
```
|
