BashWars

How to build bash oneliners

grep

Search string without case sensative

grep -i 'example' input.txt

Counting the number of matches

grep -c 'example' input.txt

Search with regex

grep -E 'example[0-9]+' input.txt

Printing strings without matches

grep -v 'example' input.txt

cut

Retrieving characters in a range

To extract the fifth through tenth characters from each string:

cut -c 5-10 input.txt

Extracting fields using a delimiter

To extract the second field, where the fields are separated by commas:

cut -d ',' -f 2 input.txt

sort

Sort strings alphabetically

sort input.txt

To sort strings in reverse order:

sort -r input.txt

Sorting numeric values

sort -n input.txt

Removing duplicates when sorting

sort -u input.txt

uniq

Remove duplicate strings

uniq input.txt

Count duplicate strings

uniq -c input.txt

tr

Replacing characters

To replace all colons to spaces:

tr ':' ' ' < input.txt

Remove characters

To remove all numbers:

tr -d '0-9' < input.txt

Case conversion

To convert all lowercase letters to uppercase:

tr 'a-z' 'A-Z' < input.txt

head & tail

Printing the first N lines

To print first 10 strings in file:

head -n 10 input.txt

Printing the last N lines

To print last 10 strings in file:

tail -n 10 input.txt

xargs

Passing arguments to a command

To pass a list of files to the rm command:

ls | xargs rm

Limiting the number of arguments

To pass only two arguments at a time:

ls | xargs -n 2 echo

Applying a command in parallel

To execute a command in parallel on each file:

ls | xargs -P 4 -n 1 echo

find

Find file by name

To search for all files named file.txt in the current directory and subdirectories:

find . -name 'file.txt'

Find file by size

To search for all files larger than 1 MB:

find . -size +1M

3. Find and execute command

To delete all file with .tmp extension:

find . -name '*.tmp' -exec rm {} \;

wc

Count strings in file

wc -l input.txt

Count word in file

wc -w input.txt

Count symbols in file

wc -m input.txt

AWK

Take info from output by column

cat file.txt | awk '{print #1}'

Count strings in file

awk 'END {print NR}' file.txt

Delete empty strings in file

awk 'NF > 0' file.txt

Sum column values

To sum the values ​​in a specific column (for example, the third column):

awk '{sum += $3} END {print sum}' file.txt

Counting unique values ​​in a column

To count the number of unique values ​​in the first column:

awk '!seen[$1]++ {unique++} END {print unique}' file.txt

sed

Replacing text

To replace all occurrences of the word "oldtext" with "newtext":

sed 's/oldtext/newtext/g' file.txt

Delete strings

To delete lines containing a specific word, such as "delete":

sed '/delete/d' input.txt

Inserting text after a line

To insert the line "Inserted text" after each line containing the word "pattern":

sed '/pattern/a\Inserted text' file.txt

To insert the line "Inserted text" before each line containing the word "pattern":

sed '/pattern/i\Inserted text' file.txt

Replace text only on certain lines

To replace the text "oldtext" with "newtext" only on lines containing the word "pattern":

sed '/pattern/s/oldtext/newtext/g' file.txt

Removing the first N rows

To remove the first 5 lines from a file:

sed '1,5d' file.txt

Printing Specific Strings

To output lines 10 to 20:

sed -n '10,20p' file.txt

jq

Format string to json

cat file.txt | jq

Retrieving a value by key

To extract the value for the name key from JSON:

jq '.name' file.json

To retrieve a value from a nested key name.age:

jq '.name.age' file.json

Filtering array elements

To filter array elements where the age field is greater than 30:

jq '.people[] | select(.age > 30)' file.json

JSON structure conversion

To change the JSON structure, for example to output only the name and age:

jq '.people[] | {name: .name, age: .age}' file.json

Retrieving all values ​​of a specific key

To retrieve all id key values ​​from an array of objects:

jq '.items[].id' file.json

Counting elements in an array

To count the number of elements in the items array:

jq '.items | length' file.json

Replacing values ​​in JSON

To replace all status key values ​​with active:

jq '(.items[].status) |= "active"' file.json