1.3 : Basic Bash scripting - head, tail, wc, sort & uniq (Part III)

Examining server logs

Take a look on the file by head , tail , wc , sort , uniq command

let say we have example log file like this:

apache.log

123.45.6.7 - - [21/Apr/2014:07:09:18 -0500] "GET / HTTP/1.1" 200 356 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.29.13 (KHTML, like Gecko) Version/6.0.4 Safari/536.29.13"

Common usage:

• basic lookup

  head apache.log # show first few lines in apache.log
  tail apache.log # show last few lines in apache.log
  wc -l apache.log # show total number of lines in apache.log
  

• show all unique IP access

  cat apache.log |  cut -d " " -f 1 | sort -u  >> ips.txt
  

• sort IP address by total number of access

  cat apache.log | cut -d " " -f 1 | sort | uniq -c | sort -urn
  

• check attackers most interested endpoint

  cat apache.log | grep '127.0.0.1' | cut -d "\"" -f 2 | uniq -c