Practical awk (for Apache logs)

Who is hotlinking?
[code lang=”bash”]awk -F\” ‘($2 ~ /\.(jpg|gif)/ && $4 !~ /^http:\/\/www\.yourdomain\.com/){print $4}’ access_log.processed \
| sort | uniq -c | sort[/code]

Blank referrers (usually indicates direct hits, such as a user typing in yourdomain.com, or a script):
[code lang=”bash”]awk -F\” ‘($6 ~ /^-?$/)’ access_log.processed | awk ‘{print $1}’ | sort | uniq[/code]

How many different IPs visited on a specific day (and how often they visited):
[code lang=”bash”]grep ’12/Dec/2008′ access_log.processed | \
awk ‘{cnt[$1]++;} END{for (ip in cnt){printf(“%-15s visited: %04d time(s).\n”, ip, cnt[ip])}}'[/code]

Amount of data transferred for a specific date:
[code lang=”bash”]grep ’12/Dec/2008’ access_log.processed | awk ‘{ SUM += $10} END { print SUM/1024/1024 }'[/code]

Leave a Reply