Posts Tagged ‘one-liner’

Auto-iptables off IPs with high connection counts

Saturday, August 29th, 2009

via Paul (lovepig.org):

netstat -npa --inet | grep :80 | sed 's/:/ /g' | awk '{print $6}' | sort | uniq -c | sort -n | while read line; do one=`echo $line | awk '{print $1}'`; two=`echo $line | awk '{print $2}'`; if [ $one -gt 100 ];
then iptables -I INPUT -s $two -j DROP; fi; done; iptables-save | grep -P '^-A INPUT' | sort | uniq -c | sort -n | while read line; do oneIp=`echo $line | awk '{print $1}'`; twoIp=`echo $line | awk '{print $5}'`; if [ $oneIp -gt 1 ]; then iptables -D INPUT -s $twoIp -j DROP; fi; done

This one-liner is quite effective when tossed into a file and run as a cronjob once per minute. Any IP with more than 100 concurrent connections — which, quite honestly, is far more than any one IP should ever have on a standard webserver — will be blocked via iptables. This script as a cronjob is extremely effective dealing with small-to-midsize DDoSes (too much traffic for Apache/whatever service to handle, but not saturating the pipe).

Obtaining Plesk user for a domain

Friday, June 19th, 2009

…for a list of domains, without digging through the database!

cat domains | sort |uniq |while read line ; do ls -ld /home/httpd/vhosts/$line/httpdocs |awk '{print $3}'

'domains', of course, is a text file with a list of domains hosted on the server. Can be populated in whatever way you need. Easily plugged into other Plesk utilities (such as changing Plesk FTP passwords).

Combining text files as columns

Friday, June 19th, 2009

To combine two (or more) text files as individual columns in the same file, such as:

file1:

foo
foo1
foo2
foo3

file2:

foobar
foobar1
foobar2
foobar3

into:

foo foobar
foo1 foobar1
foo2 foobar2
foo3 foobar3

rather than using an ugly combination of sed and awk, you can use the `paste' command:

paste file1 file2

Sort based on a column

Thursday, April 16th, 2009

You can use the `sort' utility to sort not only on the first field, but also an arbitrary column. Take this output from ps:

[kale@superhappykittymeow ~]$ ps ax -ly |grep httpd
R   502 10027  8387  0  78   0   668   979 -      pts/1      0:00 grep httpd
S   502 10247 28321  0  76   0 16924  9961 semtim ?          1:58 /usr/sbin/httpd
S     0 28321     1  0  78   0 11564  7645 -      ?          0:00 /usr/sbin/httpd
S   502 28327 28321  0  75   0 19152 10412 semtim ?          1:39 /usr/sbin/httpd
S   502 28328 28321  0  84   0 16628  9903 semtim ?          1:56 /usr/sbin/httpd
S   502 28331 28321  0  75   0 17108  9962 semtim ?          1:45 /usr/sbin/httpd
S   502 28332 28321  0  75   0 19152 10446 semtim ?          1:54 /usr/sbin/httpd
S   502 28333 28321  0  75   0 15692  9624 semtim ?          1:54 /usr/sbin/httpd
S   502 28334 28321  0  78   0 17476 10107 semtim ?          2:01 /usr/sbin/httpd
S   502 28335 28321  0  75   0 17460 10237 semtim ?          1:57 /usr/sbin/httpd
S   502 28336 28321  0  75   0 16836  9897 -      ?          1:54 /usr/sbin/httpd
S   502 30058 28321  0  75   0 15248  9622 semtim ?          0:33 /usr/sbin/httpd

This can be sorted by the SZ column, column 9, as such:

[kale@superhappykittymeow ~]$ ps ax -ly |grep httpd |sort -nr -k 9n
R   502 10045  8387  0  78   0   668   979 -      pts/1      0:00 grep httpd
S     0 28321     1  0  78   0 11564  7645 -      ?          0:00 /usr/sbin/httpd
S   502 30058 28321  0  75   0 15248  9622 semtim ?          0:33 /usr/sbin/httpd
S   502 28333 28321  0  75   0 15692  9624 semtim ?          1:54 /usr/sbin/httpd
S   502 28336 28321  0  75   0 16836  9897 -      ?          1:54 /usr/sbin/httpd
S   502 28328 28321  0  84   0 16628  9903 semtim ?          1:56 /usr/sbin/httpd
S   502 10247 28321  0  76   0 16924  9961 semtim ?          1:58 /usr/sbin/httpd
S   502 28331 28321  0  75   0 17108  9962 semtim ?          1:45 /usr/sbin/httpd
S   502 28334 28321  0  78   0 17476 10107 semtim ?          2:01 /usr/sbin/httpd
S   502 28335 28321  0  75   0 17460 10237 semtim ?          1:57 /usr/sbin/httpd
S   502 28327 28321  0  75   0 19152 10412 semtim ?          1:39 /usr/sbin/httpd
S   502 28332 28321  0  75   0 19152 10446 semtim ?          1:54 /usr/sbin/httpd

The -k switch tells sort to sort based on a key, which we specify as 9n (column 9, numeric). Much easier to review the output.

Serve current directory temporarily via web

Saturday, April 11th, 2009
python -m SimpleHTTPServer

Runs in the foreground a simple, single-threaded web server on port 8000 as the current user. Logging is to stdout/stderr, and a ctrl-c will stop the server. Great for temporarily sharing a directory.

Curl with postdata and cookies

Sunday, April 5th, 2009

Great for command-line logging into sites to pull content for whatever reason.

curl -c cookies.txt -d "username=username&password=password&action=login" -o /home/kale/outputfile.txt "http://www.domain.com/authenticated_page.php?foo=bar"

Of course, you'll have to look at the source for the target location's login page to see what variables it wants. I use it to grab a single Cacti-generated graph that is normally password protected, but I want to include a single graph on another site, so I cron'd a script to run a line similar to the above to log in and save it locally.

Pick an IP address out of any file

Wednesday, April 1st, 2009
perl -ne 'while (/([0-9]+\.){3}[0-9]+/g) {print "$&\n"};' file.txt

from command-line-fu

Who's connecting to Apache?

Saturday, March 28th, 2009

Spot DDoS's and the like quickly:

netstat -plan | grep :80 | awk '{print $5}' | sed 's/:.*$//' | sort | uniq -c | sort -rn |head

Change all of Plesk's FTP passwords to random

Sunday, March 22nd, 2009
for i in $(mysql -NB psa -uadmin -p`cat /etc/psa/.psa.shadow` -e 'select login from sys_users;'); do export PSA_PASSWD="$(openssl rand 6 -base64)"; /usr/local/psa/admin/bin/usermng --set-user-passwd --user=$i; echo "$i: $PSA_PASSWD" >> ftp_passwords; done

Thanks Geoff!

Make sure your crons run on time

Thursday, March 19th, 2009

If you add an entry to crontab that is an interval, such as */3 (every 3 minutes), you can verify that it runs at the specified interval with a bit of awk:

cat /var/log/cron |grep cron-script |awk -F\: '{if ($2/3 == 0) print $0}' |grep -v ":00:"

This essentially checks to see that the minute field of the timestamp is divisible by three — the interval. It'll also run at 00 after the hour, not divisible by three, but expected.

Cron can "run late" at times due to high load situations, so if there are any irregularities in your intervals, you may wish to investigate deeper, looking for expensive processes that are chewing up precious cron time.

What is Apache doing?

Monday, March 9th, 2009

Ever wish you knew what Apache was working on at any given moment, but kicking yourself because you forgot to enable a server-status directive? This snippet will help you diagnose timeouts and long-running scripts (for bad coders like myself):

for i in `ps -elf |grep http|awk '{print $4}'|sort|uniq`; do ls -la /proc/$i/cwd ; done|awk '{print $11}'|sort|uniq -c |sort -nr

find with spaces in filenames

Wednesday, March 4th, 2009

Set bash's internal field seperator to an enter instead of a space, so grep (or whatever) doesn't freak out:

IFS='
'
; for i in `find -name "*.php" ` ; do grep foo $i ; done

Find total file sizes

Wednesday, March 4th, 2009
find /var/www/vhosts/*/statistics/logs -type d -exec du -sm {} \; | awk '{total+=$1} END {print total,"MB"}'

Find total sizes of files in all those logs directories

Find files that do not contain a string

Wednesday, March 4th, 2009

To find files that do NOT contain a specific string you can do the following:

find -name "ifcfg-eth0:*" -type f ! -exec grep -q ONBOOT {} \; -exec ls {} \;

will list all files named ifcfg-eth0:* that do not contain the string ONBOOT.

You can script this up as such:

for i in `find -name "ifcfg-eth0:*" -type f ! -exec grep -q ONBOOT {} \; -exec ls {} \; |awk -F\/ '{print $2}'`; do echo ONBOOT=yes >> $i ; done

to add the required ONBOOT=yes line to the config.

Find the 50 largest files

Wednesday, March 4th, 2009

What's eating up all your disk space?

find / -path /dev -prune -o -path /sys -prune -o -path /proc -prune -o -type f \
 -size '+1024k' -printf "%s %h/%f\n" | sort -rn -k1 | head -n50 | \
 awk '{ printf("%5dMB\t%s\n", $1/1048576, substr($0, index($0, " ")+1, length($0))) }'

Extract a single table from a sql dump

Wednesday, March 4th, 2009

The only caveat is that you have to know the table that comes after the one you're trying to extract.  It's alphabetical, if you can get a list of tables, otherwise a quick search of the SQL file will get that info for you.

awk '/Table structure for table .table1./,/Table structure for table .table2./{print}' bigassdatabase.sql > table1.sql

Perl module infos

Wednesday, March 4th, 2009

Check to see if a perl module is installed:

perl -MMODULENAME -e1

(no output == success)

Check perl module versions:

perl -MMODULENAME -e'print "$MODULENAME::VERSION\n";'

Find out where a perl module is installed (docs, sources, etc):

perl -MExtUtils::Installed -e'$,="\n";print ExtUtils::Installed->new()->directories("MODULENAME")," "'

Silly RPM tricks

Wednesday, March 4th, 2009

Find all non-Red Hat-supplied packages:

rpm -qa --qf '%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH} %{VENDOR}\n' | grep -v 'Red Hat, Inc\.' | sort

Handy for diagnosing issues where things seem a little "off".

CPU affinity-aware `ps'

Wednesday, March 4th, 2009
ps -eo pid,tid,class,rtprio,ni,pri,pcpu,stat,wchan:14,comm,psr

The last number is the CPU the process is currently waiting on.  Quite useful when used in conjunction with `top', as hitting the number 1 while in interactive mode will display the per-CPU usage.  Helpful to find iowait.