Memory management in Linux

This is a prefab for me to paste into tickets whenever a customer is confused about “free” and “used” memory.

Memory in Linux isn’t just black and white, “used” vs “free”. Rather, there are a few states, such as cached and buffered, in addition to used and free. Each of these states has a specific purpose — buffered memory is used for block devices, while cached memory is used for disk objects, to speed up access. Free and used are just as they sound.

The catch, however, is that both cached and buffered memory can be released instantly, should an application or the system require more memory just to run. For all intents and purposes, both cached and buffered memory can be considered “free”, even though they’re actively in use to speed up the running applications by reducing the amount of disk accesses.

top breaks down all of this, whereas Webmin abbreviates this information. The actual “free” memory is, essentially, used minus buffers minus cached. Right now, for example, your server is reporting 2007MB total RAM, 108MB of which is free. 42MB is buffered and the majority, 1612MB is cached — while only 108MB is completely free, 1763MB is available to be freed. Completely free memory is a waste of the fastest medium available in your server, and Linux makes sure to take advantage of it!

Adding a new drive on an HP RAID controller

Launch `hpacucli’, the RAID interface and run the following:

[code lang=”bash”]controller slot=0 pd show all[/code]

All drives will be shown — it’s obvious which ones are not configured as a logical disk.

[code lang=”bash”]controller slot=0 create type=ld drives=2:2[/code]

We’ve only added one drive, and the controller is smart enough to know that making a logical disk out of one drive will be a RAID-0, thus no RAID level is specified. If using multiple drives, list them with commas (drives=2:2,2:3,2:4) and add a “raid” command:

[code lang=”bash”]controller slot=0 create type=ld drives=2:2,2:3,2:4 raid=?[/code]

Using the ? will tell you what RAID levels are available for the drives selected. Specify the RAID level with “raid=5″.

[code lang=”bash”]controller slot=0 ld show all[/code]

Will show your new logical disk composed of the previously unused drives. You may need to run `partprobe’ on the server to enable it to see the new logical disk, but thereafter you are free to partition away and do what must be done.

iftop

Very handy utility to see exactly what’s going over your intertubes. Requires libpcap (obviously).

iftop

Mount an HFS+ drive in linux

Modern kernels have a module for HFS+, but Apple used BSD-style partitioning, which linux’s fdisk can’t handle.

Try pdisk, an almost-10-year-old program that still builds well on modern linuxes. Download, untar, and make.

[code lang=”bash”][root@web1 pdisk]# ./pdisk /dev/sdb
Edit /dev/sdb –
Command (? for help): p

Partition map (with 512 byte blocks) on ‘/dev/sdb’
#: type name length base ( size )
1: Apple_partition_map Apple 63 @ 1
2: Apple_Free 262144 @ 64 (128.0M)
3: Apple_HFS Apple_HFS_Untitled_1 2930035632 @ 262208 ( 1.4T)
4: Apple_Free 16 @ 2930297840

Device block size=512, Number of Blocks=2930297856 (1.4T)
DeviceType=0x0, DeviceId=0x0

[root@web1 pdisk]# mount -t hfsplus /dev/sdb3 /mnt/usbdrive

[root@web1 pdisk]# ls /mnt/usbdrive/
Directory1 Directory2 [/code]

Bash portscanner

Well, why not?

[code lang=”bash”]HOST=127.0.0.1;for((port=1;port<=65535;++port));do echo -en "$port ";if echo -en "open $HOST $port\nlogout\quit" | telnet 2>/dev/null | grep ‘Connected to’ > /dev/null;then echo -en “\n\nport $port/tcp is open\n\n”;fi;done[/code]

Install OpenManage on RHEL servers

[code lang=”bash”]wget -q -O – http://linux.dell.com/repo/hardware/bootstrap.cgi | bash
up2date -i srvadmin-all
service dataeng start[/code]

Find total file sizes

[code lang=”bash”]find /var/www/vhosts/*/statistics/logs -type d -exec du -sm {} \; | awk ‘{total+=$1} END {print total,”MB”}’ [/code]
Find total sizes of files in all those logs directories

Add a system user as a Webmin user

Edit the user file:
[code lang=”bash”]vi /etc/webmin/miniserv.users[/code]

Enter the system user’s name, followed by :x:
[code lang=”bash”]kale:x[/code]

Edit /etc/webmin/webmin.acl to give access to this new user

Restart Webmin

Recover an ext3 journal

Symptom:
dmesg scrolling with “journal aborted”, filesystem in read-only

Give this a go (may need a rescue environment):
[code lang=”bash”]tune2fs -f O ^has_journal /dev/sda1
tune2fs -j /dev/sda1[/code]

Recover ext3 filesystem with missing superblock

Symptom:
[code lang=”bash”]mount: wrong fs type, bad option, bad superblock on /dev/sda1, or too many mounted filesystems[/code]

Usually, this is code for “you’re fucked”. Here’s something you can try, however:

List the proposed superblocks (filesystem must be unmounted):
[code lang=”bash”]mke2fs -n /dev/sda1[/code]

fsck the filesystem using a backup superblock (caution, should try with -n switch to fsck first):
[code lang=”bash”]fsck -b 24577 /dev/sda1[/code]

If it fails, scan for the superblocks and use one of those:
[code lang=”bash”]dumpe2fs /dev/sda1 |grep super[/code]

Then again, if using a backup superblock doesn’t work, you’re probably fucked, as originally thought.

Find files that do not contain a string

To find files that do NOT contain a specific string you can do the following:
[code lang=”bash”]find -name “ifcfg-eth0:*” -type f ! -exec grep -q ONBOOT {} \; -exec ls {} \;[/code]

will list all files named ifcfg-eth0:* that do not contain the string ONBOOT.

You can script this up as such:

[code lang=”bash”]for i in `find -name “ifcfg-eth0:*” -type f ! -exec grep -q ONBOOT {} \; -exec ls {} \; |awk -F\/ ‘{print $2}’`; do echo ONBOOT=yes >> $i ; done[/code]

to add the required ONBOOT=yes line to the config.

MegaMon RAID monitoring for MegaRAID-based cards

Cleverly hidden RAID monitoring tool for MegaRAID cards. Creates a log file at /var/log/megaserv.log that spits out all kinds of useful data — patrol reads, battery cycles, SMART status changes, sense key changes, etc. Can be configured to email x address upon errors, such as… well… a failed drive, for example. Also installs MegaCtrl, which is a CLI interface to the RAID card and allows for scriptable actions, such as deleting a logical drive.

Installing MegaMon is easy, as it’s a standard RHEL rpm, contained within the PERC/CERC tools found here. Included in that tgz is the MegaMon rpm. Install and `service raidmon start` and you’re good to go!

Disk labels

See the current label (if any):
[code lang=”bash”]e2label /dev/sda[/code]

Set a disk label:
[code lang=”bash”]e2label /dev/sda /boot[/code]

Can use in fstab as follows:

[code lang=”bash”]LABEL=/boot /boot ext3 defaults 0 0[/code]

Word of warning: disk labels add another layer of abstraction — except it’s not really abstracting the device at all. Note that some operations, such as a ghost, will not take into account disk labels and they will not be copied.

Custom drivers in the RHEL rescue environment

If one or more of devices has a custom driver not present in the rescue environment, put them on an external media source such as a thumb drive, floppy, CD, etc. Once at the rescue environment prompt, start it with

[code lang=”bash”]linux dd[/code]

Follow the prompts for success.

Syn flooding

Symptom:

Tons of the following in dmesg:

NET: 2348203 messages suppressed.
TCP: drop open request from ip
NET: 2392 messages suppressed.
TCP: drop open request from ip
etc, etc, etc

To put a quick halt to the attack, enable syn_cookies:

[code lang=”bash”]echo 1 > /proc/sys/net/ipv4/tcp_syncookies[/code]

This doesn’t persist after a reboot. To make it persist, edit /etc/sysctl.cnf as follows:
net.ipv4.tcp_syncookies = 1