tech.superhappykittymeow.com

A coworker, Alex, discovered that Horde, in conjunction with Plesk, allows users to change their passwords arbitrarily — but doesn't enforce any sort of password policy, allowing such passwords as "test" or even "" (null). This, obviously, is a huge security risk as mail compromises can lead to fairly terrible things.

From his article:

If you (or a client you are representing) want to set horde to do the typical “strict password” enforcement, look for the file:

horde/passwd/backends.php

And read the bit about password policy. An example policy that can be set in this file that would require 1 capital, 1 lowercase, 1 special character and 1 number, with a minimum password size of 8, would look like:

‘password policy’ => array(
‘minLength’ => 8,
‘maxLength’ => 64,
‘maxSpace’ => 0,
‘minUpper’ => 1,
‘minLower’ => 1,
‘minNumeric’ => 1,
‘minSymbols’ => 1
),

Can't fork but need to see what's going on? Hint: a box that can't fork can often `exec'.

Here are a pair of slick bash functions that can be lifesavers in dire situations:

`ls':

`cat':

You can easily log how much memory each request for a PHP page takes by modifying the LogFormat:

Note the latter definition, which includes %{mod_php_memory_usage}n — this will print out the amount of memory, in bytes, required to execute the script as requested. Big help in finding memory leaks. To use, just change the log definition to the newly-created "combined-php" format:

Do note that this may (probably will) break Apache log parsers that are expecting the standard combined format. If using for troubleshooting, I recommend logging to an alternate location so as to not screw up log statistics.

Credit to Brad Ison for this find

Creates a much easier-to-read view of the output of "show table status":

Name                    Rows   Avg_row_length   Update_time
 wp_users                1      140              2009-08-08 04:13:07
 wp_links                9      106              2009-10-16 12:57:32
 wp_comments             14     464              2009-11-28 16:09:43
 wp_usermeta             15     166              2009-11-29 06:41:19
 wp_term_taxonomy        53     40               2009-11-20 14:06:21
 wp_postmeta             141    46               2009-11-29 06:44:05
 wp_options              172    4624             2009-11-29 06:40:59
 wp_term_relationships   357    21               2009-11-21 02:35:42 

I hacked up a Greasemonkey script tonight to address my very real problem with Youtube: Flash.

I use a Macbook Air and dearly love it, though I'm rather cognizant of the fact that it's powered by a dying hamster. Playing a video in Youtube will inevitably stutter or cause my lap to catch fire, both undesirable. There are also advertisements and Flash sucks up memory and CPU like crazy.

I attempted to address a different issue before arriving at this solution, namely, Youtube's new throttling algorithm by. I realized, however, that I'm simply not smart enough to figure that puzzle out, though I learned quite a bit about HTML5 and its <video> tag along the way.

I'd like to point out first that I'm not a JavaScript coder — I've actually never used it until tonight, so please don't judge too harshly. This Greasemonkey script is a mishmash of about 3 other Greasemonkey scripts and some regexes I used in a PHP script.

Install Youtube Flash-No-Mo

Do note that while the script works like a champ on Safari and Chrome (and likely any other webkit-based browser), Firefox 3.5 does not have a built-in handler for mp4 content. You may need to install an additional plugin (mplayer maybe?) to get things to work right on Firefox.

The PHP eval() function is an odd beast. I've encountered this a few times before, wherein a site will display just fine, but Google won't index it, and if you review the headers for the page (using curl or telnet), you'll see that instead of the expected 200 OK response, you'll get a "HTTP/1.0 500 Internal Server Error" — bad juju for all of your hard SEO work.

Taking a look at the site's error_log may reveal a PHP error — often a syntax error or some such. However, the line of code to which it refers is just an eval() statement for an included file.

If the eval'd code has an error, the code running the eval statement will work just fine; however, because of the error in the eval'd code, the response code header will return a 500. This, interestingly enough, will only occur if display_errors is set to off.

The ideal solution is to fix the code that is being eval'd.

The hackish way, which I prefer because I'm lazy, is to do something terrible such as:

The response code will now be 200 and Google will be happy; however, the problem isn't *really* solved…

This will print the whole line except for the first field (because you've set it to null).

If you've opened a file in vi for which you don't have write privileges, you can save the file as the superuser (if your user is in the sudoers list) by running the following vi command:

After doing so, vi will detect that the file has changed and ask if you want to reload it.

You can use the `sort' utility to sort not only on the first field, but also an arbitrary column. Take this output from ps:

This can be sorted by the SZ column, column 9, as such:

The -k switch tells sort to sort based on a key, which we specify as 9n (column 9, numeric). Much easier to review the output.

Runs in the foreground a simple, single-threaded web server on port 8000 as the current user. Logging is to stdout/stderr, and a ctrl-c will stop the server. Great for temporarily sharing a directory.

Great for command-line logging into sites to pull content for whatever reason.

Of course, you'll have to look at the source for the target location's login page to see what variables it wants. I use it to grab a single Cacti-generated graph that is normally password protected, but I want to include a single graph on another site, so I cron'd a script to run a line similar to the above to log in and save it locally.

Don't do it.

While there's no technical file limit on a per-directory basis with ext3 (there is on a filesystem level — see inodes), there is *significant* performance degradation as you add more and more files to a single directory. I think the most I've seen without any user-noticeable sluggishness was about 300,000. Note that this is well beyond the point where you can't `ls' anymore and you have to resort to `find' and `xargs'. This should be your first warning sign.

Approaching 5 million files in one directory, things start to get weird. Creating new files in that directory generates significant load, though resource usage is low. However, statting a specific file (as opposed to a lookup with a glob) is decently fast. As long as you know what you want, it'll work acceptably fast.

The more files you add, the slower lookup-based operations (new file creation, for example) will go — we're talking seconds and tens of seconds here, not a few milliseconds more. As long as you give it an exact filename, though, it will be of an acceptable speed.

The filesystem option dir_indexes will help, though not hugely once you start getting into millions of files. Compared to no dir_indexing, it's faster, but it doesn't make it magically work. Converting to ext2 is a terrible idea and should not be considered — journals are good things and well worth the extremely slight (comparatively) performance hit endured.

The real solution, however, is to not put that many files into a single directory. Subdirectories are always a good idea (though keep in mind the subdirectory limit — 32k subdirs per dir!). Heck, most code can almost trivially be modified to pull content from a hash from the filename, such as /var/www/images/y/o/yourmom.jpg and /var/www/images/y/i/yipee.jpg. When designing an application, one should be mindful of the limitations of the underlying OS (and in this case, the filesystem being used).

Moving a Plesk server behind a firewall is always a pain, since the IPs are associated with domains within the Plesk database. I used to hack the database every time I had to update IPs, but doing this for 50 IPs is… not so good.

Luckily, I stumbled up on this Parallels knowledge base article, which introduces reconfigurator.pl — it reads a mapping of IPs and updates system interfaces as well as all the internal Plesky goodness.

Had a nasty one tonight that I had to turn over to Paul. up2date was segfaulting on a RHEL3 server, along with some rpm queries, such as `rpm -qa'. I was able to narrow it down to a specific package, though that did not help at all. The rpm database had some severe corruption.

I removed the __db.00* files from /var/lib/rpm and rebuilt the database with rpm --rebuilddb; however, this did not resolve the issue. I attempted to recreate the database, by doing the following:

However, the `rpm -qa' hung, as noted earlier. Tough spot.

I had to move on, but Paul dove in. After a few hours of fitful hacking, he declared himself the winner — he had solved it. What did he do?

He removed all of the files in /var/lib/rpm except for Packages, then ran `rpm --rebuilddb'. The __db.00* files are just lock files, and while removing them can help, rpm transactions and queries still read all the other files and databases, thus rereading the corruption. Removing all the other databases (except the base Packages database) and then running a --rebuilddb operation actually rebuilds the databases… and the corruption cleared.

I also found this neat snippet to see if anything has a lock on the rpm databases:

ftp://ftp.cisco.com/pub/mibs/oid/

Go forth and monitor!

Surprisingly hard to find on HP's site if you just need a copy of hpacucli.

HP Insight Diagnostics Online Edition for Linux

Set bash's internal field seperator to an enter instead of a space, so grep (or whatever) doesn't freak out:

This is a prefab for me to paste into tickets whenever a customer is confused about "free" and "used" memory.

Memory in Linux isn't just black and white, "used" vs "free". Rather, there are a few states, such as cached and buffered, in addition to used and free. Each of these states has a specific purpose — buffered memory is used for block devices, while cached memory is used for disk objects, to speed up access. Free and used are just as they sound.

The catch, however, is that both cached and buffered memory can be released instantly, should an application or the system require more memory just to run. For all intents and purposes, both cached and buffered memory can be considered "free", even though they're actively in use to speed up the running applications by reducing the amount of disk accesses.

top breaks down all of this, whereas Webmin abbreviates this information. The actual "free" memory is, essentially, used minus buffers minus cached. Right now, for example, your server is reporting 2007MB total RAM, 108MB of which is free. 42MB is buffered and the majority, 1612MB is cached — while only 108MB is completely free, 1763MB is available to be freed. Completely free memory is a waste of the fastest medium available in your server, and Linux makes sure to take advantage of it!

Very handy utility to see exactly what's going over your intertubes. Requires libpcap (obviously).

iftop

Well, why not?

Symptom:
dmesg scrolling with "journal aborted", filesystem in read-only

Give this a go (may need a rescue environment):

Symptom:

Usually, this is code for "you're fucked". Here's something you can try, however:

List the proposed superblocks (filesystem must be unmounted):

fsck the filesystem using a backup superblock (caution, should try with -n switch to fsck first):

If it fails, scan for the superblocks and use one of those:

Then again, if using a backup superblock doesn't work, you're probably fucked, as originally thought.

Cleverly hidden RAID monitoring tool for MegaRAID cards. Creates a log file at /var/log/megaserv.log that spits out all kinds of useful data — patrol reads, battery cycles, SMART status changes, sense key changes, etc. Can be configured to email x address upon errors, such as… well… a failed drive, for example. Also installs MegaCtrl, which is a CLI interface to the RAID card and allows for scriptable actions, such as deleting a logical drive.

Installing MegaMon is easy, as it's a standard RHEL rpm, contained within the PERC/CERC tools found here. Included in that tgz is the MegaMon rpm. Install and `service raidmon start` and you're good to go!

See the current label (if any):

Set a disk label:

Can use in fstab as follows:

Word of warning: disk labels add another layer of abstraction — except it's not really abstracting the device at all. Note that some operations, such as a ghost, will not take into account disk labels and they will not be copied.

If one or more of devices has a custom driver not present in the rescue environment, put them on an external media source such as a thumb drive, floppy, CD, etc. Once at the rescue environment prompt, start it with

Follow the prompts for success.