Codex

Interested in functions, hooks, classes, or methods? Check out the new WordPress Code Reference!

User:Hakre/Grep And Friends

Back to my Page

Grep and Friends

From time to time it is good to have a shell and command-line at hand to do various tasks. From file and directory operations, doing Mysql queries and database maintenance, packing and unpacking files. I start this page to collect useful *nix based command-line tips that were handy for me.

So even if you're not a Linux system administrator, you should have no fear to actually use the shell. It does provide a lot of powerful commands that are easy to learn and to use. Just ensure when it comes to editing or deleting and multiple file operations, that you've got a backup at hand. But backup should be one topic of this document.

This page contains shell commandos to download, extract and remove Wordpress as well as useful sequences to find malicious code in Wordpress as well as to remove payloads and worms from files.

Commands used in this Document

A quick overview of the commands used in this document (from A-Z):

  • find searches a directory tree for filenames.
  • gzip (GNU zip) is a compression utility and is now relatively popular on the Internet.
  • rm (short for remove) is a basic Unix command that is used to delete files and directories.
  • sed (short for stream editor) is a Unix utility that parses text files and can transform those (for example search and replace).
  • tar provides the ability to operate tar archives.
  • wc (short for wordcount) counts a file's words, lines, characters and such.
  • wget is for retrieving files using HTTP, HTTPS and FTP.

The commands and their options are explained when their usage is suggested. Those commands do display helpful information when you invoke them with the --help switch, for example

~ rm --help

will display usage information for remove. Next to this, all of these commands are free software and you can find a lot of information about them in the Internet.

Download, Install and Uninstall Wordpress

Download and extract Wordpress

A quick Wordpress download and unpacking can be done via shell. I'll show every single command and you'll find a one-liner at the end.

~  wget -nv http://wordpress.org/latest.tar.gz

That command will download the file, -nv is in there to reduce the output (non-verbose). You can use it w/o that switch if you like to see a progress bar. Now the package needs to be unpacked. The trick here is to extract it to the current directory. Because the tar.gz file has all filenames perpended with wordpress/ this first path component needs to be stripped while unpacking:

~ gzip -dc latest.tar.gz | tar --strip=1 -xf -

That's it. wordpress has been downloaded and installed. These commands can be compacted so that everything can be done in a single line and without leaving the tar.gz file around:

~ wget -qO- http://wordpress.org/latest.tar.gz | tar --strip=1 -xz

For example to download and uncompress wordpress into a a subdirectory called blog run the following commands:

~ mkdir blog && cd blog && wget -qO- http://wordpress.org/latest.tar.gz | tar --strip=1 -xz

Keep in mind that this is faster than FTP'ing normally, because you do the FTP from your desktop system and both the wordpress.org as your webserver are often better connected to the internet than your desktop computer.

Remove Wordpress

While you play around with commands, you might want to delete an extracted wordpress archive:

~ rm -r wp-* xmlrpc.php index.php readme.html license.txt

That rm (remove) command removes all wp-directories and the basic files. Keep in mind that this also deletes all themes and plugins as well as your configuration file (wp-config.php). So use it with care.

Backup Wordpress

It's easy to run a full Wordpress backup in shell. All you need to do is to pack all files and dump the database. That's it.

Backup and Restore Files

To backup files, you can use an archive. For example a tar.gz file. It stores the paths, file meta-data like date and ownership and can compress the files. This is an example how to archive and compress the current directory:

~ tar cfz files.tar.gz .

To extract from the archive again, use -x or --extract and name the archive file:

~ tar xzf files.tar.gz

Backup and Restore Database

To backup the database, the weapon of choice is mysqldump. Providing the hostname, username and password and the databasename you can dump and pack database data and structure:

~ mysqldump --host=localhost -u dbuser --password=dbpass dbname | gzip > dbname.sql.gz

I've written the command in a form that needed data can be easily selected and replaced with real values. Restoring the database can be done by first using gzip unpack and mysql to read the dump in:

~ gzip -cd dbname.sql.gz | mysql --host=localhost -u dbuser --password=dbpass dbname

Hacked Blog

It was outch when I saw an older blog that had to be moved to a new server. All PHP files were changed just some days ago - but how could that be? It was a Wordpress 2.3.x blog so this was no miracle that it got hacked. But how can you determine if a blog is hacked and are there ways to heal it?

A popular form of attack is to place eval code segments into PHP code. So one thing you can to is to list all PHP files that contain "eval" and inspect the places. This is done with find and grep. Find will find all PHP files and grep will search these files. Additionally tools like wc (wordcount) help to summarize.

Grep does support options to traverse directories and filter files, therefore find is not always needed for that. But find does have an execution option that is powerful to ignite commands and chains of commands. I'll show that later on.

Find (infected) files with grep

To check how many php files contain eval in a directory (incl. subdirectories) just run:

~ grep -r --include=*.php -PHn "(eval\(.*\);)" . | wc -l

This will output a single number containing the number of lines. E.g. the hacked block did revealed a count of 512 lines containing the eval function. I used the -P switch here so it is possible to use Perl Regular Expressions which are common for PHP as well.

Running a quick install of the latest Wordpress version did reveal a count of 4 files containing eval. So a high number here is a sign that the blog has been compromised.

To actually see the eval-code, grep can be used to display the lines:

~ grep -r --include=*.php -PHn "(eval\(.*\);)" .

This is an example output of such a run on wordpress 2.9.2:

./wp-admin/includes/class-pclzip.php:4063:// ... eval('$v_result = '.$p_options[PCLZIP_CB_PRE_EXTRACT] ...
./wp-admin/press-this.php:423: ... eval(data);
./wp-admin/import/blogger.php:280: ... eval('this.status='+text);
./wp-admin/js/revisions-js.php:28:eval(function(p,a,c,k,e,r){e=function(c){return...

As you can see most of that is Javascript code within PHP files (I've cut off some of the output to make it more readable). To only display the list of files that contain the regular epxression, the -l switch is handy:

~ grep -r --include=*.php -lPHn "(eval\(.*\);)" .

The output then looks like this:

./wp-admin/includes/class-pclzip.php
./wp-admin/press-this.php
./wp-admin/import/blogger.php
./wp-admin/js/revisions-js.php

To put all infected files into a tar.gt archive for further analysis, pass the list of files with xargs to tar:

~ grep -r --include=*.php -Hncl "eval\(.*\);" . | xargs tar zcf analyse.tar.gz

You should then transfer the file to your desktop and use it for analysis/reference.

Replace in files with sed

It's done with sed. This section needs to be written, this is an example so far:

~ find . \( -name "*.php" \) -exec grep -Hn "<\?php /\*\*/eval(base64_decode(.*));.*\?>" {} \; -exec sed -i '/<\?php \/\*\*\/eval(base64_decode(.*));.*\?>/d' {} \;

It makes use of the find command with the -exec switch, grep and sed.


Search and Replace Real Life Example

Example taken from sucuri.net to remove a current payload:

$ find ./ -name "*.php" -type f | xargs sed -i 's#<?php /\*\*/ eval(base64_decode("aWY.*?>##g' 2>&1
$ find ./ -name "*.php" -type f | xargs sed -i '/./,$!d' 2>&1

Other Useful Commands

Delete PHP session files modified more than two days ago. This is useful if PHP does not take care on it's own when using a configured directory:

~ find sess_* -mtime +2 -exec rm {} \;

Get Size of a directory and it's subdirectories (incl. total count):

~ du -hc

Get a Date/Time Stamp (incl. Nanoseconds):

~ date '+%F_%H%I%M_%S_%N'

Delete all .svn subdirectories:

~ rm -rf `find . -type d -name ".svn"`

Related