Category: PHP Programming

Categories
PHP Programming

Getting Web Scale with Memcached

The web is huge, and there are a lot of people on it. Day and night, millions upon millions of people are on the web surfing, commenting, and contributing. Normally your blog gets a few hundred visitors a day ( a few thousand on a good day ), but what happens when that number increases? Can your database server handle all that load? Will Apache come screeching to a halt due to all of the requests? The answer is probably yes, unless you implement some form of caching. Many years ago this wasn’t a huge problem, but as the web and it’s user base has grown, so has the problem of “web scale”.

Memcached

Memcached is a pretty simple concept. Just as the name implies, it’s a caching system that stores stuff in memory. That’s really all you need to know to get started. If you’re interested in learning more, check out the Memcached home page.

PHP Memcache

As this is mostly a PHP blog, I’m going to show you how to use Memcached with the PHP Memcache module. While this tutorial is language specific, the concepts here can be applied to any language to increase the speed of your web pages. That being said, the first step is to get Memcached installed on your machine. There are a ton of tutorials out there on the web for this, so I’m going to leave that as an exercise for you. Once that’s installed, you should check out my guide for getting the PHP Memcache module installed on XAMPP, that way you can run this tutorial locally.

Step 1: Make the connection

This step is pretty straight forward. If you can’t connect to your caching server, you can’t cache. If the connection is successful, continue trying to cache. Otherwise, just query your database as normal.

$memcache = new Memcache;
$memcache->connect("my.memcached.server", 11211);

Step 2: Cache something

For this step, the only potential “gotch-ya” is that the your identifier must be unique, and time to expire is in seconds.

$myValue = "hello world!";
$memcache->set("Hello World", $myValue, false, 60*60*24);

Step 3: Retrieve an item from the cache

$myValue = $memcache->get("Hello World");
echo $myValue;

Step 4: Putting it all together

So how does all this work in conjunction with your web app? The basic workflow for using caching is the following:

  1. Does my item exist in cache> (This satisfies determining if a connection to the cache has been made as well.)
  2. If so, get the item and store in a variable.
  3. If not, get the item from the database.
  4. Store the item for later use.
$memcache = new Memcache;
$memcache->connect("my.memcached.server", 11211);
 
$arrayVals = $memcache->get("My Identifier");
if(!$arrayVals) {
        //Note: This assumes that the data in the table doesn't change
       // and that it is fairly small in size.
	$query = "SELECT * FROM myTable";
	$result = mysql_query($query);
	while($row = mysql_fetch_array($result)){
		$arrayVals[] = $row;
	}
	$memcache->set("My Identifier", $arrayVals, false, 60*60*24);
}
 
foreach($arrayVals as $val) {
	print_r($val);
}

If you’re following carefully, you can see that the first time through the data will get pulled out of the database. However, for the next 24 hours the data will be coming from the Memcached server. It’s little tricks like this that can help your site survive being featured on Reddit. Moral of the story: If your site is slow because of volume, try caching almost everything and you should have noticeable improvements.

Categories
PHP Programming

Installing Memcache on MAMP

For the better part of 2 hours I tried and failed to get the Memcache extension installed on MAMP.  I tried following several guides, but everything fell flat on it’s face about 75% of the way through.  I eventually figured it out, and I wanted to share it so that other people don’t have to go through the pain and suffering that I did.  It turns out to be surprisingly easy, but YMMV.

Step 1: Install XCode

Step 2: Make the MAMP PHP binary files executable

sudo chmod +xrw /Applications/MAMP/bin/php5.3/bin/p*

Step 3: Get the Memcache extension source

cd ~
wget http://pecl.php.net/get/memcache-2.2.5.tgz
tar -zxvf memcache-2.2.5.tgz
cd memcache-2.2.5

Step 4: PHPize the Memcache extension files

/Applications/MAMP/bin/php5.3/bin/phpize

Step 5: Compile the Memcache extension

./configure
make

Step 6: Add the extension to the PHP.ini file

Add the following to the end of your PHP 5.3 ini file via the MAMP file menu.

extension=memcache.so

Step 7: Copy the memcache extension to the PHP extension folder

cp modules/memcache.so /Applications/MAMP/bin/php5.3/lib/php/extensions/no-debug-non-zts-[yourtimestamp]/

The “[yourtimestamp]” varies per installation, but you should just be able to tab complete that part.

Categories
PHP Programming Wordpress Development

PHP Alternative Syntax

Did you know that PHP has an alternative syntax structure?  Up until about two years ago, I didn’t either.  It wasn’t until I started poking around in the WordPress core that I saw it.  Intrigued, I popped over to PHP.net and read an entry on it.  In a nutshell, the alternative syntax can go far in making your PHP code much easier to read.  The control structures if, while, for, foreach, and switch can all be expressed in the alternative syntax form.  In general, I prefer to use the alternative syntax when mixing PHP in with HTML, and the standard syntax when writing pure PHP.

If Example

// This....
if($myString == "foo"):
    echo "bar";
else:
    echo "no-foo-bar";
endif;
 
//...is equal to this.
if($myString == "food") {
    echo "bar";
} else {
    echo "no-foo-bar";
}

ForEach Example

$names = array("bob", "tom", "john");
 
//This....
foreach($names as $name):
    echo "Your name is: {$name}";
endforeach;
 
//...is equal to this.
foreach($names as $name) {
    echo "Your name is: {$name}";
}

Resources

If you’re looking for more resources on PHP’s alternative syntax, check out the documentation here, or the WordPress source code for examples.

Categories
Other Other Programming PHP Programming

SVN Merge Example (Trunk to Branch)

I’m often in the situation where I need to merge changes from the trunk development into a branch.  I can never, EVER, remember the command, so I’m putting in here in hopes that I can reference it again and that other people might find it useful.

svn merge -r x:y svn://repositoryURL/repo/trunk/ .

So, how does it work? X is the revision that you branched at. This can also be the revision that you last merged changes from. Y is the version that you want to merge up to.  In most instances, HEAD is probably what you’re looking for.

Note:  Using –dry-run if you want to see the changes that will be made before actually doing it.  It takes some of the “this is scary” out of merging.

Categories
Other PHP Programming

Creating a Flexible Caching Module

I work for a great company, I really do.  Sure we have our problems (like just coming out of the developer stone age), but overall I work with smart, friendly people who are passionate about what they do.  One of the things that always bugged me though was the lack of any sort of caching in our CMS.  Most times it’s not needed, but when an operation takes about 100 queries or so to finish, then it’s time to start caching.

Since I’m a bit of an efficiency freak, I thought I would take a crack at writing a flexible caching module that is easy for our developers to use.  So what do “easy” and “flexible” mean?  To be “easy”, the caching module must be usable by even a novice developer and have a limited number of options.  For instance, I ended up deciding that we really only need two public methods, and one public property.

  • $cache->exists – If “$cache” is a cache object, calling exists checks to see if the cached object already exists in the database.  It also checks to see if it’s expired or not.  If it’s expired or non-existent, it returns false.   It returns true if the cached object exists and is up to date.
  • $cache->put($val) – This is how you store something in cache.  It can be any type of serializable PHP object.  So basically, resource types are off limits but objects, arrays, variables, entire web pages, etc. can be used.
  • $cache->get() – This fetches the object stored  in the cache.  It handles the re-serialization of it as well, so it really makes things pretty idiot proof.

What about flexible?  Well, by that I mean we need to be able to transparently implement several different types of caching.  Since we’re just crawling out of the dark ages, I opted to implement a fallback caching mechanism.  Here’s how it works.

  • The programmer defines a variable in our settings area to be which caching option he/she wants to use.  Options are memcached, file, mysql.
  • If the setting isn’t defined, we try memcached by default.  This is by far the best caching system to use, so it makes sense to try it first.
  • If memcached fails, we go to a database caching schema.  While not nearly as good as using memcached, it’s possible it could save you tons of queries on your database.
  • If the user chooses file caching, we do that.  It’s a pretty bad idea to use in most cases, but may still have it’s uses.

So why did it come this?  It’s not that we host terribly high-volume sites, but that our CMS is super slow.  A full-on page will take about 4 seconds to load to your screen completely, and that’s running local on the network.  One of the main problems is that we use output bufferring extensively.  The ENTIRE FRIGGIN PAGE is buffered.  This has 3 side effects:

  1. Slight performance loss due to bufferring.
  2. Apparent page load time sucks because the browser has to wait for the entire page to be generated before getting output.
  3. Development is super easy because you don’t ever have to worry about output being sent before doing a call like “header()”.

We can’t remove output buffering unfortunately.  It’s at the very core of our CMS and development practices, so it just won’t work.  To get the load time to generate the page as low as possible, I decided that caching was needed.

So what sort of problems do we run in to with this caching module?  Glad you asked!  Many of the problems aren’t specific to this caching module, but to caching in general.  The quick list:

  • If the original query wasn’t complicated, it’s not worth storing the results.  The number of queries the caching module does in MySQL mode is 3.  If your initial query was less than that, or not a super-complex-mega-join, it’s not worth using.  This caveat goes away in memcached mode.
  • Smart naming and design.  You have to be very careful out what you cache, and when.  Remember, page content and queries probably change when a user is logged in or on a different device.  Just things to keep in mind.
  • Getting developers to use it.  Not everyone likes to learn, let alone change their habits.  The biggest barrier to this is getting people to use it.  Some people don’t care about efficiency either (sad, I know), but at least our system administrator thanks me.

The caching module seems to work pretty well too.  On one particularly SQL heavy page, I reduced page load time from 14 seconds (ridiculous) to just under 6 seconds (still bad, but getting better).

That’s it for now.  Any questions or comments are welcome.

Categories
PHP Programming

PHP Bitwise Operations

A bitwise operation is an operation that works on the individual bits of a number.  Yes, that means the binary(base 2) representation of a number.  These bitwise operations work by exploiting properties of binary numbers to their advantage.  For instance, if the binary representation of a number ends with a 1, it’s odd.

Base 2 Refresher

For those of us who don’t manipulate bits every day (including myself), I thought that a quick refresher on binary representation would be a good idea.  First, let’s create a table.

4 3 2 1 0
0 0 0 0 1

Each column in the table represents a power of 2.  The first column (from the right) represents 20, the second 21, and so on.  Notice that I have a 1 in the row below the 0 column.  That stands for 20, and 20 is equal to 1.  Therefore, the number represented above (00001) is equal to 1.  On to the next table.

4 3 2 1 0
0 0 1 1 1
Now the base-2 number in the bottom row is 00111.  This is equal to 22 + 21 + 20, or 4 + 2 + 1, which is equal to 7.  As you can see, using binary representation is pretty easy.  On to our final example.
4 3 2 1 0
1 1 0 0 1
This example a bit (har, har) trickier.  The base-2 number in the bottom row is 11001, which is  24 + 23 +20(16 + 8 + 1) = 25.  Notice if there are zeros in the columns, we just ignore them completely.
And that’s it!  Base-2 representation of numbers is easy, and critical for every programmer to understand.  Now on to some tricks.

Trick 1 : Odd / Even

One of the most useful bitwise tricks for web developers is for determining if a number is even or odd.

for($i = 0; $i < 10; $i++) {
     if($i & 1) {
          echo "This is odd.";
     } else {
          echo "This is even.";
     }
}
So how does help your web development?  Alternating rows.  If you deal with data in a tabular format, you nearly always need to alternate row color on a table.  Using the bitwise “AND” operator is a good way to accomplish this.

Trick 2 : Multiplication and Division

Using the left-shift (<<) and right-shift (>>) operators, we can easily divide and multiple by powers of two.  These operators aren’t limited to powers of two, but it makes things easier to understand.
//Example 1
$x = 2;
echo $x &lt;&lt; 1; //4
echo $x &lt;&lt; 2; //8
 
//Example 2
$x = 16;
echo $x &gt;&gt; 1; //8
echo $x &gt;&gt; 2; //4

In example 1, we left shift the number 2 one place.  By left shifting 1 place, it’s the same as multiplying that number by 2.  By left shifting 2 places, it’s the same multiplying the number by 2 twice.

Example 2 is doing bitwise division.  Right shifting by 1 place is the same as dividing the number by 2.  Right shifting by 2 places, is the same is the same as dividing the number by 2 twice.

Others

There are most definitely a ton of other uses for bitwise operators, however these are the ones I use the most.  If you happen to have a favorite, please let everyone know about it in the comments.

For a more in depth introduction to bitwise operations please check out http://en.wikipedia.org/wiki/Bitwise_operation and http://www.litfuel.net/tutorials/bitwise.htm

Categories
PHP Programming

PHP Dark Arts: Daemonizing a Process

Note:  Full source code for the example can be downloaded here.

One of the many things you don’t often do with PHP (actually, I’m not sure you do this much with any language) is daemonize a process.  A daemon is program that runs in the background (read more here).  On Unix systems, processes are usually created by forking the init process and then manipulating the process to your liking.  To create a daemon though, you need to get the init process to adopt your process.  To do that, as soon as you fork the parent process, you kill the parent.  Since you child process is parent-less, the init process generally adopts it.  Once that happens, your process has been daemonized.

What You Need To Know

In order to follow the example, you’ll probably want to read up on multi–processing in PHP and using POSIX in PHP.  Aside from that, keep an open mind.  There are probably better ways to do this (Nanoserv), but I think that doing it manually is a great way to learn more about systems programming and PHP.

Step 1:  Fork It

The first thing that you need to do when daemonizing a process in PHP is fork the process.  After that, we promptly kill the parent process so that the child process can be adopted.

//Set the ticks
declare(ticks = 1);
 
//Fork the current process
$processID = pcntl_fork();
 
//Check to make sure the forked ok.
if ( $processID == -1 ) {
	echo "\n Error:  The process failed to fork. \n";
} else if ( $processID ) {
	//This is the parent process.
	exit;
} else {
	//We're now in the child process.
}

Step 2:  Detach It

Now that we have successfully forked the process and killed the parent, we need to detach the process from the terminal window.  We do this so that when the terminal window closes, our process doesn’t close with it. Once that’s done, we get our processes’ id.

//Now, we detach from the terminal window, so that we stay alive when
//it is closed.
if ( posix_setsid() == -1 ) {
	echo "\n Error: Unable to detach from the terminal window. \n";
}
 
//Get out process id now that we've detached from the window.
$posixProcessID = posix_getpid();

Step 3:  /var/run

Now that we have our processes’ id, we need to let the system know about it.  To do this, we create a file in /var/run.  This file can be named anything you want, just make sure that it’s unique and it ends with the .pid extension.  In that file, we place the pid of our process and that’s it.

//Create a new file with the process id in it.
$filePointer = fopen( "/var/run/phpprocess.pid" , "w" );
fwrite( $filePointer , $posixProcessID );
fclose( $filePointer );

Step 4:  Do Something

Now that all the hard stuff is done, you can get to work.  What do you want your process to do?  For this example, I have mine adding 1 + 1 every 10 seconds.  Nothing too difficult, but you can make your process do whatever you like.  For instance, you could set up a server of some sort.  Note that this the code is sitting in an infinite while loop.  This is done so that the process doesn’t exit naturally.

//Now, do something forever.
while (true) {
	$x = 1 + 1;
	sleep(10);
}

Step 5:  Run It

To run this process as a daemon, all you need to do is save you file and run php <myfile>.php.  You may need to execute it as a super user depending on how you permissions are set up.  Once it’s run, you can check out the results of your hard work by running ps aux | less.  Scroll to the bottom and your process should be there.  In the screen shot below, mine is 3rd from the bottom.

Daemonizing a ProcessNote:  Full source code for the example can be downloaded here.

Did you like this article?  Check out the rest of the Dark Arts.

Categories
PHP Programming

PHP Dark Arts: Sockets

Note:  Complete example code for this article is available here.

As an Internet user, you take part in many client-server relationships on a day to day basis.  Most of these relationships are abstracted away, but what if you wanted to make your own client?  Or your own server?  Well, if that’s the case then you’ll probably need to know some network programming.  What you’re really going to need is sockets.

A socket is a receptacle that provides a means of communication between two processes (or in this case, two computers).  Basically it allows you to accept or send information on any port you please (so long as they aren’t reserved or already in use).  So how are we going to use sockets?  Keep reading to find out.

The Server

There are a ton of different examples I could do for this, but I’m choosing to keep it simple.  The server is going to do the following:

  1. Create a socket on a specified port using socket_create_listen.
  2. Wait for incoming connections using socket_accept.
  3. Read data from the socket using socket_read.
  4. Echo the data, and then close the socket using socket_close.

Basically this amounts to creating a server, fetching data from a client, and dieing.  The code itself is very simple, so take a look.

//Set up some variables
$maxBytes = 5000;
$port = 33333;
 
//Create the socket on the specified port.
$socket = socket_create_listen($port);
 
//Wait for incoming connections.
$connection = socket_accept($socket);
print "Connection accepted\n";
 
//When a connection has been accepted, read up to $maxBytes
//then print the received message.
$bytes = socket_read($connection, $maxBytes);
echo "Message From Client: $bytes \n";
 
//Close the socket
socket_close($socket);

Notice that I set the port that I want the socket to bind on very high.  This generally a good idea, because lower ports are regularly used by other applications.  I also have the maximum number of bytes to read set fairly high.  I did this to simplify the example.

The Client

The client for this example is just as simple as the server.  It’s going to do the following.

  1. Bind to a socket using socket_create.
  2. Using the socket that was just created, connect to the server’s socket using socket_connect.
  3. Send a message to the server using socket_send.
  4. Close the connection using socket_close.
//Create a socket and connect it to the server.
$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
socket_connect($socket, 'localhost', 33333);
 
//Create a message, and send it to the server on $socket.
$message = "This is a message from the client.\n";
socket_send($socket, $message, strlen($message), MSG_EOF);
//Close the socket.
socket_close($socket);

Running The Client & Server

Running the code is easy.  Save your server code to socket_server.php and save your client code to socket_client.php.  After that, open two terminal windows.  In one window run php socket_server.php and in the other run php socket_client.php.  Your output should look like the following image.

PHP Sockets

Please ignore the library error, that’s actually unrelated to this example.  Look at the line above it.

Note:  Complete example code for this article is available here.

Did You Like This Article?

You’ll probably like these other articles from the Dark Arts series as well.