Categories
PHP Programming

Getting Web Scale with Memcached

The web is huge, and there are a lot of people on it. Day and night, millions upon millions of people are on the web surfing, commenting, and contributing. Normally your blog gets a few hundred visitors a day ( a few thousand on a good day ), but what happens when that number increases? Can your database server handle all that load? Will Apache come screeching to a halt due to all of the requests? The answer is probably yes, unless you implement some form of caching. Many years ago this wasn’t a huge problem, but as the web and it’s user base has grown, so has the problem of “web scale”.

Memcached

Memcached is a pretty simple concept. Just as the name implies, it’s a caching system that stores stuff in memory. That’s really all you need to know to get started. If you’re interested in learning more, check out the Memcached home page.

PHP Memcache

As this is mostly a PHP blog, I’m going to show you how to use Memcached with the PHP Memcache module. While this tutorial is language specific, the concepts here can be applied to any language to increase the speed of your web pages. That being said, the first step is to get Memcached installed on your machine. There are a ton of tutorials out there on the web for this, so I’m going to leave that as an exercise for you. Once that’s installed, you should check out my guide for getting the PHP Memcache module installed on XAMPP, that way you can run this tutorial locally.

Step 1: Make the connection

This step is pretty straight forward. If you can’t connect to your caching server, you can’t cache. If the connection is successful, continue trying to cache. Otherwise, just query your database as normal.

$memcache = new Memcache;
$memcache->connect("my.memcached.server", 11211);

Step 2: Cache something

For this step, the only potential “gotch-ya” is that the your identifier must be unique, and time to expire is in seconds.

$myValue = "hello world!";
$memcache->set("Hello World", $myValue, false, 60*60*24);

Step 3: Retrieve an item from the cache

$myValue = $memcache->get("Hello World");
echo $myValue;

Step 4: Putting it all together

So how does all this work in conjunction with your web app? The basic workflow for using caching is the following:

  1. Does my item exist in cache> (This satisfies determining if a connection to the cache has been made as well.)
  2. If so, get the item and store in a variable.
  3. If not, get the item from the database.
  4. Store the item for later use.
$memcache = new Memcache;
$memcache->connect("my.memcached.server", 11211);
 
$arrayVals = $memcache->get("My Identifier");
if(!$arrayVals) {
        //Note: This assumes that the data in the table doesn't change
       // and that it is fairly small in size.
	$query = "SELECT * FROM myTable";
	$result = mysql_query($query);
	while($row = mysql_fetch_array($result)){
		$arrayVals[] = $row;
	}
	$memcache->set("My Identifier", $arrayVals, false, 60*60*24);
}
 
foreach($arrayVals as $val) {
	print_r($val);
}

If you’re following carefully, you can see that the first time through the data will get pulled out of the database. However, for the next 24 hours the data will be coming from the Memcached server. It’s little tricks like this that can help your site survive being featured on Reddit. Moral of the story: If your site is slow because of volume, try caching almost everything and you should have noticeable improvements.

Categories
Other

Switching from Google to Duck Duck Go

For the longest time I’ve been hearing the praises of a little search engine called Duck Duck Go amongst the Hacker News crowd. Yesterday, I finally decided to take the plunge and set it as Chrome’s default search engine. After a day of solid use, here are some of my observations:

  • The search results are good: While Google has been taking time to improve their results lately, it’s refreshing to see original content get ranked higher than web scrapers. In fact, the web scrapers have a tendency to not show up at all on DDG.
  • Lots of documented goodies: I’m still getting my feet wet with DDG, but the ridiculous amount of goodies is going to make things a lot more enjoyable.
  • I like not having page previews by default: I’m not sure if DDH even supports this, but I absolutely HATE having preview panes pop up in Google by default. Can it be turned off? Yes. Am I too lazy to do it? Yes.
  • Directly search other sites: You can search other sites directly, which is a nice feature. Try “!amazon Founders at work”.
  • They don’t track you: You know that feeling you get when you think someone is following you down a dark alley at night? That’s the feeling you should get using Google. They track everything. I don’t like being tracked, so DDG is probably going to become my permanent search engine. Read
  • Instant Answers: Sometimes you don’t need to click-through to a page. For instance, search “jquery” and you get a handy little box that tells you what jQuery is, and where you can get more info about it. For someone new to jQuery, that little bit of information could help them make a more informed click to learn more.
  • I miss instant search: I would really like to see an option for instant searching on DDG. Being able to refine your search results letter by letter was really handy.
  • I miss other Google service integration: Searching for “coffee near 49503” would show a map with coffee houses on it in Google. In DDG, my results aren’t nearly as useful. I hope that some sort of map integration is in there future, because it would stop me from switching back to Google to use their map service.

What do you like about DDG? What features do you wish it had?

Categories
PHP Programming

Installing Memcache on MAMP

For the better part of 2 hours I tried and failed to get the Memcache extension installed on MAMP.  I tried following several guides, but everything fell flat on it’s face about 75% of the way through.  I eventually figured it out, and I wanted to share it so that other people don’t have to go through the pain and suffering that I did.  It turns out to be surprisingly easy, but YMMV.

Step 1: Install XCode

Step 2: Make the MAMP PHP binary files executable

sudo chmod +xrw /Applications/MAMP/bin/php5.3/bin/p*

Step 3: Get the Memcache extension source

cd ~
wget http://pecl.php.net/get/memcache-2.2.5.tgz
tar -zxvf memcache-2.2.5.tgz
cd memcache-2.2.5

Step 4: PHPize the Memcache extension files

/Applications/MAMP/bin/php5.3/bin/phpize

Step 5: Compile the Memcache extension

./configure
make

Step 6: Add the extension to the PHP.ini file

Add the following to the end of your PHP 5.3 ini file via the MAMP file menu.

extension=memcache.so

Step 7: Copy the memcache extension to the PHP extension folder

cp modules/memcache.so /Applications/MAMP/bin/php5.3/lib/php/extensions/no-debug-non-zts-[yourtimestamp]/

The “[yourtimestamp]” varies per installation, but you should just be able to tab complete that part.

Categories
Python

Django Power!

Over the past month I’ve been dabbling with Django by starting to convert the prototype of Mail Tally (PHP) into a useable site powered by it.  So far I have to say that I love it!  The template inheritance is mind blowing, and the ease of mapping URLs to views just blows my mind.

It’s embarrassing to say that I’ve never used a proper PHP framework before.  At my day job we have an in-house CMS that has some MVC leanings, and prior to that I’ve done a lot of work in WordPress.  It’s refreshing to write web apps with a framework that makes it fun.

Categories
Other

Startup Weekend West Michigan: The Pitch

This weekend I’ll be attending Startup Weekend West Michigan from 6pm Friday through 4pm Sunday.  It promises to be an awesome event, but up until yesterday I had no idea to pitch.  So I got to thinking, and came up with this idea.  What is said below isn’t my pitch, but it’s what I’ll be pitching about.  It’s kind of a far-reaching idea to put in to 2 minutes, but I think I can manage.

Your online social life and your local community are disconnected.  Maybe some people want it to stay that way, but a lot of people want that connection made for them.  I’m calling this idea, “Social Gone Local”.

It’s not that simple though.  Local businesses already have Facebook pages, Twitter accounts, and Check-in points on FourSquare.  The REAL question is, “How do I find new businesses to associate with?”.  Addressing that issue is the fundamental problem this startup will solve.

How do we solve it?  By aggregating all of the data from your social networks and building a profile of your interests, likes, and dis-likes.  With this information, we can recommend local activities, restaurants, shops, and events with a fairly good probability that you’ll like it.  Think “Netflix recommendation engine, for life”.

But what if you’re visiting a different city?  Change your location and we’ll give you recommendations for there too.

Categories
Other Programming

CoffeeScript with WebSQL and jQuery

Lately I’ve developed a distaste for Javascript.  I like what Javascript has done for the web, but I hate the syntax.  I hate that there are little “Gotch Ya!”‘s all over the language.  I don’t have to worry about that too much anymore though since I’ve started using CoffeeScript.  If you interested in learning more about it, check out the official web site, and then my “Getting Started” guide.

Now that I’ve had a chance to dive in to CoffeeScript a bit more, I’ve started to integrate what I’m doing with other features and libraries to see what I can create.  For work I’ve been using a lot of WebSQL and jQuery, so that was the first place I took my new found CoffeeScript powers to.

Using jQuery with CoffeeScript is really, really, easy.  For instance, let’s say we want to Ajax a page into an array:

$.get("mypage.html", (data) ->
    myArray.push data
)

The syntax changes just a hair, but overall it looks a lot cleaner.

As mentioned previously, the other thing I’ve been doing a lot at work recently was working with WebSQL.  It’s been dropped by the W3C, but Webkit has implemented it already so it’s here to stay.  Anyways, CoffeeScript makes WebSQL a little more palatable.

#Start a transaction
db.transaction( (tx) ->
    query = "SELECT * FROM table"
    tx.executeSql(query, [], (tx, results) ->
        rLength = results.rows.length
        for(i=0; i < rLength; i++)
            alert results.rows.item(i).someColName
    )
)

With WebSQL and CoffeeScript, the syntax doesn’t change a ton, but I like the look of it better without the braces.

Another feature that I wanted to share with out about CoffeeScript is it’s equivalent to PHP’s key, value loop construct.  In PHP, it would look like:

foreach($key => $value as $myArray) {
    //do stuff
}

In CoffeeScript, the equivalent is:

for own key, value of myArray
    #Do stuff.

The benefit of the CoffeeScript version is that it can loop over object properties as well, not just arrays.  If you have any other cool features or examples of CoffeeScript usage, drop a link (or example) into the comments.

Categories
Other Programming

Getting Started with CoffeeScript

Note: You may also be interested in “CoffeeScript with WebSQL and jQuery“, which gives more examples of CoffeeScript code.

Lately in web development circles there has been a movement to make Javascript easier for everyone. Well, easier is probably the wrong word.  Lets just say that they want to “modernize” it’s syntax.  A little language has been gaining a lot of attention lately that does just this, and it’s called CoffeeScript.  A quick example of assigned on a condition:

big = true
number = 100 if big

Instead of:

var big = true;
if(big) {
   number = 100;
}

Anyways, this tutorial is about getting the CoffeeScript compiler up and running. To do this, you’re going to need Node.js and the Node Package Manager(NPM).

Step 1: Node.js

Getting Node to install is pretty straight forward. All you need to do is download the latest stable version from here and then run:

./configure
make
make install (may need to run as super user)

Step 2: Node Package Manager

Node Package Manager is the equivalent of EasyInstall for Python. It allows a user to install pre-made programs, like CoffeeScript, easily without worrying about build dependencies or any annoying stuff like that.

If you have Curl installed (most Linux/Mac users will), the easiest way to make this happen is to do the following:

curl http://npmjs.org/install.sh | sh

I personally ran into problems with this because of some permissions errors. If you run into this, check https://github.com/isaacs/npm/blob/master/README.md out which has extensive documentation on how to get around it.

Step 3: Install Coffee

Now that we have all of Coffee’s dependencies solved, you can install CofeeScript by doing:

npm install coffee-script

Step 4: Brew Some Cofee

Let’s make a math object…

math =
	cube: (x) -> x * x * x
	square: (x) -> x * x
 
startNumber = 2
cubeStartNumber = math.cube(startNumber)
squareStartNumber = math.square(startNumber)
alert "Cube: " + cubeStartNumber
alert "Square: " + squareStartNumber

Now, save this file to test.cs and then run coffee -c test.cs on the file. You will get a new file called test.js that looks like:

(function() {
  var cubeStartNumber, math, squareStartNumber, startNumber;
  math = {
    cube: function(x) {
      return x * x * x;
    },
    square: function(x) {
      return x * x;
    }
  };
  startNumber = 2;
  cubeStartNumber = math.cube(startNumber);
  squareStartNumber = math.square(startNumber);
  alert("Cube: " + cubeStartNumber);
  alert("Square: " + squareStartNumber);
}).call(this);

That’s it!

If you run in to any problems during the install process, drop a line in the comments and I’d be happy to help.

Categories
Other Python

Learning How To Hack

I’m putting this here as a resource to myself, as well as to others.  If you’ve ever been interested in learning to become a better programmer, this resources is for you.

http://krainboltgreene.github.com/l/2/

Beginning to advanced Python, Ruby, and software development practices are covered here with links to free PDF books to help you learn.  I highly suggest taking a look.