Update: This blog post is so absolutely outdated, that you shouldn't waste your time to read it. Just grab your newest Linux distro, feed your Mac with it and you will (in most of the cases) get a memtest option.
If your Mac occasionally starts crashing with nice kernel panics and no fancy new kernel extensions are loaded, your RAM might have gone bad. But how to find out?
On a PC you can just grab the next Linux distro CD/DVD and boot it. The bootloader will then ask you what to boot. Normally one of the options will be memtest. But not with a Mac.
So what are the options? There is a tool called Memtest OS X. Although it is licensed under the GPL the developers want to have money before you can decrypt the DMG image. I didn't have the time to wait for someone to send me a key, so I looked around and found Rember. Rember is a GUI for Memtest OS X and has the same memtest included ... for free. So why not use the included memtest in Rember instead!?
But with a full blown Mac OS X started a fair amount of RAM is already consumed. So it would be better to start the Mac in single user mode. To do that you will have to press and hold Cmd + S when you hear the start chime. When the screen goes black you can let go. Now you will have to wait until you will get the shell and enter the following lines:
/sbin/fsck -fy /sbin/mount -uw / /Applications/Rember.app/Contents/Resources/memtest -L auto 3
The option -L will create a logfile for you, auto tells memtest to use all the memory available and 3 is the amount of loops memtest will perform.
If anything goes wrong memtest will tell you.