Actually I think Safari is one of the fastest browsers around since it uses the KHTML rendering engine (Slightly modified from whats used by KDE I think)...
Yep
http://en.wikipedia.org/wiki/KHTML wikpedia confirms it, from the article:
KHTML is faster than the Gecko layout engine, its main open source rival and core of the Mozilla and Mozilla Firefox browsers, among others.
I use Safari in OS X (using version 2.0.4 (419.3) at the moment). It missing a lot of features I like in firefox (Tabbed windows and spell checking are the two major ones), It also has a new habbit of being awkward to extract the url from objects your viewing (eg if you right click a picture you have to open it in a new window to get the pics url, though it does have an option to copy url to clipboard which sort of makes up for this)
edit: Performance may be slowing down for the original poster because of computer resource issues (lack of ram) or similar, i'd be interested to see what it's memory foot print load time etc is compared to firefox.