I always look for a few things:
Reliability, Support/Knowledge, Price - you get what you pay for really.
Well, hostnamaste.com is a good
KVM VPS based web host, and I've recommended them to several people who now also use them and are impressed with their uptime and website speeds.