The best advice on buying a bass is to find a friend who can teach you just how to hold down the strings with your left hand and how to pluck with your right hand. This shouldn't take long- we're talking 10 minutes to a half an hour, just to get the feel of the instrument. Once you've done this, go to a bunch of music stores and play a bunch of basses and see what you like.
On to specifics-
Bass- The key here is string action. By string action I mean, are the strings close to the neck and easy to hold down? If you haven't been playing for a long time, you probably haven't developed those muscles necessary to really hold down the strings. Buying a bass with good string action makes it easier to play, and consequently, quicker to learn. Bargain basses with good string action are certainly available- try yamaha, ibanez, squier, fender (mexico), cort, peavey, there are a host of others.
Once you've determined the bass has good string action, play around on different parts of the neck. Bass necks can vary widely in their shape and thickness. You may find it more comfortable to play on a tapered-flatter neck, or a fatter C-shaped neck, etc. I tend to like my strings set very low on a neck that is a little fatter (I'm more of a Fender guy). There was a time, however, when I only played low profile ibanez-style necks.
After that, check the tone. This is a little more variable depending on the amp. I suggest using a similar amp at similar settings for every bass you try. Just about every store will have one of those ampeg 1x15 cubes. It sounds dumb, but set everything flat (all EQ knobs at 5) and try to dial a tone with the bass's volume and tone knob. A word of advice- if you think you only want to play with a pick, try playing with your fingers as well. You may change your mind depending on the style of music. You can play metal, punk, etc, with your fingers, it's rare to find good tone playing jazz or blues with a pick.
Last, check the looks. Although this is initially what draws most people to a bass, remember that actually being able to play and liking what you hear is key. It's not uncommon for a good looking bass to be unplayable to a particular player (Rics always feel super awkward to me, so I don't play them, although they look cool as hell). Given the plethora of available basses, this shouldn't be a problem. Don't neglect the used section, it can afford the best value, and you will eventually put a nick in your shiny new bass.
Amp- The key here is to find something that sounds relatively good and is loud enough to get over a drumset and a guitar. Don't buy one of those little rinky-dink 15 watt amps. You'll just end up buying another amp shortly, because eventually you'll want to play with other people. There are numerous configurations you can go with here, but for your experience level a combo, where the amplifier and speakers are joined in one unit, would make the most sense. Combos have gotten really powerful in recent years, and they've gotten really good at tuning the cabs for a more full sound. At a bare minimum, I would say shoot for the 100-150 watt range. Keep in mind, this will still be quiet if your drummer turns out to be a basher.
With amps it's the same principle as picking a bass. Play lots of them. You may prefer the added attack of a 2x10 combo over the rumble of a 1x15 or vice versa. You might like something with the growl of a tube preamp, etc. Try ampeg, fender, behringer, ashdown, gallien-krueger, SWR, fender, etc..
Overall, I think the best thing you can do is buy things that sound versatile. After that, play with as many different people as you possible can. Jazz, blues, metal, rock, folk, it doesn't matter. Just make sure they are better than you. Trying to keep up is how you get better.
Best of luck!
