I second Dexy's Midnight Runners. I have Too-Rye-Ay on vinyl somewhere.
There are a lot of bands that I think suck in general, but I would absolutely go see live if I had the chance (and was also paying to see a good band along with). I'm going to catch flak for this, and probably rightly so, but Fuel, Incubus, and Linken Park put on some pretty decent live shows, even though I can't stand them otherwise. I saw them back in 2001 at HFStival.
I remember when Marcy Playground first showed up, circa 1996. I throughly enjoyed them back then. I also remember when Live showed up with Mental Jewelry and Throwing Copper. They were pretty good back then, and though they went downhill after that (though I liked that one song with Tricky) they also rock live.