I like music that to some degree fits in with these two criteria:
1. Fun
2. Significant -- having more structural "meat" than the average pop song
3. Being "anti-pop" -- incorporating noise or unusual sounds, melodies, or chord progressions
It does not have to fit all three, but 1 is the most important
This is why I like TMBG, Devo, Polysics, the Streets of Rage soundtracks, the Sonic Heroes soundtrack, or, hell, "My Sharona" and "Louie, Louie".
Though I hate to admit it, Vitamin C's vacation is enjoyable for me as well.