Can't speak for others, but I really enjoy how different Deerhoof from most bands in their eclectic sense of how music works. They switch keys constantly, their songs are schizophrenically mutable structurally, and Greg Saunier can play the fuck out of a drumset. They are sort of a hybrid child of art-rock and progressive rock, which I like despite not having a very developed taste for prog.
Honestly, most peoples' problem with the band seems to be the Japanese bird at the microphone, which is pretty much understandable.