I think the Shai-Gen are more annoying. Picture this scenario:
You climb a very tall building. You feel this great sense of accomplishment. Suddenly, a rocket hits you. You think "oh, no big deal, shields recharge and whatnot". However, a second rocket hits you, sending you flying off the tall building, straight to the ground, possibly to your death.
This has happened to me more times than I can count. Friggin' Shai-Gen.