I think it's mostly a Western thing. I'm basing this on stuff that I learned in college more than four years ago, but my understanding is that Eastern music isn't quite as locked into multiples of 2 and 3, and sometimes doesn't have any meter at all. But I may be completely wrong about that.
If I were to think about this based on human nature, I think humans just like things that come in 2's and 3's, I forget the reason. We also like things that divide evenly, which is why the most common meters are multiples of 2 and 3, such as 4/4 and 6/8.
I'm thinking it's just a chicken-or-the-egg thing in the West. Which came first, the person playing in 4, or the person copying him? I think it just kind of developed naturally, the way languages do. And actually, since musical atunement is borne from our linguistic necessities, since because rhythm and pitch are such important facets of a language, we have developed a natural affinity for the two concepts, this just may be the case.
Some of the best stuff is in weird meters, though. The opening to Zepp's The Ocean has three bars of 4, followed by one of 3. Pink Floyd's Money is in 7. I can't think of any others off hand, but I know they're out there.
EDIT: Fixed spelling.