What I'm getting from your descriptions is that you have no clear idea what to listen for. It's not the rhythm of the notes that gives you the beat, it's the rhythm of the emphases on tho notes... and they aren't always emphasized in every measure. It sounds like you've never performed any music yourself, at least not in any setting that requires you to follow a conductor. I recommend you learn to play an instrument in an orchestra or brass band, or sing in a choir, or even just take a ballrom dancing class. Anything that requires you to DO something in rhythm with music should work.
This is a song we used to sing when I was a school boy
http://www.youtube.com/watch?v=sRYQPq_iT3wI have later heard that the last line in each stanza has a rythm different from the rest of the song. I did not notice any difference back then, and i still can't hear it.
I have tried to learn an instrument. As a child I was learning the recorder. But then I got injured, and was away from school for several months. When I came back, there was no question of me keeping up with the rest of the class, so I was put in a class that consisted of an uncomfortable mix of kids who where far ahead of me and rank beginners. I gave up after a few weeks.
As for doing something with rythm, let me tell you just one episode. Needing som exercise, I took some gymnastics in night school. The instructor showed us how to walk back and forth and clap our hands. We then did it all together a couple of times. Then we stopped, and he started some music. I did just as we had rehearsed. The other did the same, only about 50% faster. It seems obvious to me that it must have been the music that told them the tempo. Just how I have no idea.