I can get the winners and losers thing, but I guess I find it hard to see how you derive enjoyment from the game itself. I can easily see how it is fun to play games, but not really to watch them. This doesn't just go for sports, I can think of little more interesting than watching a chess game, say, whereas I quite like playing chess.
For me, it's an appreciation of the level of skill involved.
Yes, watching two dolts who don't know what they're doing play chess at a party is about as entertaining as watching paint dry, but watching two excellent chess players matching wits can be exciting.
Look at it in the way that you would look at watching a skilled guitarist on stage. Watching them nail a complex solo is the same to some people as watching an American football player complete a 70 yard pass to score for some. It's the heightened emotion and excitement, especially live, that drives it. A big part of it can be the atmosphere involved. It's not so simple as just winning or losing, but the way the team goes about it.
For me, personally, a game with a high score on one side gets really boring. Seeing the opponent crushed into the ground is nifty for a couple minutes, but it gets boring when one team is leading the other by about ten for an extended period of time. I know it's not your cup of tea, but bear with me for a moment on this example:
I was at a professional baseball game on Tuesday night. The team I root for was being outright abused the whole game. It was coming to the end of the game and they were down by a rather large deficit. In the span of about five minutes, they came back and almost took the lead. While they didn't pull it off in the end, the park still absolutely exploded with emotion over the excitement of watching them claw back and then suddenly almost clear the deficit with one moment.
Stuff like that doesn't happen every game (Just like not every concert is well performed or not every piece of art is particularly thought provoking), but it's the potential fort that excitement that I watch for.
Then there's the subtleties. Player positioning and following the odds and all that. Baseball's considered, by most, to be the most boring of the big professional sports. I personally disagree, because I'm fascinated by much of the things going on between the box scores. It can be the same thing with football (Soccer to us Americans), as I've never quite gotten the appeal, but I can see where its fans probably appreciate the subtleties more than I do.