I know I can't pull off the look of baggy mens' jeans, but power to you if you can. I go by the rule of top or bottom should be at least moderately form fitting, never both (unless you are super skinny and can rock that, again, I just look like trailer trash) and never neither.
I like 'baggy' pants, in that I usually go for pants (and even sometimes jeans) cut in this way:
It is more flattering for me, and more comfortable for me than the bizarre trend of jeans being so low you can see my ass crack or so high up they get hung on my bra.
I am a big fan of little blazers, but man it is so hard to find one that fits you right. As long as it's shape-defining, you are going to probably look sharp rather than like a business-dyke, but to be safe you could wear skinny pants. I guess you could wear heels, too, but that outfit sounds uncomfortable to me.