We know that Angus has been in at least one serious relationship (at least serious enough to not visit CoD during its duration) for at least long enough to know that his ex was apt to belittle people, and it would be unusual for a hetero relationship not to include sex.
...unless the no-sex was the deal breaker in the relationship. She could have said, "No sex until we're married." He could have agreed upon it for a while, but then something else happened and boom, he ditched her. And he could have still remained a virgin.
You
can be in a serious relationship without sex, believe it or not. Especially if one or both parties is of a particular religious bent (which, I'm almost certain, the ex-GF-not-named may have been).
EDIT: Of course, you may be right with your comment about Marigold. But here's the thing with Mar-bear: he never saw her as a romantic/sexual partner. Therefore, it was easier for him to say, "uh, no, Mar, I'd rather be just friends." (Though he was tempted - "Whoa, yeah, boob, awesome.")
It may have also been easier because he walked in on Faye suddenly leaving, without explanation. Then, as Mari came on to him, he realized, "Oh, ####, she thinks I like her and she went and told FAYE, and now I'm never gonna have a shot at Miss Barista Biatch."
Angus as a virgin would work.