Here's the thing: Sven has always been clear, to a painful degree, that he is not in a committed relationship with Faye. If he were to try to soften the blow, it would be admitting that he's ok with her thinking that he cares more about her than he wants her to know that he does. Even if he does care that much, and/or she assumes he does anyway.
Because he was a friend, something he is willing to acknowledge, he had a justification to tell her that he might find difficult if she was *just* a regular sex buddy who he didn't care about at all but who cared about him.