To be "damned" is to be condemned by god and sent to eternal punishment in hell. So for a Christian like Joyce, yeah, it's a serious word. It's also not just about the word, it's a respect issue.
If my friend told me they didn't want me to say the word, "table" around them or expect them to say it, it might seem odd to me but it's basic respect to acknowledge and try to adhere to their wishes. If it remained a point of contention one or both of us couldn't get past, then we probably couldn't be friends. Although Joyce and Joe aren't friends as such so that's why the whole discussion/argument is happening.