I think the argument being made is that it's somewhat of an error to assume that the rich haven't gone through hardship. It's just a different type of hardship. Some great art can still be made out of it, but it requires a level of depth and self-awareness that allows the specific situations being outlined to be analyzed and presented in a manner that makes it somewhat universal. I suppose that is why you wrote "in an overwhelming majority of circumstances" rather than "all the dang time!"
Still, it seems an unwise gesture to rule out the capacity of anyone to make good art, regardless of socio-economic standing. If the poor can do it, why can't the rich? If you prick them, they bleed. If you tickle them, they laugh. If you poison them, they die. Granted, they can afford very nice pins, feathers and chemicals, but at the core of it they're still people - possibly distorted people but people nevertheless. I haven't had the chance to meet fabulously wealthy individuals but I've definitely met people who haven't wanted for anything in their lives, and they've had great moments and shit moments. People can treat rich people as well or as badly as they treat me or you, and often do anyways.
At the end of the day, the actual quality of the art has about as much to do with the artist's financial status as it does with gender, race and age - that is, both everything and nothing. Best to judge the music on its own merits rather than a potentially false image of the artist we've built up in our own minds.