To be honest, a lot of the difficulty of the older games was largely in what I think we can call, in hindsight, piss poor game mechanics due to just how early in the medium's life it was. On the other hand, the rest of it I feel like was a total disinterest in anything resembling game balance. This part I don't actually think is a terrible thing. It leads to a lot of interesting design choices and allows for a lot of throwing things at the wall to see what sticks, which can lead to some really fun stuff.
I mean, really think about "difficult" games from the 90's. Let's say the Baldur's Gate series for example. What made those games hard was a bunch of really bullshit game mechanics like character gibbing and save vs. death effects. It didn't really have a lot to do with character skill really. Or something like Ninja Gaiden, with extremely cheap enemy placement that required more memorization of the level layout than actual player skill. Now if you look at some of the difficult games of today, such as Demon's Souls, Dark Souls, Dwarf Fortress, higher difficulty Batman: Arkham Asylum and Arkham City and the like, if you die or fail your objective it's probably because you fucked up. This is a difficulty path that I'm a-ok with continuing in some fashion.
