She's fascinating. "Agnostic" is a good word for her. Her relationship to organized religion was always tenuous, I think you could say - as the excerpts above suggest - but she's got a strong spiritual streak in her at times. Atheist? Not if you don't allow for some kind of transcendence, at least in her earlier work. On the one hand, she's definitely wrestling with her received Christianity (there are poems that suggest a more pious side to her; my volume is deep in a box somewhere or I could look up a few), while on the other hand I think there is a part of her that feels sacred, set apart, transcendent, what have you, and it's intertwined with her reverence for the natural world. It's hard to say what she believed, exactly. For one thing, we can hardly call any of her poems statements of belief, as she kept most of them hidden from everyone else while she was alive. Also, what we do have is all over the map, and is oftentimes cryptic or tongue-in-cheek (it's part of what makes her such a pleasure to read).
Dickinson isn't a very firm figure in modern culture. She wasn't widely embraced or studied until the early/mid 20th century. Of course, the mythos of the reclusive poet took hold quickly, and she's certainly now considered a "national treasure" and a staple in English education, but she doesn't fit in in quite the same way as a Whitman, Faulkner, or Frost does. She's a little vague and out of joint, which is a shame, really, because there's a lot of depth to her that people don't often enough pursue.