I imagine a setting and characters realistically, some of them I naturally assign the actor or actress who played them in the movie if I think that person really fits the character. When I first read the book, I thought "Maggie Smith would be the perfect McGonagall," and Maggie Smith was who I always saw as McGonagall, so I was thrilled when the movie came out and apparently the casting director agreed with me.

I didn't have a "look" for Lockheart, but after seeing Kenneth Branaugh (sp?) play him, I was, like, "YEAH!" and now when I read it, I see him. However, I'm a little disappointed, because I see Moody as more of a Filch-y character (the guy who plays Filch in the movie) and the guy supposedly up for the part couldn't look more different... I saw Lupin as a bit more handsome, also Sirius (even though he's obviously filthy)...
As far as the grounds go, I'm completely confused on the layout as it's very different from what I see in my head. The only thing similar the positioning of the castle to Hagrid's hut to the forest... But the lake, greenhouses, the tree the kids sit under... all very different.