The "REMEMBER MY LAST" Howler was explained in the end... Dumbledore sent it to remind Petunia of the letter that he had tucked in with Harry's blankets when he left Harry with the Dursleys as a baby. No deep mysteries there, I think.
However, I think there is more to Petunia than meets the eye. She knew that the dementors were Azkaban guards. I really have a hard time believing that she just "accidentally" remembered that. Heck, when I was re-reading the first four books last week for the first time in three years, I couldn't even remember that they were called dementors.

I think Petunia may be hiding the fact that she might once have not been quite as vehemently opposed to the magic world as she now claims. Even though she herself never had any magical abilities, maybe she only turned her back on the wizarding world after Voldemort killed her sister...helped along, of course, by Vernon's stubbornness about anything out of the ordinary.
Needless to say, the bond of blood relationship goes deeper than her present hatred of magic. She keeps Harry around for his safety, because he is her sister's son, and despite what she says, I think she loved Lily. She may hate that Harry is a wizard, but I don't think she really hates Harry.