I have been reading too much books and watching too much shows but there is one thing I want to clarify.
We do know that Harry has a tragic past when he was young. But come to think of it, Voldemort or Tom Riddle did had a tragic past too.
His mother died when he was born. His father abondoned him before he was born. I supposed something bad happened to him when he was living in the orphanage, that led him to become Lord Voldemort. What do you think happened?
I personally think that Tom, being a wizard, had some magical powers in him even before he realised it. He probably used his powers to defend himself from some bullies without even knowing, and that led to his isolation.
I can't but felt sorry for Tom Riddle. I am sure he wasn't born evil or anything bad. Not to say, he wasn't at fault for doing evil acts but maybe it is not entirely his fault?