Especially after reading other reviews of this show, I am conflicted. I think my confusion reflects the conflict inherent in a show that is trying to bridge two very different genres of anime.
Story: The first half of the show seemed to develop a normal atmosphere that pandered to low-level otaku: a club is created in the name of solving the mystery of the school ghost. The creators did a decent (yet somewhat obvious) job of sneaking in hints that something darker was going on. The second half saw the happy-go-lucky-boobie ghost girl lose her marbles, and then delved into her split personality and traumatic past. Threaded through this story was a love story.
Ultimately, I think TOxA is a psychological anime disguised as an harem ecchi comedy. Psych fans seem to hate on the ecchi for being mindless and consider the psych elements fairly weak, while the ecchi fans are like, "Lol I liek boobs, why u make story thar?" Merging the two genres basically resulted in a loss of [what a fan considers] the best part of each genre. The ecchi wasn't really all that lewd or over the top and ended in a cute love story. On the other hand, the story was not creepy psych enough for psych fans.
My two biggest issues with the story were a) the happy ending, which was not at all appropriate for the psych atmostphere, and b) the numerous coincidences of people being descendents of the original players in Yuuko's past, which I suppose you could also argue in favor of because karma, etc.
Animation: Was pretty fab. The backgrounds were gorgeous, detailed, and perfectly fit the creepy atmosphere. The color choices were great, and I particularly enjoyed the point-of-view style in the part where Niiya ends up in Yuuko's body in her memories.
Characters: Only the ghost's character is at all complex, which is only due to her traumatic past. The main male character is only unusual because he's so emotional, otherwise, he's pretty cookie-cutter plain. The two female side characters were pretty weakly developed, but their actions were consistent with their personalities.
Overall: Not a bad anime, not a great anime, but an anime that disappoints because it fused two genres without adding the aspects most liked by the usual audiences of those genres.