The story is not bad. The animation, the music, the dances, they are all professionally done. However, there are other, more important (to me) aspects of the series that annoy me so much, I can never watch more than an episode in one sitting.
The clichés and the characterizations. ALL the characters are very, very annoying in my opinion. It is true that the fact that none of the characters are perfect should be a positive, but in this case, the typical anime clichés are so overflowing, it ruins the whole series for me.
Suzumiya Haruhi: She's a selfish, uncaring, immature, spoiled little girl, who drags everyone into her crazy childish schemes, not caring whether she causes bad situations, or puts others in danger. Whether it's about humiliating someone, forcing others to do what she wants, sexual harassment, she stops at nothing to get her selfish little desires. The fact that the world revolves (literally) around a stupid little girl like that makes my head explode.
It is an anime cliché and tsundere types like her are considered cute when they are 2D, but in real life, you would just call her a b*tch. Why like a girl who hurts her friends for her own satisfaction?
Kyon: In my humble opinion, no man should allow themselves to be f***ed around by a girl like this character does. No matter his reasons for being in Haruhi's little club, if it were me, I would have told that spoiled brat where to shove her club activities, attempt at humiliation, selfish acts and pushiness.
And again, not in the cliché tsundere way. He shouldn't refuse what she says and simply stay. In real life, anybody did something like that to you, you would tell her to leave you alone (give her a slap if you were a more violent person), and just leave. If she tries to come back, ignore. A man who comes back for more humiliation is no man in my eyes.
Asahina Mikuru: She's probably the character who annoys me the most. GROW SOME BACKBONE, YOU USELESS WALKING PAIR OF BREASTS! If this is what Japanese men think women should be like (li-li-li-li-li-li-like, pl-pl-pl-please be so ki-ki-kind and stop gro-gro-gro-gro-gro-gro-gro-groping me! desu kyaa mouu nipaa), I give my sympathies to my Japanese counterparts.
On a less rant-y note, Asahina Mikuru is also a common anime cliché character, really popular among audience. Again, in real life, anyone would make a more Naruto-ish approach to her and just call her a weirdo who can't put a decent sentence together and if left alone, would die of hunger in a day, because she's too embarrassed to ask for bread in the store.
Yuki Nagato: A stereotype of a characterization that is nonexistent in real life. She would be okay, if only she learnt to talk in a normal voice that one would be able to hear without the help of a microphone, and she dropped the apathetic act. It's unnatural; and while it suits the idea that she is a robot, it is just too overused in anime to be likeable.
Koizumi Itsuki: He is only better than Kyon because Haruhi doesn't harass him as much as she harasses Kyon.
I know everybody is in love with this show, but as far as I'm concerned, the characterizations are so cliché and (I cannot say this enough) ANNOYING, I can't sit through more than an episode without ripping my hair out.
Another aspect I forgot to mention earlier, but a kind commenter brought it up: the humor is also not-quite-there. It is supposed to derive from the characters, like how Yuki doesn't care or how Mikuru is always humiliated... However, since the characters are overused, the jokes are tiresome and boring, as well. These cliché jokes weren't funny the first time I saw them in anime, because I don't like to laugh at others getting humiliated, and hasn't grown any funnier the last fifty times I encountered it in anime.
(Since this question has been raised in an other comment: I watched the series in release order, than after finishing, I spent an hour quickly rewatching it in real order)