I don't think The Sorrow is really a boss. The entire purpose of "fighting" him is to learn about the Sorrow. It's like calling Fortune in MGS2 a boss battle. It wasn't really.
I found the hardest...first time around, was probably Solidus. I hadn't really grasped how to use the sword and he owned me a few times. Generally the bosses are very balanced in difficulty though, and are fun to verse.