I don’t ever think “it sucks to be you.” I don’t even know what that means. That line of thinking implies that my life is somehow better. I’m never, and have never been for want.
The kind of person who could think that would have to be a pretty cold and vindictive person to not be able to empathize with something bad happening to someone else – even if you don’t like that person.
Ironically, about the only person it may suck to be is the person who could generate the thought, “it sucks to be you.”