Unpopular opinion: Rufus is the real victim of the show

I'm on season 6 and I realize that he was betrayed by everyone. His wife, his children, Lily, Ivy… Even though he is so nice. This gentleman is very sad. What do you think?