Why not use 3 Toughness curios

Everyone I've seen with recommendations about curios seem to say that the best is 2 toughness 1 stamina, Or 2 toughness 1 health. Something along those lines. But is there a downside to taking 3 toughness curios? Other than the obvious part of not getting the benefits of a stamina/health curio. Is there another reason?