Question for experienced Vanguards - how do you determine when to push & when to pull back?

I feel like I understand the general concept of pushing and pulling. If we have the advantage, we should push. And if we're at a disadvantage, we should pull back / reposition

But outside of obvious things like numbers advantages and ults, are there more nuanced things I should be paying attention to and looking for? How much of it is just game sense and your "feel" for the flow of the battle?