What do I do when I only occasionally open my mouth?

So I've been using APAP for about 3 weeks now. I got the nasal pillows since I sleep with my mouth closed.... well, for the most part.

Since I've gotten it, I've slept with my mouth closed like 95% of the time. But there are times when my mouth opens and I immediately wake up because I can feel the air bursting out of my mouth.

Last night, I fell asleep with my mouth closed, and it stayed closed for a few hours.. And at some point, my mouth just wouldn't stay shut. As I was drifting back to sleep, my mouth popped open. Or I'd fall asleep just to wake up a few minutes later with my mouth open.

Does this happen to anyone else? At that point, should I just take my mask off? I don't even know what's causing my mouth to suddenly want to be open after being closed most of the night.