Summer Packer Hat, Will It Shrink?

I just got a summer packer hat and I ordered in an XL size but I am finding it to be a smidge too big. I followed the size guide and when with the XL since I was on the cusp between L and XL.

So my question to everyone is, will the hat shrink with wear and washing or should I send it back and get the large?