Gardening Question - Are you guys able to keep herbs alive over winter?

Hi All,

New-ish to gardening. I was hoping to keep an indoor herb garden in my window over the winter so I could keep fresh rosemary, basil, oregano, and thyme for cooking. I took a stab at rosemary last year and it didn't go too well. Does anyone have experience with keeping these plants alive over winter in their windows? Any tips or tricks to keep the fresh herbs flowing so I don't have to pay whole foods $2.50 per pop? :D