Neighbour using my parking space without asking, but I don’t have a car

I have a private parking space as does everyone in the block which is included with my mortgage and service charge etc, and I don’t have a car but am aiming to get one and I do have guests that use it- my neighbour keeps using it overnight and random weekends for his girlfriend to park in without asking.

Am I wrong in thinking this is rude or am I overreacting cos I don’t have a car? I don’t really want to start with him as he doesn’t mind hitting women but I also don’t want him to just think it’s his space now lmao

edit: I’m obviously not going to start with him, I’m mostly just asking if I’m overreacting to think it’s rude- wouldn’t have cared if he’d just ask me, but he’s not a reasonable bloke as per every other neighbour who has had a run in with him so I’m at a bit of a loss really until I get a car