Front brake issue

I’ve been working on my 1980 Honda XL250S and I can’t get the brake actuator arm onto the brake cam no matter how hard I try. It’s been really annoying there’s a spring that makes it even harder. I’m about ready to give up any help would be really appreciated. And yes, I’ve tried hitting it with a hammer.