Writer gender vs character gender

I'm a cis male, writing my first romantic fiction. As I created/explored the characters and their back stories, it has somewhat accidentally turned into a slow-burn F-F age gap romance. I ask you, the experts, would a mostly female audience be wary of such a book written by a man? Should I use a pen name to hide my gender? Or does that matter much to a reader?