The correct answer when someone tells you that you misgendered them is “oh, sorry, thanks for letting me know, I’ll use the correct pronouns in the future,” not “actually, a historically male-oriented word is gender-neutral and I am absolved of all guilt.” It’s likely not a big deal, so there’s no need to defend yourself over it, acknowledge it and move on.