The main idea for gender neutral services appears to pivot on how people identify themselves. The presumption is that you are who you think you are no matter the real facts. I will say bluntly and emphatically that this is wrong. A guy is a guy no matter what he believes, as is a woman. I am not, nor is the larger society, obliged to feed the delusion of those who are wrong about who they are. It is not charity to acquiesce in delusion anyway. Charity is trying to help such folk rid themselves of their error, as compassionately as circumstances allow. So no, people confused as to who they are have no right to use the facilities of their choice.
The other major argument for allowing unisex restrooms is that hey, bathrooms in private homes are unisex. This is as true as it is obvious. But it is intellectual dishonesty to take the point further than propriety allows and argue that all bathrooms everywhere must (or ought to be) unisex. A bathroom in a private home is single service: only one person at a time uses it, unless that person chooses to let someone else in while they're there. Public services, even those with stalls, by their nature do not afford that same degree of privacy.
I am willing to argue it is axiomatic that propriety demands that public facilities be gender specific: when dealing with private functions in public areas there is simply nothing wrong with keeping genders separate. To do otherwise is to feed delusion. Or, worse, perversion.
This is not a difficult issue. Yet it is being made difficult by those who will not see reason. It is a prime example of the tyranny of the individual. It is as bad, and arguably worse, than the tyranny of the state.