Anyone whose gender presentation does not align with society's conventions would face fewer difficulties.
The argument
If the law dictates that transgender citizens cannot use the restroom of the gender they identify with, it opens the door for cis men and cis women whose physical appearance does not match their birth sex to face scrutiny. The reason is that such a law cannot be enforced in the way it is written. Nobody checks documents at a restroom door, so in practice the rule is enforced by strangers judging from appearance — which means the test actually being applied is not whether a person is transgender but whether they look the way others expect their gender to look. That distinction is what makes the policy backfire on the people it was not aimed at. It could lead to cis women who are often mistaken for men because of their masculine features being challenged over their right to use the female restroom, with no way to settle the challenge short of the kind of confrontation the law has now licensed. By allowing transgender citizens to use the restroom of their choice, the law would instead reduce the margin for confrontation among everyone whose appearance does not meet society's narrowly-defined preconceptions. This is not hypothetical. In a Walmart bathroom, a cisgender woman was insulted after being mistaken for a trans woman, and the victim of the abuse said she believed the perpetrator singled her out because she had short hair. The episode shows the mechanism operating exactly as described: the woman was targeted on the basis of appearance, by someone acting on their own judgement, and being cisgender offered her no protection. A rule that invites members of the public to police who belongs in a restroom will fall on gender non-conforming people of every kind.
Context
Anyone whose appearance deviates from societies narrowly defined conventions of gender can face scrutiny in a public restroom. Cismen with feminine features and ciswomen with masculine features could face difficulties in public restrooms if transgenders are challenged on their freedom to use their preferred restroom.
Premises
Counter-arguments
Cis bathroom users have their birth certificates which show their legal sex. Any confrontation could be easily diffused through the production of documentation showing their official, state-recognised sex.
Rejecting the premises
[Rejecting P2] Any confrontation with cis citizens would be easily diffused by showing ID with the individuals sex assigned at birth.