Bathrooms are creepy. Kids should be allowed to go in alone when the parent feels that the kid is mature enough and the the location is safe enough. I don't think it's ideal for 8 or 9 year old girls to go into mens restrooms or 8 or 9 year old boys to go into womens restrooms, but it's also not always ideal for them to go into the appropriate restroom by themselves, especially at large, crowded, public places where anything could happen and no one would likely notice. Ideally, young boys would be able to go into the men's room with their fathers and young girls would be able to go into the women's room with their mothers, but that just isn't realistic. Even in families where there is a mom and a dad and the family often does things together, there are plenty of times where one parent is with an opposite sex child who is in that in-between age where they are old enough that they should be using the 'right' bathroom, but young enough that the cirmumstances just don't make it as safe as the parent would like.
I don't think that there is a 'right' answer - there are too many factors at play. If the kid goes in alone, all could be perfectly fine or something horrible could happen; if an adult goes in alone, all could be perfectly fine or something horrible could happen. I'd say any younger than around 5 and it's a given that a kid needs an adult with them, but after that, it's a grey area for several years.