They were all martial artists, and Zhou Ruying and Lu Mingshu used a sword. Even though Ling Wuxia didn't, she practised sword energy too. Thus, there was nothing wrong in calling them swordsmanship cultivators.
But what's wrong with that? Why can't we go out freely?
Not mentioning Zhou Ruying and Ling Wuxia, but Lu Mingshu had access to the Heavenly Wheel, so she knew that, in this kind of world, swordsmanship cultivators were normally part of the upper class, as their ability was mostly above average and they could fight well.
So why did Su Ze make it sound like they were discriminated against here?
"Misses, could you spare some of your time?"
Seeing the trio staring at each other, Su Ze laughed. "It's not appropriate to chat here, and there are three kids here too. It wouldn't be nice if they were dragged in."
"Are you threatening us?" Zhou Ruying asked coldly.