No you don't have to turn it :)
If the stem bothers you & you can remove the cup without it you can cut it off.
If the grips really bother you, you can try turning the cup inside out but it'll change the length & capacity so you might want to see it it still works for you that way around.

You can also try filing the grips down with a nail file. I know there have been a few people who have done that with their fleur cups and it was helpful in making the cup more comfy to wear. I am one who does not like to turn my cups inside out so I would cut the stem off and file the grips down. Hopefully some will chime in that have filed there grippers down on their cup.

As long as you're sure you can reach the cup without it, you can try just cutting off the stem and that may fix your issue. From the post it sounds like you've just started trying it today? If that's the case, quite possible the irritation is worse because you're sore from messing around inserting and removing it. Let your body rest for a few days before trying again, and you may find that irritation is not an issue. Between waiting a little and trimming the stem if needed, you probably won't have to mess with the grip rings unless the cup sits very low for you.

As the others have said... no, you don't have to turn it. Skim/Read this http://kuradi8.livejournal.com/ for other helpful hints.