No, it won't be too harsh on the silicone. I'm not fanatical about "sterilizing" (it's really only sanitizing) my cups. "As clean as cutlery" is good enough for me. I wash it well before and after each cycle, and just give it a quick rinse in between wearings during. If you use soap or chemicals, make sure you rinse your cup very very well. Some ingredients in soaps (including soap residue on your hands) can make yeast infections bloom. Other ingredients can be irritating to your delicate membranes. WATER is your cup's best friend -- not soaps or chemicals.

Well, sanatizing at the beginning of each new cycle is what I plan on doing.
For the rest it's the quick rinse, squirting water out the suction holes and put it back in.