I'm of the opinion that it's okay to be silent, to not speak if you don't have anything to say.
or simply: