If any man despises me, that is his problem. My only concern is not doing or saying anything deserving of contempt.
or simply: