The rest of the crowd were friends of my fortune, not of me. [Lat., Caetera fortunae, non mea, turba fuit.]
or simply: