The reason for forgiving your enemies is not for their benefit but for your own benefit. Holding grudges against other people doesn’t hurt them; doesn’t even bother them much - in fact, even pleases them if they are still mad at you. It is not in your enlightened self-interest to hold grudges, regardless of whether it bothers the person you hate or not.