Remove your name from Truecaller so that it won't appear whenever anyone search for your number.
Read this post on techantic.net