WhatsApp confirms that, your contact details will be shared with Facebook whether you like it or not. Even if you opt-out from Contact Sharing, it will be shared with Facebook.
Read this post on techrounder.com