If you export the header using dd (instead of the backup-header option in crypsetup) with the following options: "dd if=/dev/part bs=1024 count=2020 of=dd_header_backup", you end up getting the same file (hashes match) as if exported using cryptsetup.

It should be possible to overwrite the relevant sectors or blocks of the encrypted partition with zeros or random data (the latter might be the better option) without touching the encrypted data but that is just a theoretic approach, I personally never tested this.