Insofar as the non-GPL part is specific to the GPL code, yes. If you reimplement the GPL code under a different license, you could reuse the public domain bits. I expect in most cases, it wouldn't be terribly worthwhile.