SpeechGuard: Recoverable and Customizable Speech Privacy Protection

Uploading speech data to cloud servers poses privacy risks, making the protection of both acoustic and content privacy essential. Users often need the cloud to process non-sensitive information while protecting sensitive parts, with the ability to recover original data locally. However, achieving speech privacy protection that supports fine-grained customization and full recoverability remains a significant challenge. Existing methods often rely on irreversible or inflexible techniques, such as uniformly anonymizing the entire speech or replacing sensitive texts, making them inadequate for this purpose. We introduce SpeechGuard, a system that enables recoverable and customizable speech privacy protection by applying reversible protection methods and assigning private information to permission groups. We design a multi-parameter warping function with an inverse function for voice conversion to protect acoustic privacy. We also develop a mechanism for automatic or manual detection and encryption of sensitive texts to protect content privacy. By categorizing listeners into permission groups and assigning warping parameters and encryption keys, SpeechGuard enables different listeners to recover varying levels of acoustic and content information according to their permissions, ensuring personalized access to speech data. Experiments on three datasets show that SpeechGuard outperforms three baseline systems in anonymity, sensitive content confidentiality, and attack resistance. Moreover, it provides recoverable and customizable protection for acoustic and content privacy, allowing for tailored privacy definitions and protection strength.