Differentially private natural language models: Recent advances and future directions

L Hu, I Habernal, L Shen, D Wang - arXiv preprint arXiv:2301.09112, 2023 - arxiv.org
Recent developments in deep learning have led to great success in various natural
language processing (NLP) tasks. However, these applications may involve data that …

DP-mix: mixup-based data augmentation for differentially private learning

W Bao, F Pittaluga, VK BG… - Advances in Neural …, 2024 - proceedings.neurips.cc
Data augmentation techniques, such as simple image transformations and combinations,
are highly effective at improving the generalization of computer vision models, especially …

Formal Privacy Proof of Data Encoding: The Possibility and Impossibility of Learnable Encryption

H Xiao, GE Suh, S Devadas - Proceedings of the 2024 on ACM SIGSAC …, 2024 - dl.acm.org
We initiate a formal study on the concept of learnable obfuscation and aim to answer the
following question: is there a type of data encoding that maintains the" learnability" of …

Differentially private non-convex learning for multi-layer neural networks

H Shen, CL Wang, Z Xiang, Y Ying, D Wang - arXiv preprint arXiv …, 2023 - arxiv.org
This paper focuses on the problem of Differentially Private Stochastic Optimization for (multi-
layer) fully connected neural networks with a single output node. In the first part, we examine …

Geometry of Sensitivity: Twice Sampling and Hybrid Clipping in Differential Privacy with Optimal Gaussian Noise and Application to Deep Learning

H Xiao, J Wan, S Devadas - Proceedings of the 2023 ACM SIGSAC …, 2023 - dl.acm.org
We study the fundamental problem of the construction of optimal randomization in
Differential Privacy (DP). Depending on the clipping strategy or additional properties of the …

Gradient sparsification for efficient wireless federated learning with differential privacy

K Wei, J Li, C Ma, M Ding, F Shu, H Zhao… - Science China …, 2024 - Springer
Federated learning (FL) enables distributed clients to collaboratively train a machine
learning model without sharing raw data with each other. However, it suffers from the …

PAC-Private Algorithms

M Sridhar, H Xiao, S Devadas - Cryptology ePrint Archive, 2024 - eprint.iacr.org
Provable privacy typically requires involved analysis and is often associated with
unacceptable accuracy loss. While many empirical verification or approximation methods …

Inference and Interference: The Role of Clipping, Pruning and Loss Landscapes in Differentially Private Stochastic Gradient Descent

L Watson, E Gan, M Dantam, B Mirzasoleiman… - arXiv preprint arXiv …, 2023 - arxiv.org
Differentially private stochastic gradient descent (DP-SGD) is known to have poorer training
and test performance on large neural networks, compared to ordinary stochastic gradient …

Delving into Differentially Private Transformer

Y Ding, X Wu, Y Meng, Y Luo, H Wang… - arXiv preprint arXiv …, 2024 - arxiv.org
Deep learning with differential privacy (DP) has garnered significant attention over the past
years, leading to the development of numerous methods aimed at enhancing model …

Private and Communication-Efficient Federated Learning based on Differentially Private Sketches

M Zhang, Z Xie, L Yin - arXiv preprint arXiv:2410.05733, 2024 - arxiv.org
Federated learning (FL) faces two primary challenges: the risk of privacy leakage due to
parameter sharing and communication inefficiencies. To address these challenges, we …