A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications

S Liu, PY Chen, B Kailkhura, G Zhang… - IEEE Signal …, 2020 - ieeexplore.ieee.org
Zeroth-order (ZO) optimization is a subset of gradient-free optimization that emerges in many
signal processing and machine learning (ML) applications. It is used for solving optimization …

Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks

CC Tu, P Ting, PY Chen, S Liu, H Zhang, J Yi… - Proceedings of the AAAI …, 2019 - aaai.org
Recent studies have shown that adversarial examples in state-of-the-art image classifiers
trained by deep neural networks (DNN) can be easily generated when the target model is …

Patdnn: Achieving real-time dnn execution on mobile devices with pattern-based weight pruning

W Niu, X Ma, S Lin, S Wang, X Qian, X Lin… - Proceedings of the …, 2020 - dl.acm.org
With the emergence of a spectrum of high-end mobile devices, many applications that
formerly required desktop-level computation capability are being transferred to these …

Maze: Data-free model stealing attack using zeroth-order gradient estimation

S Kariyappa, A Prakash… - Proceedings of the IEEE …, 2021 - openaccess.thecvf.com
Abstract High quality Machine Learning (ML) models are often considered valuable
intellectual property by companies. Model Stealing (MS) attacks allow an adversary with …

Structured adversarial attack: Towards general implementation and better interpretability

K Xu, S Liu, P Zhao, PY Chen, H Zhang, Q Fan… - arXiv preprint arXiv …, 2018 - arxiv.org
When generating adversarial examples to attack deep neural networks (DNNs), Lp norm of
the added perturbation is usually used to measure the similarity between original image and …

Zeroth-order stochastic variance reduction for nonconvex optimization

S Liu, B Kailkhura, PY Chen, P Ting… - Advances in Neural …, 2018 - proceedings.neurips.cc
As application demands for zeroth-order (gradient-free) optimization accelerate, the need for
variance reduced and faster converging approaches is also intensifying. This paper …

Forms: Fine-grained polarized reram-based in-situ computation for mixed-signal dnn accelerator

G Yuan, P Behnam, Z Li, A Shafiee… - 2021 ACM/IEEE 48th …, 2021 - ieeexplore.ieee.org
Recent work demonstrated the promise of using resistive random access memory (ReRAM)
as an emerging technology to perform inherently parallel analog domain in-situ matrix …

Distributed zero-order algorithms for nonconvex multiagent optimization

Y Tang, J Zhang, N Li - IEEE Transactions on Control of …, 2020 - ieeexplore.ieee.org
Distributed multiagent optimization finds many applications in distributed learning, control,
estimation, etc. Most existing algorithms assume knowledge of first-order information of the …

Zo-adamm: Zeroth-order adaptive momentum method for black-box optimization

X Chen, S Liu, K Xu, X Li, X Lin… - Advances in neural …, 2019 - proceedings.neurips.cc
The adaptive momentum method (AdaMM), which uses past gradients to update descent
directions and learning rates simultaneously, has become one of the most popular first-order …

Non-structured DNN weight pruning—Is it beneficial in any platform?

X Ma, S Lin, S Ye, Z He, L Zhang… - IEEE transactions on …, 2021 - ieeexplore.ieee.org
Large deep neural network (DNN) models pose the key challenge to energy efficiency due
to the significantly higher energy consumption of off-chip DRAM accesses than arithmetic or …