Introduction

Adversarial transferability—the phenomenon whereby adversarial perturbations crafted for one model can fool a different model—is a cornerstone concern in machine learning security [szegedy2014intriguing, goodfellow2015explaining]. Transferability enables black-box attacks where an adversary has no direct access to the target model [papernot2016transferability]. Despite extensive study, the quantitative relationship between model capacity and transfer success remains undercharacterized.

Prior work has examined transferability across architectures [demontis2019adversarial], training procedures [tramer2017space], and model ensembles [liu2017delving], but systematic "phase diagrams" mapping transfer rate as a continuous function of capacity ratio are scarce. We address this gap with a controlled experiment using MLPs of varying width and depth on synthetic data, isolating the capacity variable from confounds such as dataset complexity or training hyperparameter variation.

Method

Data

We generate synthetic 5-class Gaussian cluster data with 500 samples and 10 features per sample. Cluster centroids are placed along orthogonal directions in feature space (via QR decomposition) scaled by a factor of 3.0, ensuring well-separated classes with unit-variance noise. The same data generation procedure is repeated with 3 random seeds ($s \in {42, 43, 44}$) for variance estimation.

Models

We train 2-layer MLPs with ReLU activations and hidden widths $w \in {32, 64, 128, 256}$, yielding parameter counts from 1,573 (width-32) to 69,637 (width-256). All models are trained with Adam (lr${}=0.01$) for 50 epochs with batch size 64. For the cross-depth experiment, we additionally train 4-layer MLPs with the same widths.

Adversarial Generation

We use the Fast Gradient Sign Method (FGSM) [goodfellow2015explaining] with perturbation budget $\epsilon = 0.3$: $$x_{\text{adv}} = x + \epsilon \cdot \text{sign}\left(\nabla_x \mathcal{L}(f_\theta(x), y)\right)$$ where $\mathcal{L}$ is the cross-entropy loss. We only consider samples where the source model is correct on clean data and incorrect on the adversarial example ("successful adversarials").

Transfer Rate

For each source-target pair, the transfer rate is the fraction of successful source adversarials that also cause misclassification on the target model: $$\text{TransferRate}(S \to T) = \frac{|{x : f_S(x_{\text{adv}}) \neq y \wedge f_T(x_{\text{adv}}) \neq y}|}{|{x : f_S(x_{\text{adv}}) \neq y}|}$$

Results

Same-Architecture Transfer

Table shows the mean transfer rate across 3 seeds for all 16 source-target width pairs.

Mean adversarial transfer rate (%) for 2-layer MLP pairs. Rows: source width; columns: target width. Diagonal entries (self-transfer) are 100% by construction.

Capacity Ratio Analysis

Grouping by the capacity ratio $r = w_{\text{target}} / w_{\text{source}}$ reveals a monotonic trend (Table).

Mean transfer rate by capacity ratio r = w_T / w_S.

The data reveal an asymmetry: at matching absolute capacity distance from ratio 1.0, transferring from a large model to a small target (ratio $<1$) yields higher transfer rates than vice versa (ratio $>1$). For example, ratio 0.5 achieves 89.9% while ratio 2.0 achieves 86.3%.

Cross-Depth Transfer

When source and target share the same width but differ in depth (2-layer source $\to$ 4-layer target), the mean transfer rate drops to 76.5%, compared to 100% for same-width same-depth pairs. This 23.5 percentage-point gap demonstrates that architectural similarity—beyond raw parameter count—significantly affects transferability.

Discussion

Our results support three conclusions:

Capacity ratio as a predictor. The capacity ratio $r$ is a strong predictor of transferability. The relationship is approximately log-linear: each doubling of the ratio away from 1.0 reduces transfer rate by ${~}5$--7 percentage points. This aligns with the intuition that models of similar capacity learn similar decision boundaries.

Asymmetry favoring large-to-small transfer. Adversarials generated by larger models transfer better to smaller targets than vice versa. We hypothesize that larger models learn more complex perturbation directions that subsume those of smaller models, while smaller models' adversarial subspace is a strict subset.

Depth as an independent factor. The depth mismatch effect (${~}23.5$ pp drop) is comparable to a $4\times$--$8\times$ width mismatch, suggesting that depth introduces qualitatively different feature representations that resist transfer even when parameter counts are similar.

Limitations

• Synthetic data: Gaussian clusters are linearly separable; real-world data may show different transfer patterns due to more complex decision boundaries.
• Single attack: FGSM is a simple one-step attack. Iterative attacks (PGD) or optimization-based attacks (C&W) may transfer differently.
• MLP only: We do not test convolutional or attention-based architectures, which may exhibit different transfer dynamics.
• Perfect clean accuracy: All models achieve 100% clean accuracy on this separable dataset, which may inflate transfer rates relative to harder tasks.

Conclusion

We present a systematic "phase diagram" of adversarial transferability as a function of model capacity ratio. The key insight is that transferability degrades monotonically with capacity mismatch, with an asymmetry favoring large-to-small transfer and an independent penalty for depth mismatch. These findings provide quantitative guidance for threat modeling in adversarial ML: the most vulnerable target models are those with similar capacity to the adversary's surrogate.

\bibliographystyle{plainnat}

References

• [szegedy2014intriguing] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In ICLR, 2014.

• [goodfellow2015explaining] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.

• [papernot2016transferability] N. Papernot, P. McDaniel, and I. Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv:1605.07277, 2016.

• [liu2017delving] Y. Liu, X. Chen, C. Liu, and D. Song. Delving into transferable adversarial examples and black-box attacks. In ICLR, 2017.

• [tramer2017space] F. Tram{`e}r, N. Papernot, I. Goodfellow, D. Boneh, and P. McDaniel. The space of transferable adversarial examples. arXiv:1704.03453, 2017.

• [demontis2019adversarial] A. Demontis, M. Melis, M. Pintor, M. Jagielski, B. Biggio, A. Oprea, C. Nita-Rotaru, and F. Roli. Why do adversarial attacks transfer? {E}xplaining transferability of evasion and poisoning attacks. In USENIX Security, 2019.