Since the discovery of adversarial examples in machine learning, researchers have designed several techniques to train neural networks that are robust against different types of attacks (most notably ℓ∞ and ℓ2 based attacks). However, it has been observed that the defense mechanisms designed to protect against one type of attack often offer poor performance against the other. In this paper, we introduce Randomized Adversarial Training (RAT), a technique that is efficient both against ℓ2 and ℓ∞ attacks. To obtain this result, we build upon adversarial training, a technique that is efficient against ℓ∞ attacks, and demonstrate that adding random noise at training and inference time further improves performance against ℓ2 attacks. We then show that RAT is as efficient as adversarial training against ℓ∞ attacks while being robust against strong ℓ2 attacks. Our final comparative experiments demonstrate that RAT outperforms all state-of-the-art approaches against ℓ2 and ℓ∞ attacks.