Our focus is on the diversity of generated paraphrase sentences, as high diversity in paraphrase generation improves robustness on many downstream applications by improving generalization.