May 27, 2022 · We propose a simple and scalable network architecture, the Multimodal Masked Autoencoder (M3AE), which learns a unified encoder for both vision and language data via masked token prediction.

Apr 4, 2022 · We propose a pre-training strategy called Multi-modal Multi-task Masked Autoencoders (MultiMAE).

We introduce Multi-modal Multi-task Masked Autoencoders (MultiMAE), an efficient and effective pre-training strategy for Vision Transformers. Given a small random sample of visible patches from multiple modalities, the MultiMAE pre-training objective is …

We propose a simple and scalable network architecture, the Multimodal Masked Autoencoder (M3AE), which learns a uni-fied encoder for both vision and language data via masked token prediction.

We introduce Multi-modal Multi-task Masked Autoencoders (MultiMAE), an efficient and effective pre-training strategy for Vision Transformers. Given a small random sample of visible patches from multiple modalities, the MultiMAE pre-training objective is …

1 day ago · Keywords: Survival prediction, Multimodal, Graph convolutional network, Improved masked autoencoder, attention, Feature fusion Suggested Citation: Suggested Citation Han, Cong and Han, Qi and Li, Zhong and Weng, Tengfei and Tian, Yuan and Ran, Dan and Ye, Peng and Cao, Long and Cao, Peng, Improved Masked Autoencoder-Based Multimodal Graph ...

3 days ago · We propose Denoising Masked Autoencoder (Deno-MAE), a novel multimodal autoencoder framework for denoising modulation signals during pretraining. DenoMAE extends the concept of masked autoencoders by incorporating multiple input modalities, including noise as an explicit modality, to enhance cross-modal learning and improve denoising performance. The network is pre-trained using unlabeled ...

May 1, 2025 · Multimodal Masked Autoencoder Pre-training for 3D MRI-Based Brain Tumor Analysis with Missing Modalities ... However, applying this paradigm to multimodal medical data introduces a challenge: most existing approaches assume that all imaging modalities are available during both pre-training and fine-tuning. In practice, missing modalities often ...

Oct 22, 2022 · In this paper, we present Multi-modal Multi-task Masked Autoencoders (MultiMAE), a simple and effective method to make masked autoencoding include multiple modalities and tasks (see Fig. 2).

This is a JAX/Flax re-implementation for the paper Multimodal Masked Autoencoders Learn Transferable Representations.