A vision encoder is a necessary component for allowing many leading LLMs to be able to work with images uploaded by users.

The core idea is to embed wavelet transform into CNN architecture to reduce the resolution of feature maps while at the same time, increasing receptive field. Specifically, MWCNN for image restoration ...