KT-GAN: Knowledge-Transfer Generative Adversarial Network for Text-to-Image Synthesis. Academic Article uri icon

abstract

  • This paper presents a new framework, Knowledge-Transfer Generative Adversarial Network (KT-GAN), for fine-grained text-to-image generation. We introduce two novel mechanisms: an Alternate Attention-Transfer Mechanism (AATM) and a Semantic Distillation Mechanism (SDM), to help generator better bridge the cross-domain gap between text and image. The AATM updates word attention weights and attention weights of image sub-regions alternately, to progressively highlight important word information and enrich details of synthesized images. The SDM uses the image encoder trained in the Image-to-Image task to guide training of the text encoder in the Text-to-Image task, for generating better text features and higher-quality images. With extensive experimental validation on two public datasets, our KT-GAN outperforms the baseline method significantly, and also achieves the competive results over different evaluation metrics.

published proceedings

  • IEEE Trans Image Process

author list (cited authors)

  • Tan, H., Liu, X., Liu, M., Yin, B., & Li, X.

citation count

  • 31

complete list of authors

  • Tan, Hongchen||Liu, Xiuping||Liu, Meng||Yin, Baocai||Li, Xin

publication date

  • January 2021