Chris Thomas

Selected Publications

1. 

   MetaSumPerceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checking

   Ting-Chih Chen, Chia-Wei Tang, and Christopher Thomas

   In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024

   Abs Bib Website

   Fact-checking real-world claims often requires reviewing multiple multimodal documents in order to assess the claim’s truthfulness, a highly laborious and time-consuming task. In this paper, we present a summarization model crafted to generate claim-specific summaries useful for fact-checking from multimodal multi-document datasets. The model takes inputs in the form of documents, images, and a claim, with the objective of assisting in fact-checking tasks. We introduce a dynamic perceiver-based model that is able to handle inputs from multiple modalities of arbitrary lengths. To train our model, we leverage a novel reinforcement learning-based entailment objective in order to generate summaries that provide evidence distinguishing between different truthfulness labels. To assess the efficacy of our approach, we conduct experiments on both an existing benchmark as well as a new dataset of multi-document claims which we contribute. Our approach outperforms the SOTA approach by 4.6% in the claim verification task on the MOCHEG dataset and demonstrates strong performance on our new Multi-News-Fact-Checking dataset.

   @inproceedings{chen-etal-2024-metasumperceiver,
     title = {{M}eta{S}um{P}erceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checking},
     author = {Chen, Ting-Chih and Tang, Chia-Wei and Thomas, Christopher},
     editor = {Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek},
     booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
     year = {2024},
     address = {Bangkok, Thailand},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2024.acl-long.474},
     pages = {8742--8757},
   }

2. 

   Semantic Shield: Defending Vision-Language Models Against Backdooring and Poisoning via Fine-grained Knowledge Alignment

   Alvi Md Ishmam, and Christopher Thomas

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

   Bib Website

   @inproceedings{ishmam2024semantic,
     title = {Semantic Shield: Defending Vision-Language Models Against Backdooring and Poisoning via Fine-grained Knowledge Alignment},
     author = {Ishmam, Alvi Md and Thomas, Christopher},
     booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
     pages = {24820--24830},
     year = {2024},
   }

3. 

   Beyond Grounding: Extracting Fine-Grained Event Hierarchies across Modalities

   Hammad Ayyubi, Christopher Thomas, Lovish Chum, and 8 more authors

   In Proceedings of the AAAI Conference on Artificial Intelligence, 2024

   Bib Website

   @inproceedings{ayyubi2024beyond,
     title = {Beyond Grounding: Extracting Fine-Grained Event Hierarchies across Modalities},
     author = {Ayyubi, Hammad and Thomas, Christopher and Chum, Lovish and Lokesh, Rahul and Chen, Long and Niu, Yulei and Lin, Xudong and Feng, Xuande and Koo, Jaywon and Ray, Sounak and others},
     booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
     volume = {38},
     number = {16},
     pages = {17664--17672},
     year = {2024},
   }

4. 

   Fine-Grained Visual Entailment

   Christopher Thomas, Yipeng Zhang, and Shih-Fu Chang

   In Proceedings of the European Conference on Computer Vision, 2022

   arXiv Bib HTML

   @inproceedings{thomas2022fine,
     title = {Fine-Grained Visual Entailment},
     author = {Thomas, Christopher and Zhang, Yipeng and Chang, Shih-Fu},
     booktitle = {Proceedings of the European Conference on Computer Vision},
     pages = {398--416},
     year = {2022},
   }

5. 

   InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection

   Yi Fung, Christopher Thomas, Revanth Reddy, and 6 more authors

   In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), 2021

   Bib HTML

   @inproceedings{fung2021infosurgeon,
     title = {InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection},
     author = {Fung, Yi and Thomas, Christopher and Reddy, Revanth and Polisetty, Sandeep and Ji, Heng and Chang, Shih-Fu and McKeown, Kathleen and Bansal, Mohit and Sil, Avi},
     booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL)},
     year = {2021},
   }

6. 

   Preserving Semantic Neighborhoods for Robust Cross-modal Retrieval

   Christopher Thomas, and Adriana Kovashka

   In Proceedings of the European Conference on Computer Vision (ECCV), 2020

   Bib HTML PDF Website

   @inproceedings{thomas2020preserving,
     title = {Preserving Semantic Neighborhoods for Robust Cross-modal Retrieval},
     author = {Thomas, Christopher and Kovashka, Adriana},
     booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
     year = {2020},
   }

7. 

   Predicting the politics of an image using webly supervised data

   Christopher Thomas, and Adriana Kovashka

   In Advances in Neural Information Processing Systems (NeurIPS 2019), 2019

   Bib PDF Supp Poster Slides Website

   @inproceedings{thomas2019predicting,
     title = {Predicting the politics of an image using webly supervised data},
     author = {Thomas, Christopher and Kovashka, Adriana},
     booktitle = {Advances in Neural Information Processing Systems (NeurIPS 2019)},
     year = {2019},
   }