Semantic similarity models for automated fact-checking: ClaimCheck as a claim matching tool

• Larraz, Irene ^[1] ; Míguez, Rubén ^[2] ; Sallicati, Francesca ^[2]
  1. [1] Universidad de Navarra

     Universidad de Navarra

     Pamplona, España

     
  2. [2] Newtral
• Localización: Profesional de la información, ISSN-e 1699-2407, ISSN 1386-6710, Vol. 32, Nº 3, 2023 (Ejemplar dedicado a: Network activisms)
• Idioma: inglés
• DOI: 10.3145/epi.2023.may.21
• Títulos paralelos:
  • Uso de modelos de similitud semántica para la automatización de fact-checking: ClaimCheck como software de claim matching
• Enlaces
  • Texto completo
• Dialnet Métricas: 5 Citas
• Resumen
  • español

    Este artículo presenta el diseño experimental de ClaimCheck, un programa de inteligencia artificial para detectar men-tiras repetidas en el discurso político a partir de un modelo de similitud semántica desarrollado por el medio de verificación Newtral en colaboración con ABC Australia. El estudio revisa el estado del arte sobre el uso de algoritmos para fact-checking y propone una definición de claim matching. Además, detalla el esquema de anotación de frases similares y presenta los resultados de los experimentos con el programa.

  • English

    This article presents the experimental design of ClaimCheck, an artificial intelligence tool for detecting repeated falsehoods in political discourse using a semantic similarity model developed by the fact-checking organization Newtral in collaboration with ABC Australia. The study reviews the state of the art in algorithmic fact-checking and proposes a definition of claim matching. Additionally, it outlines the scheme for annotating similar sentences and presents the results of experiments conducted with the tool.

• Referencias bibliográficas
  • Adair, Bill (2021). “The lessons of Squash, Duke’s automated fact-checking platform”. Poynter, 16 June. https://www.poynter.org/fact-checking/2021/the-lessons-of-squash-the-first-automated-fact-checking-platform
  • Adair, Bill; Li, Chengkai; Yang, Jun; Yu, Cong (2018). Automated pop-up fact-checking: challenges & progress. https://ranger.uta.edu/~cli/pubs/2019/popupfactcheck-cj19-adair.pdf
  • Agadjanian, Alexander; Bakhru, Nikita; Chi, Victoria; Greenberg, Devyn; Hollander, Byrne; Hurt, Alexander; Kind, Joseph; Lu, Ray; Ma, Annie;...
  • Arslan, Fatma (2021). Modeling factual claims with semantic frames: definitions, datasets, tools, and fact-checking applications. Doctoral...
  • Babakar, Mevan; Moy, Will (2016). The state of automated factchecking. How to make factchecking dramatically more effective with technology...
  • Baker, Collin F.; Fillmore, Charles J.; Lowe, John B. (1998). “The Berkeley FrameNet project”. In: Proceedings of the joint conference of...
  • Beltrán, Javier; Míguez, Rubén; Larraz, Irene (2019). “ClaimHunter: an unattended tool for automated claim detection on Twitter”. KnOD@WWW....
  • Corney, David (2021a). “How does automated fact checking work?”. Full Fact, 5 July. https://fullfact.org/blog/2021/jul/how-does-automated-fact-checking-work
  • Corney, David (2021b). “Towards a common definition of claim matching”. Full Fact, 5 October. https://fullfact.org/blog/2021/oct/towards-common-definition-claim-matching
  • Dolan, William B.; Brockett, Chris (2005). “Automatically constructing a corpus of sentential paraphrases”. In: Proceedings of the third international...
  • Floodpage, Sebastien (2021). “How fact checkers and Google.org are fighting misinformation”. Google, 31 March. https://blog.google/outreach-initiatives/google-org/fullfact-and-google-fight-misinformation
  • Graves, Lucas (2018). Understanding the promise and limits of automated fact-checking. Reuters Institute for the Study of Journalism. Factsheets....
  • Hassan, Aumyo; Barber, Sarah J. (2021). “The effects of repetition frequency on the illusory truth effect”. Cognitive research: principles...
  • Hassan, Naeemul; Adair, Bill; Hamilton, James T.; Li, Chengkai; Tremayne, Mark; Yang, Jun; Yu, Cong (2015). “The quest to automate fact-checking”....
  • Hassan, Naeemul; Arslan, Fatma; Li, Chengkai; Tremayne, Mark (2017). “Toward automated fact-checking: detecting check-worthy factual claims...
  • Hövelmeyer, Alica; Boland, Katarina; Dietze, Stefan (2022). “SimBa at CheckThat! 2022: lexical and semantic similarity based detection of...
  • Jiang, Ye; Song, Xingyi; Scarton, Carolina; Aker, Ahmet; Bontcheva, Kalina (2021). “Categorising fine-to-coarse grained misinformation: an...
  • Kazemi, Ashkan; Garimella, Kiran; Gaffney, Devin; Hale, Scott A. (2021). “Claim matching beyond English to scale global fact-checking”. In:...
  • Kazemi, Ashkan; Li, Zehua; Pérez-Rosas, Verónica; Hale, Scott A.; Mihalcea, Rada (2022). “Matching tweets with applicable fact-checks across...
  • Kessler, Glenn; Fox, Joe (2021). “The false claims that Trump keeps repeating”. The Washington Post, 20 January. https://www.washingtonpost.com/graphics/politics/fact-checker-most-repeated-disinformation
  • Lan, Zhenzhong; Chen, Mingda; Goodman, Sebastian; Gimpel, Kevin; Sharma, Piyush; Soricut, Radu (2020). “ALBERT: a lite Bert for self-supervised...
  • Lim, Chloe (2018). “Checking how fact-checkers check”. Research & politics, v. 5, n. 3. https://doi.org/10.1177/2053168018786848
  • Mansour, Watheq; Elsayed, Tamer; Al-Ali, Abdulaziz (2022). “Did I see it before? Detecting previously-checked claims over Twitter”. Lecture...
  • Martín, Alejandro; Huertas-Tato, Javier; Huertas-García, Álvaro; Villar-Rodríguez, Guillermo; Camacho, David (2021). “FacTeR-check: semi-automated...
  • Mukherjee, Amit; Sela, Eitan; Al-Saadoon, Laith (2020). “Building an NLU-powered search application with Amazon SageMaker and the Amazon opensearch...
  • Murray, Samuel; Stanley, Matthew; McPhetres, Jon; Pennycook, Gordon; Seli, Paul (2020). “‘I’ve said it before and I will say it again…’: repeating...
  • Nakov, Preslav; Corney, David; Hasanain, Maram; Alam, Firoj; Elsayed, Tamer; Barrón-Cedeño, Alberto; Papotti, Paolo; Shaar, Shaden; Da-San-Martino,...
  • Nakov, Preslav; Da-San-Martino, Giovanni; Alam, Firoj; Shaar, Shaden; Mubarak, Hamdy; Babulkov, Nikolay (2022). “Overview of the CLEF-2022...
  • Nguyen, Vincent; Karimi, Sarvnaz; Xing, Zhenchang (2021). “Combining shallow and deep representations for text-pair classification”. In: Proceedings...
  • Phillips, Whitney (2018). The oxygen of amplification. Better pratices for reporting on extremists, antagonists, and manipulators online....
  • Porter, Ethan; Wood, Thomas J. (2021). “The global effectiveness of fact-checking: Evidence from simultaneous experiments in Argentina, Nigeria,...
  • Real, Andrea (2021). “Casado mezcla diferentes estadísticas de empleo para asegurar que hay 4 millones de parados, pero es falso”. Newtral,...
  • Reimers, Nils; Gurevych, Iryna (2019). “Sentence-bert: sentence embeddings using siamese bert-networks”. In: Proceedings of the 2019 Conference...
  • Shaar, Shaden; Alam, Firoj; Da-San-Martino, Giovanni; Nakov, Preslav (2021a). “The role of context in detecting previously fact-checked claims”....
  • Shaar, Shaden; Babulkov, Nikolay; Da-San-Martino, Giovanni; Nakov, Preslav (2020). “That is a known lie: detecting previously fact-checked...
  • Shaar, Shaden; Haouari, Fatima; Mansour, Watheq; Hasanain, Maram; Babulkov, Nikolay; Alam, Firoj; Da-San-Martino, Giovanni; Elsayed, Tamer;...
  • Sheng, Qiang; Cao, Juan; Zhang, Xueyao; Li, Xirong; Zhong, Lei (2021). “Article reranking by memory-enhanced key sentence matching for detecting...
  • Sippitt, Amy (2020). What is the impact of fact checkers’ work on public figures, institutions and the media?. Africa Check, Chequeado and...
  • Stanford Institute for Human-Centered Artificial Intelligence (2023). Artificial intelligence index. Stanford University. https://aiindex.stanford.edu/report
  • The Washington Post (2018). “Meet the bottomless Pinocchio | Fact Checker”. [Video]. YouTube, 10 December. https://www.youtube.com/watch?v=zoS1sVZRfUU
  • Thorne, James; Vlachos, Andreas (2018). “Automated fact checking: task formulations, methods and future directions”. Arxiv. https://doi.org/10.48550/arXiv.1806.07687
  • Wardle, Claire (2018). “Lessons for reporting in an age of disinformation”. Medium, 28 December. https://medium.com/1st-draft/5-lessons-for-reporting-in-an-age-of-disinformation-9d98f0441722
  • Zeng, Xia; Abumansour, Amani S.; Zubiaga, Arkaitz (2021). “Automated fact-checking: a survey”. Language and linguistics compass, v. 15, n....