Cover Image

Is Misinformation More Open? A Study of robots.txt Gatekeeping on the Web

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Steinacker-Olsztyn, Nicolas, Gosain, Devashish, Dao, Ha
Format:	Preprint
Published:	2025
Subjects:	Computers and Society
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Scrapers selectively respect robots.txt directives: evidence from a large-scale empirical study
by: Kim, Taein, et al.
Published: (2025)

"Nobody should control the end user": Exploring Privacy Perspectives of Indian Internet Users in Light of DPDPA
by: Athar, Sana, et al.
Published: (2025)

The Liabilities of Robots.txt
by: Chang, Chien-yi, et al.
Published: (2025)

Misinformation by Omission: The Need for More Environmental Transparency in AI
by: Luccioni, Sasha, et al.
Published: (2025)

From Verification to Amplification: Auditing Reverse Image Search as Algorithmic Gatekeeping in Visual Misinformation Fact-checking
by: Lin, Cong, et al.
Published: (2026)

Intractable Cookie Crumbs: Unveiling the Nexus of Stateful Banner Interaction and Tracking Cookies
by: Rasaii, Ali, et al.
Published: (2025)

Auditing the Impact of Cross-Site Web Tracking on YouTube Political and Misinformation Recommendations
by: Chouaki, Salim, et al.
Published: (2026)

An integrated view of Quantum Technology? Mapping Media, Business, and Policy Narratives
by: Suter, Viktor, et al.
Published: (2024)

Explore the Potential of LLMs in Misinformation Detection: An Empirical Study
by: Chen, Mengyang, et al.
Published: (2023)

Generative Debunking of Climate Misinformation
by: Zanartu, Francisco, et al.
Published: (2024)

Rescuing Counterspeech: A Bridging-Based Approach to Combating Misinformation
by: Peng, Kenny, et al.
Published: (2024)

Navigating the Ethics of Internet Measurement: Researchers' Perspectives from a Case Study in the EU
by: Amin, Sahibzada Farhan, et al.
Published: (2025)

The Psychology of Falsehood: A Human-Centric Survey of Misinformation Detection
by: Nandi, Arghodeep, et al.
Published: (2025)

Digital Gatekeeping: An Audit of Search Engine Results shows tailoring of queries on the Israel-Palestine Conflict
by: Damião, Íris, et al.
Published: (2025)

Robust Misinformation Detection by Visiting Potential Commonsense Conflict
by: Wang, Bing, et al.
Published: (2025)

Web crawler strategies for web pages under robot.txt restriction
by: Vyas, Piyush, et al.
Published: (2023)

A Guide to Misinformation Detection Data and Evaluation
by: Thibault, Camille, et al.
Published: (2024)

Evaluating the Simulation of Human Personality-Driven Susceptibility to Misinformation with LLMs
by: Pratelli, Manuel, et al.
Published: (2025)

Diverse Misinformation: Impacts of Human Biases on Detection of Deepfakes on Networks
by: Lovato, Juniper, et al.
Published: (2022)

AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wild
by: Dufour, Nicholas, et al.
Published: (2024)

Mobile Health Text Misinformation Identification Using Mobile Data Mining
by: Hu, Wen-Chen, et al.
Published: (2024)

Decoding the Sociotechnical Dimensions of Digital Misinformation: A Comprehensive Literature Review
by: Puska, Alisson Andrey, et al.
Published: (2024)

The Dynamics of (Not) Unfollowing Misinformation Spreaders
by: Ashkinaze, Joshua, et al.
Published: (2024)

SoK: Machine Learning for Misinformation Detection
by: Xiao, Madelyne, et al.
Published: (2023)

When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms
by: Chun, Chaewan, et al.
Published: (2026)

The Goldilocks zone of governing technology: Leveraging uncertainty for responsible quantum practices
by: Meckel, Miriam, et al.
Published: (2025)

Misinformation Dissemination: Effects of Network Density in Segregated Communities
by: Karimi, Soroush, et al.
Published: (2024)

Building Resilience to Misinformation: A Cross-National Development of the Digital Media and Information Literacy Scale (DMILS)
by: Qian, Sijia, et al.
Published: (2026)

Synergy: A Next-Generation General-Purpose Agent for Open Agentic Web
by: Nie, Xiaohang, et al.
Published: (2026)

You Can't Fool Us: Understanding the Resilience of LLM-driven Agent Communities to Misinformation
by: Lin, Chichen, et al.
Published: (2026)

MERIT: Modular Framework for Multimodal Misinformation Detection with Web-Grounded Reasoning
by: Shopnil, Mir Nafis Sharear, et al.
Published: (2025)

Diverse, but Divisive: LLMs Can Exaggerate Gender Differences in Opinion Related to Harms of Misinformation
by: Neumann, Terrence, et al.
Published: (2024)

Merging AI Incidents Research with Political Misinformation Research: Introducing the Political Deepfakes Incidents Database
by: Walker, Christina P., et al.
Published: (2024)

Crafting Synthetic Realities: Examining Visual Realism and Misinformation Potential of Photorealistic AI-Generated Images
by: Peng, Qiyao, et al.
Published: (2024)

When Truth Misleads -- Phase-Aware Coherence Detection for Misinformation Correction Across Epistemic Communities
by: Müller, Heimo, et al.
Published: (2026)

ViMGuard: A Novel Multi-Modal System for Video Misinformation Guarding
by: Kan, Andrew, et al.
Published: (2024)

Are Rationales Necessary and Sufficient? Tuning LLMs for Explainable Misinformation Detection
by: Wang, Bing, et al.
Published: (2026)

How Good Is Open Bicycle Infrastructure Data? A Countrywide Case Study of Denmark
by: Vierø, Ane Rahbek, et al.
Published: (2023)

A Peek Behind the Curtain: Using Step-Around Prompt Engineering to Identify Bias and Misinformation in GenAI Models
by: Hickerson, Don, et al.
Published: (2025)

Artificial Intelligence and Misinformation in Art: Can Vision Language Models Judge the Hand or the Machine Behind the Canvas?
by: Fu, Tarian, et al.
Published: (2025)