Towards reliable hypothesis validation in social sensing applications

Abstract

Social sensing has become a new crowdsourcing application paradigm where humans function as sensors to report their observations about the physical world. While many previous studies in social sensing focus on the problem of ascertaining the reliability of data sources and the truthfulness of their reported claims (often known as truth discovery), this paper investigates a new problem of hypothesis validation where the goal is to validate some high-level statements (referred to as hypotheses) from the low-level statements (referred to as claims) embedded in the social sensing data. The truthfulness of hypotheses cannot be directly obtained from the truth discovery results and two key challenges are involved in solving the hypothesis validation problem: (i) how to match the hypotheses generated by end users to the relevant claims generated by social sensors? (ii) How to accurately validate the truthfulness of the hypotheses given the unknown reliability of data sources and unvetted truthfulness of the claims? This paper proposes a Reliable Hypothesis Validation (RHV) scheme to address the above challenges. In particular, we develop a critical claim selection approach to match the hypotheses with the relevant claims and derive an optimal solution to validate their truthfulness by exploring the complex relationship between hypotheses and claims. The performance of RHV scheme is evaluated on three datasets collected from real-world social sensing applications. The results show that the RHV scheme significantly outperformed the state-of-the-art baselines in terms of validating the truthfulness of hypotheses.