by , , , ,
Abstract:
Taxonomies of concepts are important across many application domains, for instance online shopping portals use catalogs to help users navigate and search for products. Task-dependent taxonomies, e.g., adapting the taxonomy to a specific cohort of users, can greatly improve the effectiveness of navigation and search. However, taxonomies are usually created by domain experts and hence designing task-dependent taxonomies can be an expensive process: this often limits the applications to deploy generic taxonomies. Crowdsourcing based techniques have the potential to provide a cost-efficient solution to building task-dependent taxonomies. In this paper, we present the first quantitative study to evaluate the effectiveness of these crowdsourcing based techniques. Our experimental study compares different task-dependent taxonomies built via crowdsourcing and generic taxonomies built by experts. We design randomized behavioral experiments on the Amazon Mechanical Turk platform for navigation tasks using these taxonomies resembling real-world applications such as product search. We record various metrics such as the time of navigation, the number of clicks performed, and the search path taken by a participant to navigate the taxonomy to locate a desired object. Our findings show that task-dependent taxonomies built by crowdsourcing techniques can reduce the navigation time up to 20%. Our results, in turn, demonstrate the power of crowdsourcing for learning complex structures such as semantic taxonomies.
Reference:
Evaluating Task-Dependent Taxonomies for Navigation Y. Sun, A. Singla, T. Yan, A. Krause, D. FoxIn AAAI Conference on Human Computation and Crowdsourcing (HCOMP), 2016
Bibtex Entry:
@inproceedings{sun16taxonomies,
	Author = {Yuyin Sun and Adish Singla and Tori Yan and Andreas Krause and Dieter Fox},
	Booktitle = {AAAI Conference on Human Computation and Crowdsourcing (HCOMP)},
	Month = {October},
	Title = {Evaluating Task-Dependent Taxonomies for Navigation},
	Year = {2016}}