#Abstract In this paper, we present a user study aimed at understanding how human observers judge scatter plot similarity when presented with a large set of iconic scatter plot representations. The study intends to advance the state of the art of the use of quality measures as an exploration and discovery tool in information visualization. Quality measures have been designed and developed to help human analysts look into large sets of plots aided by computational methods that try to capture/simulate human perception. The work we present, involves 18 participants with a scientific background in a similarity perception study. The study asks the participants to group a carefully selected set of plots into groups according to their subjective similarity judgments, and it integrates this result into a consensus similarity grouping we use to generate insights on similarity perception. The main output of the study is a list of concepts we derive to describe major perceptual features and a description of how these concepts relate and rank. The study also include the evaluation of the scagnostics - popular scatter plot descriptors, showing that they are not able to reliable reproduce the judgments generated by our participants. The paper includes a discussion of the major implications of this study and how these results can be used for future research. (Download preprint here), ACM Digital Library Link (not available yet)