Cognitive Science Program

Do altmetrics work? Twitter and ten other social web services.(2013)
Mike Thelwall, Stefanie Haustein, Vincent Larivière and Cassidy R Sugimoto
PloS one, 8 (5), e64841

Abstract

Altmetric measurements derived from the social web are increasingly advocated and used as early indicators of article impact and usefulness. Nevertheless, there is a lack of systematic scientific evidence that altmetrics are valid proxies of either impact or utility although a few case studies have reported medium correlations between specific altmetrics and citation rates for individual journals or fields. To fill this gap, this study compares 11 altmetrics with Web of Science citations for 76 to 208,739 PubMed articles with at least one altmetric mention in each case and up to 1,891 journals per metric. It also introduces a simple sign test to overcome biases caused by different citation and usage windows. Statistically significant associations were found between higher metric scores and higher citations for articles with positive altmetric scores in all cases with sufficient evidence (Twitter, Facebook wall posts, research highlights, blogs, mainstream media and forums) except perhaps for Google+ posts. Evidence was insufficient for LinkedIn, Pinterest, question and answer sites, and Reddit, and no conclusions should be drawn about articles with zero altmetric scores or the strength of any correlation between altmetrics and citations. Nevertheless, comparisons between citations and metric values for articles published at different times, even within the same year, can remove or reverse this association and so publishers and scientometricians should consider the effect of time when using altmetrics to rank articles. Finally, the coverage of all the altmetrics except for Twitter seems to be low and so it is not clear if they are prevalent enough to be useful in practice.

Bibliometrics: Global gender disparities in science.(2013)
Vincent Larivière, Chaoqun Ni, Yves Gingras, Blaise Cronin and Cassidy R Sugimoto
Nature News, 504 (7479), 211

Abstract

Despite many good intentions and initiatives, gender inequality is still rife in science. Although there are more female than male undergraduate and graduate students in many countries 1, there are relatively few female full professors, and gender inequalities in hiring 2, earnings 3, funding 4, satisfaction 5 and patenting 6 persist.One focus of previous research has been the'productivity puzzle'. Men publish more papers, on average, than women 7, although the gap differs between fields and subfields. Women publish significantly fewer papers in areas in which research is expensive 8, such as high-energy physics, possibly as a result of policies and procedures relating to funding allocations 4. Women are less likely to participate in collaborations that lead to publication and are much less likely to be listed as either first or last author on a paper 7. There is no consensus on the reasons for these gender differences in …

Bias in peer review.(2013)
Carole J Lee, Cassidy R Sugimoto, Guo Zhang and Blaise Cronin
Journal of the American Society for Information Science and Technology, 64 (1), 17-Feb

Abstract

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self‐regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status …

Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature.(2014)
Stefanie Haustein, Isabella Peters, Cassidy R Sugimoto, Mike Thelwall and Vincent Larivière
Journal of the Association for Information Science and Technology, 65 (4), 656-669

Abstract

Data collected by social media platforms have been introduced as new sources for indicators to help measure the impact of scholarly research in ways that are complementary to traditional citation analysis. Data generated from social media activities can be used to reflect broad types of impact. This article aims to provide systematic evidence about how often Twitter is used to disseminate information about journal articles in the biomedical sciences. The analysis is based on 1.4 million documents covered by both PubMed and Web of Science and published between 2010 and 2012. The number of tweets containing links to these documents was analyzed and compared to citations to evaluate the degree to which certain journals, disciplines, and specialties were represented on Twitter and how far tweets correlate with citation impact. With less than 10% of PubMed articles mentioned on Twitter, its uptake is low in …

Mapping world scientific collaboration: Authors, institutions, and countries.(2012)
Ali Gazni, Cassidy R Sugimoto and Fereshteh Didegah
Journal of the American Society for Information Science and Technology, 63 (2), 323-335

Abstract

International collaboration is being heralded as the hallmark of contemporary scientific production. Yet little quantitative evidence has portrayed the landscape and trends of such collaboration. To this end, 14,000,000 documents indexed in Thomson Reuters's Web of Science (WoS) were studied to provide a state‐of‐the‐art description of scientific collaborations across the world. The results indicate that the number of authors in the largest research teams have not significantly grown during the past decade; however, the number of smaller research teams has seen significant increases in growth. In terms of composition, the largest teams have become more diverse than the latter teams and tend more toward interinstitutional and international collaboration. Investigating the size of teams showed large variation between fields. Mapping scientific cooperation at the country level reveals that Western countries …

Big data, bigger dilemmas: A critical review.(2015)
Hamid Ekbia, Michael Mattioli, Inna Kouper, Gary Arave, Ali Ghazinejad, Timothy Bowman ...
Journal of the Association for Information Science and Technology, 66 (8), 1523-1545

Abstract

The recent interest in Big Data has generated a broad range of new academic, corporate, and policy practices along with an evolving debate among its proponents, detractors, and skeptics. While the practices draw on a common set of tools, techniques, and technologies, most contributions to the debate come either from a particular disciplinary perspective or with a focus on a domain‐specific issue. A close examination of these contributions reveals a set of common problematics that arise in various guises and in different places. It also demonstrates the need for a critical synthesis of the conceptual and practical dilemmas surrounding Big Data. The purpose of this article is to provide such a synthesis by drawing on relevant writings in the sciences, humanities, policy, and trade literature. In bringing these diverse literatures together, we aim to shed light on the common underlying issues that concern and affect all of …

The cognitive structure of library and information science: Analysis of article title words.(2011)
Staša Milojević, Cassidy R Sugimoto, Erjia Yan and Ying Ding
Journal of the American Society for Information Science and Technology, 62 (10), 1933-1953

Abstract

This study comprises a suite of analyses of words in article titles in order to reveal the cognitive structure of Library and Information Science (LIS). The use of title words to elucidate the cognitive structure of LIS has been relatively neglected. The present study addresses this gap by performing (a) co‐word analysis and hierarchical clustering, (b) multidimensional scaling, and (c) determination of trends in usage of terms. The study is based on 10,344 articles published between 1988 and 2007 in 16 LIS journals. Methodologically, novel aspects of this study are: (a) its large scale, (b) removal of non‐specific title words based on the “word concentration” measure (c) identification of the most frequent terms that include both single words and phrases, and (d) presentation of the relative frequencies of terms using “heatmaps”. Conceptually, our analysis reveals that LIS consists of three main branches: the traditionally …

Scholarly use of social media and altmetrics: A review of the literature.(2017)
Cassidy R Sugimoto, Sam Work, Vincent Larivière and Stefanie Haustein
Journal of the Association for Information Science and Technology, 68 (9), 2037-2062

Abstract

Social media has become integrated into the fabric of the scholarly communication system in fundamental ways, principally through scholarly use of social media platforms and the promotion of new indicators on the basis of interactions with these platforms. Research and scholarship in this area has accelerated since the coining and subsequent advocacy for altmetrics—that is, research indicators based on social media activity. This review provides an extensive account of the state‐of‐the art in both scholarly use of social media and altmetrics. The review consists of 2 main parts: the first examines the use of social media in academia, reviewing the various functions these platforms have in the scholarly communication process and the factors that affect this use. The second part reviews empirical studies of altmetrics, discussing the various interpretations of altmetrics, data collection and methodological limitations …

Tweets as impact indicators: Examining the implications of automated “bot” accounts on T witter.(2016)
Stefanie Haustein, Timothy D Bowman, Kim Holmberg, Andrew Tsou, Cassidy R Sugimoto and Vincent Larivière
Journal of the Association for Information Science and Technology, 67 (1), 232-238

Abstract

This brief communication presents preliminary findings on automated Twitter accounts distributing links to scientific articles deposited on the preprint repository arXiv. It discusses the implication of the presence of such bots from the perspective of social media metrics (altmetrics), where mentions of scholarly documents on Twitter have been suggested as a means of measuring impact that is both broader and timelier than citations. Our results show that automated Twitter accounts create a considerable amount of tweets to scientific articles and that they behave differently than common social bots, which has critical implications for the use of raw tweet counts in research evaluation and assessment. We discuss some definitions of Twitter cyborgs and bots in scholarly communication and propose distinguishing between different levels of engagement—that is, differentiating between tweeting only bibliographic …

A bibliometric chronicling of library and information science's first hundred years.(2012)
Vincent Larivière, Cassidy R Sugimoto and Blaise Cronin
Journal of the American Society for Information Science and Technology, 63 (5), 997-1016

Abstract

This paper presents a condensed history of Library and Information Science (LIS) over the course of more than a century using a variety of bibliometric measures. It examines in detail the variable rate of knowledge production in the field, shifts in subject coverage, the dominance of particular publication genres at different times, prevailing modes of production, interactions with other disciplines, and, more generally, observes how the field has evolved. It shows that, despite a striking growth in the number of journals, papers, and contributing authors, a decrease was observed in the field's market‐share of all social science and humanities research. Collaborative authorship is now the norm, a pattern seen across the social sciences. The idea of boundary crossing was also examined: in 2010, nearly 60% of authors who published in LIS also published in another discipline. This high degree of permeability in LIS was …

Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact.(2014)
Blaise Cronin and Cassidy R Sugimoto
MIT Press.

Abstract

A comprehensive, state-of-the-art examination of the changing ways we measure scholarly performance and research impact. Bibliometrics has moved well beyond the mere tracking of bibliographic citations. The web enables new ways to measure scholarly productivity and impact, making available tools and data that can reveal patterns of intellectual activity and impact that were previously invisible: mentions, acknowledgments, endorsements, downloads, recommendations, blog posts, tweets. This book describes recent theoretical and practical advances in metrics-based research, examining a variety of alternative metrics—or “altmetrics”—while also considering the ethical and cultural consequences of relying on metrics to assess the quality of scholarship. Once the domain of information scientists and mathematicians, bibliometrics is now a fast-growing, multidisciplinary field that ranges from webometrics to scientometrics to influmetrics. The contributors to Beyond Bibliometrics discuss the changing environment of scholarly publishing, the effects of open access and Web 2.0 on genres of discourse, novel analytic methods, and the emergence of next-generation metrics in a performance-conscious age. Contributors Mayur Amin, Judit Bar-Ilan, Johann Bauer, Lutz Bornmann, Benjamin F. Bowman, Kevin W. Boyack, Blaise Cronin, Ronald Day, Nicola De Bellis, Jonathan Furner, Yves Gingras, Stefanie Haustein, Edwin Henneken, Peter A. Hook, Judith Kamalski, Richard Klavans, Kayvan Kousha, Michael Kurtz, Mark Largent, Julia Lane, Vincent Larivière, Loet Leydesdorff, Werner Marx, Katherine W. McCain, Margit Palzenberger, Andrew Plume …

Team size matters: Collaboration and scientific impact since 1900.(2015)
Vincent Larivière, Yves Gingras, Cassidy R Sugimoto and Andrew Tsou
Journal of the Association for Information Science and Technology, 66 (7), 1323-1332

Abstract

This article provides the first historical analysis of the relationship between collaboration and scientific impact using three indicators of collaboration (number of authors, number of addresses, and number of countries) derived from articles published between 1900 and 2011. The results demonstrate that an increase in the number of authors leads to an increase in impact, from the beginning of the last century onward, and that this is not due simply to self‐citations. A similar trend is also observed for the number of addresses and number of countries represented in the byline of an article. However, the constant inflation of collaboration since 1900 has resulted in diminishing citation returns: Larger and more diverse (in terms of institutional and country affiliation) teams are necessary to realize higher impact. The article concludes with a discussion of the potential causes of the impact gain in citations of collaborative papers.

The shifting sands of disciplinary development: Analyzing North American Library and Information Science dissertations using latent Dirichlet allocation.(2011)
Cassidy R Sugimoto, Daifeng Li, Terrell G Russell, S Craig Finlay and Ying Ding
Journal of the American Society for Information Science and Technology, 62 (1), 185-204

Abstract

This work identifies changes in dominant topics in library and information science (LIS) over time, by analyzing the 3,121 doctoral dissertations completed between 1930 and 2009 at North American Library and Information Science programs. The authors utilize latent Dirichlet allocation (LDA) to identify latent topics diachronically and to identify representative dissertations of those topics. The findings indicate that the main topics in LIS have changed substantially from those in the initial period (1930–1969) to the present (2000–2009). However, some themes occurred in multiple periods, representing core areas of the field: library history occurred in the first two periods; citation analysis in the second and third periods; and information‐seeking behavior in the fourth and last period. Two topics occurred in three of the five periods: information retrieval and information use. One of the notable changes in the topics was …

Scientists popularizing science: characteristics and impact of TED talk presenters.(2013)
Cassidy R Sugimoto, Mike Thelwall, Vincent Larivière, Andrew Tsou, Philippe Mongeon and Benoit Macaluso
PloS one, 8 (4), e62403

Abstract

The TED (Technology, Entertainment, Design) conference and associated website of recorded conference presentations (TED Talks) is a highly successful disseminator of science-related videos, claiming over a billion online views. Although hundreds of scientists have presented at TED, little information is available regarding the presenters, their academic credentials, and the impact of TED Talks on the general population. This article uses bibliometric and webometric techniques to gather data on the characteristics of TED presenters and videos and analyze the relationship between these characteristics and the subsequent impact of the videos. The results show that the presenters were predominately male and non-academics. Male-authored videos were more popular and more liked when viewed on YouTube. Videos by academic presenters were more commented on than videos by others and were more liked on YouTube, although there was little difference in how frequently they were viewed. The majority of academic presenters were senior faculty, males, from United States-based institutions, were visible online, and were cited more frequently than average for their field. However, giving a TED presentation appeared to have no impact on the number of citations subsequently received by an academic, suggesting that although TED popularizes research, it may not promote the work of scientists within the academic community.

P‐Rank: An indicator measuring prestige in heterogeneous scholarly networks.(2011)
Erjia Yan, Ying Ding and Cassidy R Sugimoto
Journal of the American Society for Information Science and Technology, 62 (3), 467-477

Abstract

Ranking scientific productivity and prestige are often limited to homogeneous networks. These networks are unable to account for the multiple factors that constitute the scholarly communication and reward system. This study proposes a new informetric indicator, P‐Rank, for measuring prestige in heterogeneous scholarly networks containing articles, authors, and journals. P‐Rank differentiates the weight of each citation based on its citing papers, citing journals, and citing authors. Articles from 16 representative library and information science journals are selected as the dataset. Principle Component Analysis is conducted to examine the relationship between P‐Rank and other bibliometric indicators. We also compare the correlation and rank variances between citation counts and P‐Rank scores. This work provides a new approach to examining prestige in scholarly communication networks in a more …