Wednesday, July 3, 2024

"The telltale words that could identify generative AI text"

As noted in the first of two posts on FSLR, July 2:

....The article concludes by saying "This article was generated with the support of AI and reviewed by an editor. For more information see our T&C.".

Which, if you've read enough of these things you could tell by the third paragraph....

From Ars Technica, July 1:

New paper counts "excess words" that started appearing more often in the post-LLM era.

Thus far, even AI companies have had trouble coming up with tools that can reliably detect when a piece of writing was generated using a large language model. Now, a group of researchers has established a novel method for estimating LLM usage across a large set of scientific writing by measuring which "excess words" started showing up much more frequently during the LLM era (i.e., 2023 and 2024). The results "suggest that at least 10% of 2024 abstracts were processed with LLMs," according to the researchers. 

In a pre-print paper posted earlier this month, four researchers from Germany's University of Tubingen and Northwestern University said they were inspired by studies that measured the impact of the COVID-19 pandemic by looking at excess deaths compared to the recent past. By taking a similar look at "excess word usage" after LLM writing tools became widely available in late 2022, the researchers found that "the appearance of LLMs led to an abrupt increase in the frequency of certain style words" that was "unprecedented in both quality and quantity."

Delving in
To measure these vocabulary changes, the researchers analyzed 14 million paper abstracts published on PubMed between 2010 and 2024, tracking the relative frequency of each word as it appeared across each year. They then compared the expected frequency of those words (based on the pre-2023 trendline) to the actual frequency of those words in abstracts from 2023 and 2024, when LLMs were in widespread use.

The results found a number of words that were extremely uncommon in these scientific abstracts before 2023 that suddenly surged in popularity after LLMs were introduced. The word "delves," for instance, shows up in 25 times as many 2024 papers as the pre-LLM trend would expect; words like "showcasing" and "underscores" increased in usage by nine times as well. Other previously common words became notably more common in post-LLM abstracts: the frequency of "potential" increased 4.1 percentage points; "findings" by 2.7 percentage points; and "crucial" by 2.6 percentage points, for instance....

....MUCH MORE

See if you can tell whether this paper was written by an LLM or a person:

Glaciers, gender, and science
A feminist glaciology framework for global environmental change research

Abstract

Glaciers are key icons of climate change and global environmental change. However, the relationships among gender, science, and glaciers – particularly related to epistemological questions about the production of glaciological knowledge – remain understudied. This paper thus proposes a feminist glaciology framework with four key components: 1) knowledge producers; (2) gendered science and knowledge; (3) systems of scientific domination; and (4) alternative representations of glaciers. Merging feminist postcolonial science studies and feminist political ecology, the feminist glaciology framework generates robust analysis of gender, power, and epistemologies in dynamic social-ecological systems, thereby leading to more just and equitable science and human-ice interactions.

I Introduction

Glaciers are icons of global climate change, with common representations stripping them of social and cultural contexts to portray ice as simplified climate change yardsticks and thermometers. In geophysicist Henry Pollack’s articulation, ‘Ice asks no questions, presents no arguments, reads no newspapers, listens to no debates. It is not burdened by ideology and carries no political baggage as it crosses the threshold from solid to liquid. It just melts’ (Pollack, 2009: 114). This perspective appears consistently in public discourse, from media to the Intergovernmental Panel on Climate Change (IPCC). But the ‘ice is just ice’ conceptualization contrasts sharply with conclusions by researchers such as Cruikshank (2005), who asks if glaciers listen, Orlove et al. (2008b), who analyze the cultural framing of glaciers, Carey (2007), who sees an endangered species narrative applied to glaciers, Jackson (2015), who exposes how glaciers are depicted as ruins, and Sörlin (2015), who refers to the present as a cryo-historical moment because ‘ice has become historical, i.e. that ice is an element of change and thus something that can be considered as part of society and of societal concern’ (Sörlin, 2015: 327).....