Driving while looking in the rearview mirror

8 March 2024, 1004 EST

The takeaway from last night’s State of the Union address is that Biden’s language continued decades-long trends of decreasing positive emotion, use of the term “we,” and overall fewer words per sentence. Polarization data comes from V-Dem and the speeches were analyzed with LIWC. The State of the Union is a useful tool for seeing the U.S. in context over time, as well as demonstrating features that pundits this morning have suggested are insights not into the state of our country, but the state of Biden’s presidency. The speech itself is highly vetted and wordsmithed, and even under the watchful eye of speechwriters and policy advisors we see patterns that transcend parties and individual language features.

In this first figure (where language trends are shown as z-scores calculated using all SOTU speeches from 1789 forward), we see that word count has remained fairly stable but all presidents post-1970 have used fewer and fewer words per sentence. Short and sweet sentences make for good soundbites, and reflect the audience preferences for more accessible messages. It’s not exactly a social media effect, though, as word count by sentence have been on the decline for the past four decades.

There are several theories from language‘, psychology, and cognition studies that suggest how more complicated language is more difficult to parse and retain (in English, anyway). (Note: It’s not clear that other languages that follow different syntactic structures – English is SVO, or subject-verb-object – or those that are more oral than written, would find more lexically dense information as complex.) The structure of SOTU speeches has changed dramatically since 1789, in audience, structure, and complexity. The moral of the story here is that fewer words per sentence is a trend that predates the prevalence of online communication – thinking and writing in 140 characters. We can’t blame the Internet for everything.

Polarization increases precipitously post-Cold War, word count remains stable post-WWII, and words per sentence declines over the past four decades.

Sentiment analysis using an off-the-shelf and dictionary-based tool is somewhat of a blunt instrument, but also a trusty indicator given the constrained lexicon of the State of the Union speeches. Negative emotion has waffled, but positive emotion has steadily declined as polarization has increased. This is somewhat unsurprising, but also stark to see against the backdrop of rising polarization in American society.

Tentative language has remained fairly stable, but certainty language has decreased as polarization has increased. In the Hofstede cultural dimensions perspective, the United States has a low tolerance for uncertainty. More uncertainty likely means more anxiety, and is indicative of a more reactive and reactionary system. People don’t always make the best choices and decisions under crisis conditions, so the decrease in certainty terms may reflect overall less certainty in the system, and this is particularly worrisome.

Next, and somewhat unsurprisingly, there is a decrease in the use of “we’ pronouns (we, us, our, ours). Not the most pronounced trend, but taken alongside less positive sentiment this is relevant. “We” are not all in this together.

This is perhaps the most fascinating. References to the past trend tightly with polarization. As polarization increases, so do references to the past, while post-2000 references to the present and future have declined.

We are driving looking in the rearview mirror.

If you like this analytical snapshot but want more information on syntax and language structure in the SOTU, I’d be happy to show some pretty pictures in a follow-up post.