Tag: Supreme Court Oral Argument

Posted on

Visualizing the content of justices’ and advocates’ cross-references

In our previous two posts (this one and this one), we examined the implications of justices cross-referencing other justices at oral argument. In this post, we take the analysis a step further and focus on the content of those cross-references.

The figure below is a word cloud derived from every sentence in the Supreme Court oral argument transcripts from the 1985 to the 2017 Terms in which one justice refers to another sitting justice by name. (We also treated references to “the chief” and “the Chief Justice” as a reference to Rehnquist or Roberts, depending on the era.) The word cloud provides a visualization of the relative importance of the 100 most frequent words in that set, excluding common words such as and, the, it, to, …

Supreme Court Oral Argument, 1985 – 2017 Terms: Justice cross-reference word cloud 

Word clouds are the text mining equivalent of a Rorschach test. The frequency of each individual word in a “bag of words” is of course an objective fact; but the significance each viewer gives to that fact depends on his or her expectations. Our interpretation of the word cloud above is that when justices are referring back to other justices in the modern era, they are overwhelmingly attempting to go back to a specific question raised by that justice. “Question” is the dominant token in this word cloud, occurring more than twice as often as the next two tokens, “answer” and “think.” Likewise, “answer” suggests a reference is being made to an earlier question; “think” is more generic.


Supreme Court Oral Argument, 1985 – 2017 Terms: Advocate-justice cross-reference word cloud 

Looking at the same visualization for advocates in the above word cloud, we see questions are still important, but not so clearly dominant. In the advocate word cloud, “question” is only the third ranked token, after “think” and “court.”

Previously, we have shown that the modern era of oral argument is defined more by comments than by questions; we have also shown that questions are more often asked of one’s friends, and comments are put to one’s foes. Finally, we have shown that judicial cross-references are generally made between two justices who ultimately agree with one another in the case at hand. As such, it follows logically that justices’ cross-references tend to focus substantially on fellow justices’ questions. This content analysis further buttresses this prior evidence of the relationship between questions and agreement, comments and disagreement, and judicial cross-references and agreement.

Posted on

Cross-referencing one’s friends: A universal pattern among the justices

In our previous post, we showed that when one justice refers to one another by name at oral argument, it usually signals agreement. We also showed the practice of justice name checking each other has increased over time. The next natural question is whether this strategy is one utilized by all of the justices or only a select few. We might expect that there would be significant differences amongst the justices, since previously we observed significant variation in naming patterns among the justices when referring advocates. Do the same patterns emerge when justices use each other’s names?

Which justices cross-reference the most?

To explore these questions, the following table shows the cross-reference tendencies of a selection of justices serving from 1985 to 2017.

Speaker Makes Reference Is Referenced Agreement Ratio
Alito 0.48 1.13 2.60
Breyer 0.78 2.86 2.64
Ginsburg 0.77 1.69 2.77
Gorsuch 2.13 0.05 1.55
Kagan 1.47 0.90 3.46
Kennedy 2.05 1.30 2.71
O’Connor 0.29 2.22 4.05
Rehnquist 0.82 1.04 3.05
Roberts 1.04 1.12 3.22
Scalia 0.57 2.51 2.43
Sotomayor 1.62 0.67 3.41
Souter 1.47 0.53 3.02
Stevens 0.88 1.54 1.62
Thomas 0.76 0.02 3.00

The first column of the table above shows the rate per thousand words by which each justice references another justice. The results are very different for which justices cross-reference their colleagues than the advocates. Justices Gorsuch, Kennedy, Sotomayor, Souter, Kagan, and Chief Justice Roberts lead the pack in naming their brethren more than once per thousand words. In contrast, when using the names of advocates, Kagan, Rehnquist, Ginsburg, Scalia, and O’Connor make up the top 5. Only Kagan is on both top referencer lists. Kennedy, second only to Gorsuch in judicial cross-references, was barely discernible when measuring the use of advocates’ names. Clearly, then, using names serves a very different function when the justices are referring to colleagues than when referring to advocates.

Which justices are referenced most often?

The second column shows the tendency of each justice to be referred to by another justice. Some interesting patterns emerge here, too. First, Breyer and Scalia are way ahead of the others on this dimension. This could be because, as one of us (Jacobi) has argued elsewhere, their ideological and methodological distance led to such intense disagreements that their mutual interruptions dwarfed any others between any judicial pair. If that is what is driving these two justices’ high rates of cross-references, it would suggest that sometimes cross-references do indicate disagreement. However, it is also possible that these two justices constitute the contrasting personifications of not only liberal and conservative ideology but also of their methodologies, since Breyer and Scalia have each written influential books on their very contrasting constitutional views.

In fact, the data supports both of these interpretations. With 235 cross-references, Breyer referenced Scalia far more than he referenced any other justice—other conservatives, such as Alito and Roberts, were referenced by Breyer only 77 and 26 times, respectively (although they served with Breyer for less time). Similarly, Scalia most often referenced Breyer, with 182 references; in contrast, he referenced Ginsburg only 94 times, despite serving one year longer with her, and referenced Stevens 116 times, while serving even longer with him. Scalia and Breyer were exceptionally focused on each other, and seldom for reasons of agreement.

Yet, unlike with interruptions, Scalia and Breyer did not leave the other justices in their wakes. Kennedy was an active and bipartisan cross-referencer, invoking Roberts 239 times, Ginsburg 204 times, Breyer 201 times and Scalia 239 times. Other high pairings were Stevens referencing Scalia, at 225 times, Souter referencing Scalia, at 247 times, and Souter referencing Breyer, at 227 times. Significantly, though, Kennedy’s references to Ginsburg and Roberts were the only instances of any justice other than Breyer or Scalia being referenced more than 200 times by any other justice. Scalia and Breyer were exceptional not only in referencing each other, but in being referenced by others. As such, this supports the idea that these two justices’ high reference rates are a sign of their influence, even more so than the medians on the Court during this time, Kennedy and O’Connor.

The universality of cross-referencing one’s friends

The third column shows what we call “the agreement ratio”—the rate at which a justice is referenced (column 1) where that justice and the referencing justice are in agreement versus where they disagree (it does not relate to column 2). This is the most striking result: every single justice, even Justice Thomas, who talks so infrequently, displays the same trend, of cross-referencing other justices with whom the justice eventually will agree in the case in which they make the cross-reference. Once again, then, we have a potentially powerful predictor of eventual outcomes in Supreme Court cases, stemming from in-depth analysis of what happens in the oral argument of any given case.

Altogether, we can see that all of the contemporary justices cross reference each other overwhelmingly when they are in agreement. But the consistency only applies to doing the referencing: when it comes to being referenced, there are tiers of influence among the justices. The justices cross-reference selectively, particularly cross-referencing Justices Breyer and Scalia, two of the leading thinkers on the contemporary Court of their respective ideologies and methodologies.