Category: Name Check

Posted on

Visualizing the content of justices’ and advocates’ cross-references

In our previous two posts (this one and this one), we examined the implications of justices cross-referencing other justices at oral argument. In this post, we take the analysis a step further and focus on the content of those cross-references.

The figure below is a word cloud derived from every sentence in the Supreme Court oral argument transcripts from the 1985 to the 2017 Terms in which one justice refers to another sitting justice by name. (We also treated references to “the chief” and “the Chief Justice” as a reference to Rehnquist or Roberts, depending on the era.) The word cloud provides a visualization of the relative importance of the 100 most frequent words in that set, excluding common words such as and, the, it, to, …

Supreme Court Oral Argument, 1985 – 2017 Terms: Justice cross-reference word cloud 

Word clouds are the text mining equivalent of a Rorschach test. The frequency of each individual word in a “bag of words” is of course an objective fact; but the significance each viewer gives to that fact depends on his or her expectations. Our interpretation of the word cloud above is that when justices are referring back to other justices in the modern era, they are overwhelmingly attempting to go back to a specific question raised by that justice. “Question” is the dominant token in this word cloud, occurring more than twice as often as the next two tokens, “answer” and “think.” Likewise, “answer” suggests a reference is being made to an earlier question; “think” is more generic.


Supreme Court Oral Argument, 1985 – 2017 Terms: Advocate-justice cross-reference word cloud 

Looking at the same visualization for advocates in the above word cloud, we see questions are still important, but not so clearly dominant. In the advocate word cloud, “question” is only the third ranked token, after “think” and “court.”

Previously, we have shown that the modern era of oral argument is defined more by comments than by questions; we have also shown that questions are more often asked of one’s friends, and comments are put to one’s foes. Finally, we have shown that judicial cross-references are generally made between two justices who ultimately agree with one another in the case at hand. As such, it follows logically that justices’ cross-references tend to focus substantially on fellow justices’ questions. This content analysis further buttresses this prior evidence of the relationship between questions and agreement, comments and disagreement, and judicial cross-references and agreement.

Posted on

Cross-referencing one’s friends: A universal pattern among the justices

In our previous post, we showed that when one justice refers to one another by name at oral argument, it usually signals agreement. We also showed the practice of justice name checking each other has increased over time. The next natural question is whether this strategy is one utilized by all of the justices or only a select few. We might expect that there would be significant differences amongst the justices, since previously we observed significant variation in naming patterns among the justices when referring advocates. Do the same patterns emerge when justices use each other’s names?

Which justices cross-reference the most?

To explore these questions, the following table shows the cross-reference tendencies of a selection of justices serving from 1985 to 2017.

Speaker Makes Reference Is Referenced Agreement Ratio
Alito 0.48 1.13 2.60
Breyer 0.78 2.86 2.64
Ginsburg 0.77 1.69 2.77
Gorsuch 2.13 0.05 1.55
Kagan 1.47 0.90 3.46
Kennedy 2.05 1.30 2.71
O’Connor 0.29 2.22 4.05
Rehnquist 0.82 1.04 3.05
Roberts 1.04 1.12 3.22
Scalia 0.57 2.51 2.43
Sotomayor 1.62 0.67 3.41
Souter 1.47 0.53 3.02
Stevens 0.88 1.54 1.62
Thomas 0.76 0.02 3.00

The first column of the table above shows the rate per thousand words by which each justice references another justice. The results are very different for which justices cross-reference their colleagues than the advocates. Justices Gorsuch, Kennedy, Sotomayor, Souter, Kagan, and Chief Justice Roberts lead the pack in naming their brethren more than once per thousand words. In contrast, when using the names of advocates, Kagan, Rehnquist, Ginsburg, Scalia, and O’Connor make up the top 5. Only Kagan is on both top referencer lists. Kennedy, second only to Gorsuch in judicial cross-references, was barely discernible when measuring the use of advocates’ names. Clearly, then, using names serves a very different function when the justices are referring to colleagues than when referring to advocates.

Which justices are referenced most often?

The second column shows the tendency of each justice to be referred to by another justice. Some interesting patterns emerge here, too. First, Breyer and Scalia are way ahead of the others on this dimension. This could be because, as one of us (Jacobi) has argued elsewhere, their ideological and methodological distance led to such intense disagreements that their mutual interruptions dwarfed any others between any judicial pair. If that is what is driving these two justices’ high rates of cross-references, it would suggest that sometimes cross-references do indicate disagreement. However, it is also possible that these two justices constitute the contrasting personifications of not only liberal and conservative ideology but also of their methodologies, since Breyer and Scalia have each written influential books on their very contrasting constitutional views.

In fact, the data supports both of these interpretations. With 235 cross-references, Breyer referenced Scalia far more than he referenced any other justice—other conservatives, such as Alito and Roberts, were referenced by Breyer only 77 and 26 times, respectively (although they served with Breyer for less time). Similarly, Scalia most often referenced Breyer, with 182 references; in contrast, he referenced Ginsburg only 94 times, despite serving one year longer with her, and referenced Stevens 116 times, while serving even longer with him. Scalia and Breyer were exceptionally focused on each other, and seldom for reasons of agreement.

Yet, unlike with interruptions, Scalia and Breyer did not leave the other justices in their wakes. Kennedy was an active and bipartisan cross-referencer, invoking Roberts 239 times, Ginsburg 204 times, Breyer 201 times and Scalia 239 times. Other high pairings were Stevens referencing Scalia, at 225 times, Souter referencing Scalia, at 247 times, and Souter referencing Breyer, at 227 times. Significantly, though, Kennedy’s references to Ginsburg and Roberts were the only instances of any justice other than Breyer or Scalia being referenced more than 200 times by any other justice. Scalia and Breyer were exceptional not only in referencing each other, but in being referenced by others. As such, this supports the idea that these two justices’ high reference rates are a sign of their influence, even more so than the medians on the Court during this time, Kennedy and O’Connor.

The universality of cross-referencing one’s friends

The third column shows what we call “the agreement ratio”—the rate at which a justice is referenced (column 1) where that justice and the referencing justice are in agreement versus where they disagree (it does not relate to column 2). This is the most striking result: every single justice, even Justice Thomas, who talks so infrequently, displays the same trend, of cross-referencing other justices with whom the justice eventually will agree in the case in which they make the cross-reference. Once again, then, we have a potentially powerful predictor of eventual outcomes in Supreme Court cases, stemming from in-depth analysis of what happens in the oral argument of any given case.

Altogether, we can see that all of the contemporary justices cross reference each other overwhelmingly when they are in agreement. But the consistency only applies to doing the referencing: when it comes to being referenced, there are tiers of influence among the justices. The justices cross-reference selectively, particularly cross-referencing Justices Breyer and Scalia, two of the leading thinkers on the contemporary Court of their respective ideologies and methodologies.

Posted on

What does it mean when justices reference each other at oral argument?

What’s in a name?

Previously, we showed that justices often refer to advocates by name as a form of “throat clearing,” that is, using prefatory words to ease into interrupting an advocate. Since interruptions are highly predictive of justices voting against the person they interrupt, that may suggest justices use of an advocate’s name is likely to signal disagreement. Is the pattern the same when justices use each other’s names? There is reason to think not: the justices may refer to each when they want an advocate to return to a question of shared interest. In this post, we begin to explore the phenomena of justice-to-justice name checking and conclude that, although there is significant variation between justices and over time, the practice is highly correlated with, and predictive of, agreement among the justices.

Judicial cross-references over time

As we have shown elsewhere, oral argument has changed significantly in recent decades, so the first natural question is whether judicial cross-references have changed also. The following figure shows a distinct shift in the practice of justice-to-justice name-checking over time. The figure shows the rate of justices referring to each other by name at oral argument from the 1985 to 2017 Terms, normalized per thousand words spoken by the justices.

Justices Naming Other Justices at Supreme Court Oral Argument (1985-2017 Terms)

It is important to normalize figures by word count in order to adjust for the fact that the justices talk more during oral argument than they used to. In the mid-1990s, the justices began to play a far more active role in oral argument; the justices now talk for about 13 minutes more in the course of a 60-minute proceeding than they did in the pre-1995 era. . Yet even controlling for that enormous upward shift in judicial verbosity, justices referring to each other by name has measurably increased over the last 30 years. Since 2007, the rate of justice-to-justice name checking has been consistently above one per thousand words. Given the competitive cut and thrust of oral argument, that makes name-checking a significant part of the dialogue among the justices.

Why the increase has occurred is a matter of interpretation. At least part of the cause is likely to be a response to the other dramatic changes we have observed in oral argument, in particular the significant increase in judicial activity at oral argument. With justices talking so much more and interrupting each other and the advocates more often, and the advocates interrupting back at greater rates, it is easy to have questions go unanswered or partially answered. As such, we would expect justices to refer back to earlier questions that they want to hear the answer to.

That raises the associated question of when the justices refer back to each other’s questions. If the above interpretation is correct, it suggests that Justice A will be refer to Justice B when he or she wants an advocate to complete a previously disrupted answer to Justice B’s question. That in turn suggests that cross-references will occur among justices who are likely to agree rather than to disagree.

The relationship between agreement and justice-to-justice name checking

The following figure explores that agreement-based intuition. It shows the rate of judicial cross-references, once again normalized per thousand words spoken by the justices, but this time divided among whether the cross-references occur between justices who ultimately vote together, or those who ultimately disagree, in the case at hand.

Justices Naming Other Justices at Supreme Court Oral Argument (1985-2017 Terms) (Agree, Disagree)

The results support the interpretation above: justices cross reference each other about three times as often when they will ultimately agree with the person they are cross-referencing than when with they will ultimately disagree. This means that whereas we have a “disagreement gap” for justices interrupting advocates, we have an “agreement gap” between justices cross-referencing each other. (Note we could also develop a disagreement gap between justices, as Jacobi and Rozema’s research shows interruptions are associated with future disagreement among pairs of justices).

Note also that the very large increase seen in the first figure is primarily being driven by an increase in justices cross-referencing those with whom they agree, rather than among justices who disagree. Put another way, while there has always been a tendency to more often cross reference one’s friends than one’s foes, the difference is considerably increasing in the last three decades. This result suggests that cross-references may be a counter-strategy to the more conflictual strategies we have previously highlighted, particularly of taking up the time of the advocate with whom a justice disagrees. By pointing an advocate back to a question of a future ally, the justice is emphasizing what he or she agrees with, rather than solely what he or she disagrees with.

In a time of increasing partisan division on the Court, mirroring the increased partisan conflict in Congress and among the public, here at least we have a manifestation of agreement rather than disagreement. Of course, agreeing with one’s allies is a form of tribalism, as much as disagreeing with one’s foes, but next week we explore in more detail which justices specifically cross reference which other justices, and the broader implications of such behavioral patterns.