Lying with statistics: Public Citizen edition

Businesses “More Likely to Sue Frivolously” trumpets Bizarro-Overlawyered and Greedy Trial Lawyer, quoting a Public Citizen report. Except not even the Public Citizen report supports this claim, and no mathematically-literate person reading the report could think so.

Here’s what the Public Citizen press release says: “The study also found that when these corporations did file lawsuits, they and their attorneys were 69% more likely than individual tort plaintiffs and their attorneys to be sanctioned by federal judges for filing frivolous claims or defenses.”

In fact, all the study did was look at the most recent hundred Rule 11 sanctions issued by federal judges. As we’ve discussed before, what Rule 11 considers “frivolous” is considerably narrower what the common non-lawyer considers “frivolous.” But let’s take Public Citizen’s methodology at face value and ignore the issue with the small sample size, or the fact that Public Citizen didn’t investigate whether the sanctions were reversed on appeal. Out of the 100 sanctions, a whopping thirty-five were issued to corporations. Thus, corporations, say Public Citizen, are more likely to be sanctioned.

Now, those of you who have graduated second grade might be wondering, “But isn’t 35 a minority of 100?” And you would be right. Public Citizen gerrymandered corporations into the leading source of sanctions by (1) combining all corporate suits into one group; and (2) separating individual suits into three groups: individual tort suits with an attorney, individual non-tort suits with an attorney, and pro se suits. Once those 65 sanctions are divided into three different categories, only 22 of them are for represented individuals suing in tort, and 35 is 69% more than 22. Thus, says Public Citizen, corporate lawsuits are 69% more likely to be sanctioned than individual tort plaintiffs.

Except even this manipulatedly bogus statistic is wrong on its own terms. “More likely” requires a denominator. And, as Public Citizen points out, corporations file 3.3 to 5.8 times as many lawsuits as individual tort plaintiff, in part because half of all corporate suits are contract suits, and Public Citizen omitted individual contract suits from the other side of the equation. But only 1.7 times as many sanctions are issued to corporations. Do the junior-high-school math with denominators, and one finds that, in fact, using Public Citizen’s data, individual tort plaintiffs represented by attorneys are between 107% and 264% more likely to be sanctioned for frivolous filings: Public Citizen and DMI got it precisely backwards. (NB also that Public Citizen’s press release made the similar mathematically-illiterate error of translating “3.3 times as many lawsuits” into “3.3 times more likely to file a lawsuit,” despite the absence of a comparable denominator: Costco, for example, engages in millions of times as many transactions as an individual.)

Of course, the main problem with the study is that it’s attacking a straw-man. Have you seen a reformer that doesn’t care about corporate litigation? Certainly not the reformers who post to Overlawyered, because many of the lawsuits mentioned in the 2004 Public Citizen study were criticized here first: April 2000; Jun. 29, 2004; Nov. 22, 2003. It’s almost as if they got the cases from Overlawyered itself. The study also criticizes the number of subrogation suits without mentioning that many reformers wish to abolish the collateral source rule that makes subrogation suits necessary. (I’m not one of those reformers, though I believe in other reforms that would reduce the need for separate subrogation litigation.) Regular Overlawyered commenter David Nieporent notes:

1) Since the subject is “tort reform,” why are you bringing up a study that talks about litigation generally, rather than tort litigation? The study itself makes clear that it includes such litigation as landlord-tenant and mortgage foreclosures, not to mention routine collection cases. Indeed, businesses almost never file tort suits, if you check the data at the back of the study.

2) Why you think “hourly defense firms,” rather than trial lawyers, are filing these suits on behalf of businesses is beyond me. You think “hourly defense firms” are handling routine collection work or landlord-tenant cases?

3) Counting the number of suits is pretty misleading, since suits can have different numbers of parties. One class action may have thousands of plaintiffs. If we’re trying to measure the rate at which individuals sue, such a suit should count as thousands of suits, not one; the fact that those are consolidated tells you nothing about how likely individuals are to sue.

That Cyrus Dugger and GTL got it so wrong shows the danger of reflexively regurgitating trial-lawyer propaganda without thinking.


  • But:

    (1) Whatever its faults, Rule 11 does supply a pre-existing standard for “frivolousness” that a partisan can’t jigger to suit the occasion. And if Rule 11’s definition is indeed narrower than most people’s everyday understanding of the term, then the only net effect, seemingly, would be that we’re discussing comparative propensities to file claims at the most “frivolous” end of the spectrum – a datum of interest in itself, and also for what it says about propensities to file slightly less frivolous ones.

    (2) I’m not a statistician, but some rough back-of-the envelope calculations suggest that a sample size of 100 will make for a margin of error of under 10%, at the 95% confidence level. Not the narrowest margin of error, but not one of staggering breadth either. The problem may be partly that to get the sample much larger, you’d need to draw it from a longer temporal period. A span of nearly 3 years is needed to accumulate the sample of 100. To get the margin of error to the neighborhood of 5%, you’d need perhaps a decade’s worth of cases, by my amateur reckoning. To get it to the neighborhood of 3%, you might need about three decades. At that point, we’re arguably doing historiography – talking, as we’d be, about the Ford administration. Even leaving the historiography objection aside, we can’t get three decades of data on Rule 11 sanctions, because the modern sanctions provisions were added to Rule 11 only in 1983.

    (3) Many putative “reforms” are specifically targeted at tort suits brought by individuals against businesses. A picture is often painted according to which it’s those suits that are responsible for increased business costs, insurance premiums, consumer prices, etc. And in that oft-painted picture, it’s largely greedy contingent-fee plaintiffs’ lawyers who foment the litigation. So it’s a legitimate intellectual mission to compare individual tort suits brought by plaintiffs’ counsel with other categories of litigation. I understand that some “reformers” also register token complaints, or more than token complaints, about lawsuits brought by businesses. But if the claim is that we have an overlitigious society, the problem is partly to identify who’s allegedly being overlitigious.

    (4) I would probably fault any study that didn’t segregate pro se litigants. Pro se litigants are much more likely to commit a sanctionable offense from sheer ignorance, because as a group, they don’t know the procedural and substantive law as well. Nor do pro se suits constitute instances of the alleged problem portrayed in the aforementioned, oft-painted picture: greedy contingent-fee plaintiffs’ lawyers who supposedly foment frivolous litigation. (I understand that you’ve previously raised clever arguments that would blame plaintiffs’ lawyers for frivolous pro se suits too, but those arguments have no evident empirical support and indeed seem almost empty of testable content.) Last but not least, corporations are prohibited, in most places, from even appearing pro se. If a hot-headed corporation can’t find a lawyer who’ll bring the case, it has nowhere to go. If a hot-headed individual can’t, he can still be the architect of his own folly.

    (5) You’ve articulated a scathing critique of one English-language sentence, from a press release, summarizing the statistics. It’s important, I agree, that sentences be accurate, in press releases and elsewhere. But the issue is also the numbers themselves. The following sentence, for example, would be a basically accurate summary of the Public Citizen numbers (with that caveat about the margin of error): “When a lawyer in a federal lawsuit is sanctioned for frivolous conduct, he’s 69% more likely to be representing a business than to be representing an individual suing in tort.” There isn’t one right denominator, you know. It’s a matter of what aspect of the numbers might interest someone. And if the question on the floor is who’s responsible for the costs of the frivolous litigation that is supposedly placing such a damper on commerce, it’s scant consolation, perhaps, if the responsible parties also bring a very high number of more meritorious suits.

    (6) If it’s true, by the way, that corporations file 3.3 to 5.8 more lawsuits than individuals, despite being so outnumbered by those individuals, then that fact, by itself, may deserve more prominence than “reformers” tend to give it.

    (7) Your “straw-man” point seems strained. Speaking for myself, I have indeed seen putative “reformers” who don’t seem to “care about corporate litigation.” I’ve very certainly seen few who make crusading against corporate litigiousness their top priority. (See item 6.) In any event: that you and some other “reformers” may also advocate “reforms” that would affect business litigation isn’t particularly germane to an empirical inquiry into what the sources of alleged existing problems may be. Indeed, the empirical findings might turn out to suggest that the existing problems have been misidentified, misdiagnosed, or exaggerated. Certainly it should aid public understanding, in the face of all the overheated rhetoric people have heard, if information on the prevalence of different types of “frivolous” litigation is presented in quantitative terms.

    (8) I’ve not gone back to read David Nieporent’s comments in their original context, but they seem, perhaps, to have been addressed to other parts of the Public Citizen report. I don’t think routine collection actions or landlord-tenant disputes figure prominently in the federal caseload. Meanwhile, anybody who equates one class action with the maintenance of thousands of individual lawsuits doesn’t understand very well how class actions work. But that last topic I must leave for another occasion.

  • While Peter has some excellent points, what bothers me most is that PC (how whimsical) wrote the report (and the press release) with such a deliberate end in mind.

  • I read both Peter’s comment (in full, not scanning) and wavemaker’s.

    wavemaker makes a much better point, and he did it in roughly 3% of the space.


    An honest tort reform advocate (such as myself) DOES NOT CARE who instigated the suit. The anti-tort reform side ACCUSES us of being “pro-business”. I want reform that makes our system FAIRER and MORE JUST; I want that equally for everyone, Joe Citizen and XYZ Megacorp.

    Specific points:

    1. Analogy: to get Rul 11 sanctions takes the procedural equivalent of murder. How many murders there are tells us little of how many non-fatal assaults there are. Same thing here – it gives little more than a vague notion.

    2. Too long since statistics for me (10+ years), but off-hand, 10% margin on sample size of only 100 seems… not right, anyway. Could be wrong.

    3. That’s because businesses seldom sue on individual for $1 gazillion over “pain and suffring” among other excesses, but many “putative “reforms”” are also targetted against businesses running their citizn opponent out of money with litigation fees – different problems, different solutions. Whether one is more common than the other, both are common, and both need to be addressed.

    4. “corporations are prohibited, in most places, from even appearing pro se.”
    So, citizens get a pass? That is, we can just remove thos frivolous cases from the count because corporations are legally preventd from doing it quite the same way? That DOES NOT help your argument. As to “oft-painted picture”, it’s quit an accurate picture of some circumstances and of some of the prverse incntivs created by our system. *If* there are fewer lawyers succumbing to the temptation than it sometimes, well, yay… that dosn’t mean we don’t need to fix the system.

    5. Your “one sentence” is certainly more accurate, but that accuracy comes at the expense of alerting the reader that you’ve had to narrow the field significantly to get your happy little statistic. That is, it tells the reader that there’s some significant manipulation going on. It does have the distinction of being honest, but, well, PC (and others) are obviously willing to do without that distinction. That was a large part of the point of this post.

    6. Even with the point granted, it still puts the lie headline. And, on the point, Ted mentioned, that includes contract law, which is a whole different ball of wax, and corporations have a grat many more activities which with could potentially lead to lawsuits. Another way to consider this would be lawsuits/person – a corporation made up of 1000 employees has a lot more points of contact and a lot more potential for screw ups than the actions of any one person.

    7. Th “straw-man” seems strained to you because you haven’t been on the receiving end of accusations of being a shill for corporate America. ALL THE FRICKIN TIME. If you have indeed seen “reformers” who don’t care about corporate litigation, then those people might indeed be open to such accusations, but to lump veryone in with them is like lumping all Christians in with that Phelps guy who goes around to military funerals and such with his gay-bashing signs. And information on “frivolous” lawsuits does indeed aid public understand… of how the legal definition of “frivolous” is so narrowly defined as to be useless. Going back to my analogy in point 1: if the law didn’t criminalize assualt unless the victim DIED, do you think the public would give a rat’s hind-quarters that “crime” was low or who did it? Not really – they’d be too busy tarring and feathering politicians and police chiefs who didn’t protect them from getting the crap beat out of them all the time.

    8. Beats me.

  • Another way to address the “margin of error” issue, without getting into a morass over how to compute it, would be to observe that the PC study purported to sample 100% of Rule 11 cases over a roughly three-year period. So the margin of error is zero, in one sense, over that period — assuming that PC did indeed capture every case. Again, it all depends on what the question is — i.e., on whether we’re using the Rule 11 cases to make some kind of extrapolation to non-Rule 11 cases.

    My personal belief is that the frequency of Rule 11 sanctions is not an especially precise indicator of frivolousness. Its use varies as a function of too many factors other than frivolousness — as does the imposition of sanctions when the rule is invoked. So to me, the results of the PC Rule 11 study are indicative and suggestive only. They may call certain aspects of some people’s received wisdom into question, but that’s the most they do.

  • i aced jr. high math but i have to admit that you lost me somewhere along the way. how did you arrive at 107% and 264%?

    [TF: 22 is 107% greater than 35 / 3.3 ; it’s 264% greater than 35 / 5.8.]

  • Peter: if PC had said “When a lawyer in a federal lawsuit is sanctioned for frivolous conduct, he’s 69% more likely to be representing a business than to be representing an individual suing in tort,” I would have no complaint, because that’s an accurate statement of the study.

    But that wasn’t what Public Citizen said. What Public Citizen said, and what Dugger and GTL regurgitated, was fundamentally dishonest.