Lucia de Berk and the amateur statisticians

My father was a judge. The only reason for bringing this up is that the cheapest argument, which has been used to fight people who have an opinion on the Lucia de Berk case which runs against the court decisions, is to say that they have no understanding of how the Dutch law system works. This “argument” has even recently been used by the “statistician of the court” Henk Elffers. It is similar to the argument, used against people expressing a negative opinion on “Apartheid” in South Africa at its prime time: “You must have been there to make a judgment”. So if it is going to be like this, I can only say that I have a lot of first hand experience with Dutch Law and Dutch lawyers.

I have been following the Lucia de Berk case for several years now. Among other things, I attended, on April 2, 2004, a discussion afternoon of the Dutch Statistical Society at the “Vrije Universiteit” in Amsterdam. The statistics professor Aad van der Vaart was chairman of that meeting, and among the discussants was Henk Elffers. In the intermission I had a short conversation with him. Henk Elffers and I knew each other from the time that we were at the Mathematical Centre, Amsterdam. Although part of our conversation was on music, I also asked him whether there was some “hard” evidence for Lucia’s guilt. Then he mentioned the traces of intoxication found in the blood of the victims, which at that time still sounded rather convincing. In view of this, his computation of the very small probability of the “coincidences” in the Lucia de Berk case, which had not convinced me at all, sounded like an academic exercise without much consequence. However, at present I think that this computation is one of the terrible errors in a whole chain of unfortunate errors that have led to the conviction of Lucia de Berk.

How did all this start? One of the initiators was Mr. Smits, general director of the Juliana children’s hospital and the Red Cross hospital in the Hague, the first amateur statistician on the scene. As a good manager, he had access to the Microsoft Excel spreadsheet tool, containing a button called “random”. Using this, on the (retrospectively) historical day of September 4, 2001, he (or one of his employees, this is not entirely clear) made a quick computation of the probability that Lucia de Berk had been present with so many “incidents” (as had been “observed” by other nurses). And by this quick computation he became convinced that so many incidents “could not be a coincidence”. Twelve days later the police was contacted and Lucia de Berk was accused of 5 murders and 5 attempted murders in the Juliana children’s hospital. In the mean time Mr. Smits had also looked into the possibility of additional murders by Lucia de Berk in other hospitals and yes, more potential murders were quickly found. As expressed in the book of Ton Derksen on the Lucia de Berk case (“Lucia de B. Reconstruction of a Miscarriage of Justice”, in Dutch): “We have the suspect. Could you please find for us the murders she committed?”. And yes again, on September 17, 2001, Mr. Smits can report to the police 4 additional suspicious deaths in his other hospital, the Red Cross hospital.

On top of all this, the boards of Mr. Smits’ hospitals contacted the morning newspaper “De Telegraaf” (having a high profile on sensation news, in this respect somewhat comparable to the British “The Sun”). This newspaper issued on September 17, 2001, an item on the case, which in fact stated that a nurse in the two hospitals had possibly been involved in the killing of several adults and children. The newspaper item also said that the directors of the hospitals regretted very much what had happened and that they expressed their sympathy with the parents (after expressing their concern that the trust of patients in their hospitals might be somewhat shaken by this message). Note that this suggested that at this point the “murders” had already been proved “beyond reasonable doubt” (I note in passing that the Dutch law system does not make use of this concept, I am sorry to say). And if all this were still not enough, a spokes(wo)man of the office of the Prosecution made it known (in the same news item of “De Telegraaf”) that “the possibility of more cases” (of killing or attempted killing by Lucia de Berk) “was not excluded”. All this 13 days after the Microsoft Excel spreadsheet calculation!

Ironically, another spreadsheet table of director Smits (exhibited on p. 22 of the second edition of Derksen’s book) shows that in the years that Lucia de Berk was not employed by the Juliana children’s hospital there were in fact more deaths than during the time that Lucia de Berk was employed by the hospital, whereas one would be inclined to expect that the opposite would hold if a serial killer were around in this hospital. Moreover, all the suspicious death cases of which Lucia was accused had been declared “natural” and now had to be redeclared as “unnatural” by the hospital. But anyway, after all this heat an unstoppable chain reaction started, Henk Elffers did his nonsensical computation and the court in The Hague had enough to start the prosecution.

Long ago, while still a student, I gave, in cooperation with Willem Albert Wagenaar and Gerard de Zeeuw, a course called “The intuitive statistician”. This course was investigating what people understand of “randomness”, for example by presenting them with sequences produced by a random number generator (and also with sequences with some very strong embedded structure). Many experiments have shown that people have the tendency to expect random sequences of coin flippings to have much faster alternation of heads and tails than in in fact happens in random sequences of independent flippings. And indeed, the attendants of our course were also prone to this fallacy. This particular fallacy is called the “gambler’s fallacy”. I think that Mr. Smits’ unfounded calculation arose from a similar type of faulty intuition.

This brings up a more general point. In such an important case as the case of Lucia de Berk, where even the Dutch supreme court was involved, the court could have consulted a professor of mathematical statistics in the Netherlands. The court could have consulted, for example, professor Aad van der Vaart in Amsterdam, professor Sara van de Geer in Leiden (at that time, now she is professor of statistics in Zurich), professor Richard Gill in Utrecht (at present professor of mathematical statistics in Leiden) and also myself in Delft. Why didn’t the court do that? Why were they relying on amateur statisticians all the time?

How would the judges feel if, at the “moment suprême”, just before being rolled into the surgery room for a brain operation and perhaps wondering why the anesthesiologist did not already send them into unconsciousness, the nurse would casually mention that the operation would be performed by an amateur surgeon?

First there was amateur statistician Mr. Smits. After that there was, I am sorry to say, amateur statistician Henk Elffers. On the personal level I have absolutely nothing against Henk. But I think that he made a formal and totally pointless calculation, using the Fisher exact test for a 2×2 table, about the only thing he learned from his professor of mathematical statistics in Amsterdam (see below). As I mentioned above, Henk Elffers and I were at the Mathematical Centre at the same time (although I stayed a little bit longer, after finishing my dissertation). At the time that Richard Gill and I were writing our dissertations at the Mathematical Centre, Henk did not write a dissertation, and in fact went on to start another study, and to do completely different things (a summary of his career is in fact given on the Home Page of Henk Elffers). That’s in itself fine of course, but it means that he lost contact with what is going on in mathematical statistics. Mathematical statistics is not about balls and boxes. Mathematical statistics is about the choice of an appropriate model. In that sense mathematical statistics is also fundamentally different from Probability Theory, and certainly from discrete Probability Theory, working with balls and boxes. Probability Theory works with one model of which it explores all the consequences. Mathematical statistics usually deals with a continuum of models of which it tries to pick the most appropriate one. One can wrestle through 100 pages of boxes and balls without understanding anything of what mathematical statistics is about.

Unfortunately Henk Elffers had his training from a professor of mathematical statistics who spent almost all his teaching time on balls and boxes (erroneously assuming that this was needed before going on to the real thing). I know, because I followed courses of the same professor of mathematical statistics, whom I actually succeeded at the University of Amsterdam in 1984. He was also our boss at the Mathematical Centre. I think that Henk Elffers’ understanding of mathematical statistics has not gone very far beyond this understanding of the behavior of balls in boxes. Moreover, even his understanding of these basic facts had apparently eroded somewhat during all these years he was doing other things, since he made some elementary errors in his computation, as shown in Elffers corrected. For example, he assumed that the product of (so-called) p-values is again a p-value. The absurd consequence of this type of error is that a nurse who has worked in several different hospitals has a higher chance of being accused of murders than a nurse who only has worked in one hospital at the same time, although the number of “suspicious coincidences” might be exactly the same for the two nurses.

Furthermore, it is a mistake to think that these balls in boxes provide “objectivity”. There is only the illusion of science and objectivity, nothing more. The (in my view) most satisfactory computation in the Lucia de Berk case, which still has to be further expanded, is given in: Elementary Statistics on Trial. This indeed works with a continuous (that means, among other things: an infinite) family of models, and arrives at a probability of 1 in 25 that a nurse can be confronted with a series of coincidences as in the Lucia de Berk case (calculations, based on earlier available data even gave a probability of 1 in 9; since the hospital defines what an incident is, and since “natural deaths” were later declared to be “unnatural deaths” if Lucia de Berk happened to be on duty, the data on which one has to base these calculations cannot be trusted anyway). Compare this to the amateur calculations of Mr. Smits and Henk Elffers. Henk Elffers gets probabilities of the order of 1 out of 342 million! Mathematical statistics is a rather subtle discipline in which one has to be very careful! It is not for amateurs!

Still another amateur statistician, consulted by the court is (Prof. Mr.) Richard de Mulder, who studied Law and Business Administration. One would like to know why the court chose him as an expert on statistics. Was it because he confirmed the computations of the other amateurs, Smits and Elffers? Again, why didn’t the court consult a real professor of mathematical statistics?

Until recently I was under the impression that Prof. Mr. de Mulder was chosen as independent expert, who was asked to verify the computation of the “law psychologist” Henk Elffers. But I recently learned that the situation is even more grim and that the court did not even try to get an independent expert or a second opinion: Prof. Mr. de Mulder’s task was to explain Elffers’ computation to the court. Apparently, the court did not understand his computation. Joint publications of Henk Elffers and Richard de Mulder date at least back to 1999, as can be verified on the page Richard de Mulder. If I am rightly informed, the court asked Prof. Mr. de Mulder because he is a lawyer, and because the claim of the court was that an explanation of a real statistician would be incomprehensible. But I think that the latter claim has no basis, since they simply never tried to consult a real statistician.

In the final court verdict (also issued to the press) it is stated that statistics did not play a role in the final verdict. This is only true in the sense that the court did not consult even one professor of mathematical statistics in the Netherlands. On the other hand, Ton Derksen’s book makes it very clear that faulty and amateur “statistical” arguments are in fact the only thing the court has. All cases which first were brought forward as “hard evidence”, based on the digoxin intoxication, etc. have in the mean time dissolved into thin air. As an example, the anxiously waited for report from Strasbourg on the supposed digoxin intoxication of the liver of the child “Amber” did not confirm this intoxication at all (and has remained in a drawer for two years for still unexplained reasons). Medical expert witnesses who expressed the opinion that the suspicious deaths could very well be natural deaths have not seen their declarations on these matters make it into the final report of the court. So in the end a silly quasi-statistical computation on coincidences is the only thing that remains. For further news and discussion, see: The Lucia de Berk case, part 2. The petition for reopening the Lucia de Berk case can still be signed, see: Petition for reopening the Lucia de Berk case.