Predictive Coding in the Court Room

By | January 8, 2020

The previous blog post provided a basic insight into how predictive coding, an offshoot of Artificial Intelligence (AI)  technology is now being applied in e-discovery document review in civil litigation. This edition will examine judicial approach in various jurisdictions toward mandating the use of predictive coding in e-discovery.

United States

The case of Da Silva Moore, et al v. Publicis Group 2012 U.S. Dist. LEXIS 58742 was probably the first case in the US where the court was called upon to mandate the use of predictive coding in e-discovery. For proponents of predictive coding, perhaps the fact that the case came before Magistrate Judge Andrew Peck, a Magistrate well-versed in predictive coding technology, was like an icing in the cake. Magistrate Peek had previously published an article advocating the use of predictive coding in document review. In reaching a decision in Da Silva Moore, he reviewed predictive coding vis-à-vis linear review and keyword search. With respect to linear review, Magistrate Peck noted that even though it is considered by lawyers to be the “gold standard”, it would be too expensive in the present case which involved more than three million documents. Keyword search was also considered inadequate due to tendency towards over-inclusiveness.

The Magistrate ruled in favour of the use of predictive coding basing his decision on five factors: (1) the agreement of the parties, (2) the amount of documents involved, (3) the finding that predictive coding was better than other alternatives, (4) cost effectiveness and proportionality, and (5) defendants’ transparency in the discovery process. The Magistrate’s statement that predictive coding was agreed upon by the parties resulted in even more controversy than resolution of the problem. The plaintiffs asserted that they never consented to any agreement to use predictive coding, that they were “steamrolled” by the Magistrate. On appeal, District Judge Andrew Carter, Jr. affirmed the decision of Magistrate Peck stating that the confusion relating to the plaintiffs’ consent was immaterial. What is material is the fact that there is no evidence to conclude that the use of predictive coding will deny the plaintiffs access to discovery as required under the rules.

Shortly after the decision in Da Silva Moore, the use of predictive coding was sanctioned by the Virginia State Court in the Global Aerospace Inc. v. Landow Aviation No. 61040 Loudoun County, Va. Cir. Ct. Apr. 23, 2012. The size of the document in Global Aerospace was about 200 million. In Re: Biomet M2a Magnum Hip Implant Products Liability Litigation No. 3:12-MD-2391 (N.D. Ind. Aug. 21, 2013), dispute arose when one of the parties sought access to the ‘seed set’ used by the producing party in training the predictive coding algorithm. Unlike the other cases where the dispute relates to whether predictive coding should be used or not, this case dealt with a novel issue – the right of access to information relating to ‘seed set’ and discovery techniques used by the producing party. Refusing to order the disclosure, Judge Miller Jr. noted that the “request reaches well beyond the scope of any permissible discovery by seeking irrelevant or privileged documents used to tell the algorithm what not to find.” Thus there is now a growing tendency (and rightly) to treat such information as litigation privileged. Curiously, in some cases the courts have tended to treat a responding party’s refusal to disclose its seed set to the opposing party as an indication of lack of transparency and failure to fully disclose. In determining transparency and full disclosure, it is important that the court take into consideration valid objection relating to privilege raised by a party.

United Kingdom

The first case in which an English court was asked to make an order mandating the use of predictive coding in e-discovery was Pyrrho Investments Limited v. MWB Property Limited [2016] EWHC 256 (Ch). The original number of documents involved in the discovery exercise was in the range of 17.6 million. Through a process of de-duplication, the number was ultimately reduced to some 3.1 million documents, and even then the latter number was considered large and costly to search. Faced with a novel case, the presiding Magistrate Master Mathews’ task was somewhat eased by the fact that both parties had consented to the use of predicting coding. All they needed was the ‘blessing’ of the court to proceed.

In sanctioning the use of predictive coding in this case, Master Mathews set out ten factors justifying his order. First, the experience in other jurisdiction such as the United States and Ireland where the courts have variously endorsed the use of predictive coding even in contentious cases. The fact that the parties in this case has agreed to the use of predictive coding further justified the decision. Master Mathews also noted that there is no provision in the Civil Procedure Rules or Practice Directions which expressly or impliedly prohibits the use of predictive coding. Other factors cited by the learned Master in justification of his order includes the volume of document to be reviewed (over 3 million), the cost of manually searching the document (which was considered unreasonable) compared to predictive coding which was held to be far less expensive and hence a viable alternative.

Shortly after the decision in Pyrrho Investments Limited, the case of Brown v. BCA Trading Limited [2016] EWHC 1464 (Ch), another predictive coding case, came before the English court. Unlike the Pyrrho case which was non-contentious, the parties in Brown v. BCA Trading could not agree on the use of predictive coding. The respondents at the case management hearing presented evidence to show why they think that predictive coding was “the most reasonable and proportionate method of disclosure.” This evidence includes favourable cost implication in favour of predictive coding. Surprisingly, this evidence was not contradicted by the opposing party. Mr. Registrar Jones order to use predictive coding in this case was based on two main factors – the fact that predictive coding would at least be able to identify documents which would otherwise be identified using other search methods such as keyword search. Secondly, the court accepted the cost saving argument that predictive coding would be considerably cheaper than keyword search.


Irish Bank Resolution Corporation Ltd & Ors. v. Quinn & Ors. [2015] IEHC 175 was the first case in which a party sought an order before the Irish court to use of predictive coding in document discovery. At the onset of the discovery process, the defendants partially consented to the use of predictive coding. Subsequently, they reversed their position asserting that predictive coding does not comply with the rules of the courts. The court thus had to determine for the first time whether the use of predictive coding in discovery complies with the Rules of the Superior Court, and the appropriate methodology to be used in this case.

What is noteworthy about this case is the fact that the plaintiffs came to the court with substantive expert and empirical evidence to persuade the court in granting their request for use of predictive coding. The plaintiffs estimated that with a team of 10 experienced reviewers using a traditional linear review, it would take about 9 months and €2 million in cost (in addition to supervision and technology costs) to complete the review. The plaintiffs’ expert suggested that using predictive coding would require “a fraction of the cost” and could be completed within a shorter time frame.

The plaintiffs presented their proposed ten stage protocol before the court. The protocol addressed issues relating to transparency, keyword search, disclosure procedure for documents to be used in the training set (seed set), and the process for conducting a linear review of relevance and privilege in respect of applicable set of documents. The protocol also outlines quality control measures, as well as disclosure procedure on the completion of the review. One of the ground for the defendants’ objection is that predictive coding would not identify all relevant document and hence incompatible with the obligation of the disclosing party. This objection seems to be out of tune with reality as no discovery methodology is capable of capturing all relevant documents. Thus the court rightly noted that all review methods would inevitably result in omission of some relevant documents, and compared to manual review, predictive coding results in fewer omission of relevant documents. Having examined the plaintiffs’ proposed protocol and their conducts during the course of the discovery process, Fullam J. was satisfied that they acted bona fide and their proposal was transparent. He then endorsed the use of predictive coding in the discovery process.


Canada has yet to witnesses any legal development resulting in a clear judicial acceptance of predictive coding as evident in US, UK and Ireland. In fact, a search of Canadian Legal Information Institute (CanLII) online case law database using the search term “predictive coding” returned only two responsive relevant cases – L’Abbé v. Allen-Vanguard, 2011 ONSC 7575 (at para. 23) and Bennett v Bennett 2016 ONSC 503 (CanLII). The most relevant of the two cases is the Ontario Superior Court decision in Bennett v Bennett Environmental Inc. Not surprisingly, the facts giving rise to the Ontario case originated from United States. In that case, the plaintiff who was a former officer and director of the defendant corporation was extradited to the United States to face criminal prosecution by the United States Department of Justice for white-collar crime.

Following his arraignment, the Department delivered a “document dump” to his US attorney. To review the documents, the US attorney used predictive coding for the first level of review to reduce the number of documents which was later subject to human review. Back in Canada, the plaintiff obtained an order under s.124(1) of the Canadian Business Corporations Act (CBCA) and defendant corporation’s by-laws for advance of fund from the corporation for his legal fees in relation to the criminal charges in the US. The defendant corporation sought a determination of the “reasonableness” of certain legal fees submitted by the plaintiff including the fees incurred by the plaintiff’s US attorney in document review. In his ruling, Mesbur J. accepted not just the reasonableness of the document review fees, but also the use of predictive coding for the review. According the Judge:

Given the use of predictive coding for the first level review of massive document disclosure, I do not find it unreasonable for the lawyer to then use paralegals to conduct the next level or levels of review. I make no adjustment on this account.

While falling short of a judicial endorsement of the use of predictive coding in document review, the decision in Bennett v. Bennett Environmental Inc. may well mark a starting point in Canada towards a move to the trend in other jurisdictions.

Leave a Reply

Your email address will not be published. Required fields are marked *