Bayes' Theorem follow up question

Question

I posted this question the other day, and I was wondering if there is any literature or guiding principles on using Bayesian Analysis to determine the reliability of forensic evidence or other ways that Bayes' theorem is used by practicing forensic scientists other than analysing DNA samples.

For example, I am wondering something along the lines of if is possible to use Bayes' Theorem to determine the reliability of evidence X (in other words, can we determine if evidence X pristine vs. if it contaminated or somehow planted/doctored, and how would this affect the hit rate/false alarm rate of that particular piece of evidence).

Answer 1

is possible to use Bayes' Theorem to determine the reliability of evidence X (in other words, can we determine if evidence X pristine vs. if it contaminated or somehow planted/doctored, and how would this affect the hit rate/false alarm rate of that particular piece of evidence).

You wouldn't be able to introduce that evidence at trial in the form of a Bayes Theorem analysis in U.S. Courts. It also wouldn't really be relevant in a Daubert hearing which goes to the reliability of the field and methodologies relied upon in general, not to the way that a particular expert chooses to apply those methodologies to the particular facts of the case before the court.

You could argue that the evidence is more or less reliable as a result and explain why this is the case in non-technical terms to a trier of fact (i.e. to a jury in a jury trial, or a judge in a bench trial). One could also introduce discussion of these underlying factual issues or a frequentist analysis to impeach or rebut the testimony of an expert witness giving testimony on a frequentist basis about its accuracy.

The weight given to the admitted evidence is in the exclusive province of the trier of fact.

A Bayes Theorem analysis would be inadmissible at trial because it would intrude on the authority of the trier of facts to be the sole person or persons weighing the evidence and determining its credibility. This is because Bayes Theorem is essentially a specific methodology for determining how much weight to give evidence based upon prior experience outside the circumstances upon which admissible evidence has been presented at trial that is used to generate a Bayesian prior.

But Bayes Theorem is not really within its domain of applicability in the context of something as global as a guilt/innocence determination in a criminal case which is too complex for that to be applied with the kind of scientific certainty that posing the question in that manner suggests. So, this rule is basically a wise one.

When the law wants to apply something comparable to a Bayesian prior into a legal analysis, the usual form that this takes in legal doctrine is some sort of rebuttable legal presumption (e.g. the docrrine of res ipsa loquitor in tort law). These presumptions may either be established by statute, or arise at common law through a line of case law decisions, a method quite different from the way that Bayesian priors are developed in probability and statistics.

Answer 2

That's not how expert testimony works

If you are talking about Bayes' Theorem or any other statistical technique or, for that matter, any other area of specialist knowledge, you are in the realm of expert testimony.

An expert witness has a special relationship with the court. First, they are the only type of witness allowed to state opinions, lay witnesses can only state what they saw, heard, felt etc. - they cannot draw conclusions from this. Second, they are officers of the court and are independent "advisers" to the court, not advocates working for the prosecution or the defence. This includes your forensic scientist who is often an employee of the state.

For example, the role of an expert witness in a civil trial in new-south-wales is subject to the code of conduct in Schedule 7 of the Uniform Civil procedure Rules. There are equivalent requirements in the criminal code.

One of the things they must do is state "the reasons for and any literature or other materials utilised in support of each such opinion". It's perfectly acceptable for an expert to use Bayes' Theorem in arriving at their conclusion if that is appropriate and that must be disclosed as one of "the reasons".

The court may also direct an expert to "(a) confer with any other expert witness, and (b) provide the court with a joint report specifying (as the case requires) matters agreed and matters not agreed and the reasons for the experts not agreeing". This allows the opportunity for "the reasons" to the questioned and, if necessary, altered. So, for example, if an expert has misapplied Bayes' Theorem or any other scientific principle, then that can be challenged and resolved. Or, if the experts continue to disagree, the disagreement can be placed in evidence for the trier of fact to sort out.

Court cases are not decided probabilistically

The role of a court in common law jurisdictions is to decide the dispute (guilty/innocent in a criminal case, plaintiff/defendant in a civil case) based on the evidence presented by the parties, the law as determined by the trier of law (the judge) and the facts decided by the trier of fact (usually a jury in criminal matters, the judge in civil matters). The facts that the trier of fact decides may bear no resemblance to what actually happened because they weren't there. They have to decide how much to believe each witness (and whether to believe some parts of their testimony more than others) and resolve the conflicts between witnesses and then decide if the party bearing the burden of proof has met that burden.

In a criminal case, the burden is beyond reasonable dount and exactly what that is is up to each individual trier of fact. In a civil case, it is on the balance of probabilities which, despite its name, can't be reduced to a number - it just means more likely true than not true; again, a standard determined by the trier of fact.