Bootstrap analysis - why it is called bootstrap?

Question

Bootstrap analysis, bootstrapping etc are quite common jargons of bioinformatics and phylogenetics.

However, it is not very much clear to us, what exactly being meant by "a boot's straps".

Does it means a "comparison"? (Such as we hold the two straps of a boot (shoe) in "two hands"; may be an analogy to comparing 2 things (say left-'hand' side and right-'hand' side of an algebraic proof-task)) or it is something about sequence alignment (since we 'tie' a knot in boot (shoe) so that the 2 ends of the strap remain attached to each-other).

Some of my classmates even assumed it may be a process during computer booting (but I could not agree with that).

I've searched internet, but could not found anything very helpful. This wikipedia page (permalink) about bootstrapping, tells,

^{Tall boots may have a tab, loop or handle at the top known as a bootstrap, allowing one to use fingers or a boot hook tool to help pulling the boots on. The saying "to pull oneself up by one's bootstraps" was already in use during the 19th century as an example of an impossible task. The idiom dates at least to 1834, when it appeared in the Workingman's Advocate: "It is conjectured that Mr. Murphee will now be enabled to hand himself over the Cumberland river or a barn yard fence by the straps of his boots." In 1860 it appeared in a comment on philosophy of mind: "The attempt of the mind to analyze itself an effort analogous to one who would lift himself by his own bootstraps". Bootstrap as a metaphor, meaning to better oneself by one's own unaided efforts, was in use in 1922. This metaphor spawned additional metaphors for a series of self-sustaining processes that proceed without external help.}

^{The term is sometimes attributed to a story in Rudolf Erich Raspe's The Surprising Adventures of Baron Munchausen, but in that story Baron Munchausen pulls himself (and his horse) out of a swamp by his hair (specifically, his pigtail), not by his bootstraps – and no explicit reference to bootstraps has been found elsewhere in the various versions of the Munchausen tales.}

(In the quotation, highlighted portion seemed to tried to tell a meaning), but it wasn't also very much helpful. Could not correlate with bioinformatics. (Btw the same wikipedia article mentions computer booting).

Wikipedia article about Bootstrapping (statistics) (permalink) tells:

^{The bootstrap was published by Bradley Efron in "Bootstrap methods: another look at the jackknife" (1979), inspired by earlier work on the jackknife.Improved estimates of the variance were developed later. A Bayesian extension was developed in 1981. The bias-corrected and accelerated (BCa) bootstrap was developed by Efron in 1987, and the ABC procedure in 1992.}

Still that is not very helpful.

However, the textbook, Introduction to Bioinformatics/ Arthur M. Lesk/ 3rd Edition/ Oxford / (Low Price edition) ; in its chapter 5 (Alignments and phylogenetic trees), section The problem of varying rates of evolution -> computational consideration (page 296); a clearer definition has been given:

3. Formal statistical tests, involving rerunning the calculation on subsets of the original data, are known as jackknifing and bootstrapping.
   • jackknifing is calculation with data sets sampled randomly from the original data. For phylogeny calculations from from multiple sequence alignments, select different subsets of the position in the alignment, and rerun the calculation. Finding that each subset gives the same phylogenetic tree lends it credibility. If each subset gives a different tree, none of them are trustworthy.
   • Bootstrapping is similar to jackknifing except that the position chosen at random may include multiple copies of the same position, to form data sets of the same size as original, to preserve statistical properties of data sampling.

More clear definition of "Boot-strapping"; but it does not explain, where the relation exists with the boot (shoe) and its straps.

Answer 1

The method itself has nothing really to do with boots or straps, just as the jack-knife method also has nothing to do with knives. In the case of bootstrapping, the goal is to determine the accuracy of an estimate from random subsets of a population. Normally estimating something like the variance of the mean requires multiple independent samples, but bootstrapping allows you to perform estimates from a single population. This is essentially allowing the estimate to "pull itself up by its bootstraps", which is the only reasonable source for the term. See also the more general wikipedia article on bootstrapping outside of statistics.

Answer 2

In general, branch support values indicate how much of the total data support that branch in a proposed tree. But what do you do if you used all of the available data to infer the tree in the first place?

With Bootstrap supports, you generate new alignments (almost) ex nihilo. You calculate a trees support values, using its own data.

This is the connection to "pulling yourself up by your bootstraps": you don't "pull" on additionaly collected data (a rope or something in the metaphor), but rather on the very thing that your observation is based on (your boots).

Answer 3

Origin of name Bootstrapping is a term of derision in many ways, prior widespread use in phylogenetics. The response here is also the story that I recall the name meaning 'pulling yourself up by your bootstraps'. If I remember the story of it origin correctly, it means its impossible to 'pull-yourself-up' with you bootstraps because there's nothing external to leverage.

Historic usage Thus traditionally, it was a last resort approach in statistics, or else a crude data mining approach, because a lot statistics at that time worked by comparing a result against a theoretical reference, i.e. critical probability. So for example, T-test, Chi-squared, ANOVA etc ... a theoretical null distribution is used or else created against which the observation is compared. The idea of not using any theoretical external distribution was not deemed cool in the days of critical probability. Bootstrapping does weird things to the distribution that few really understand (spent time investigating).

Method Technically bootstrapping is simply resampling with replacement. So if you've a data set of 100 nucleotides replicates of a resampled data set are constructed each of 100 nucleotides by randomly selecting 100 nucleotides (with replacement) from the original data set. Performing tree estimate calculations over multiple replicates permits a robustness estimate of each 'node' in the tree (hypothetical common ancestor).

History in phylogenetics With onset of phylogenetics however bootstrapping became a statistical norm for several different reasons, which are quite complex. For example, attempting to perform critical probability across ever branch of a tree could run into statistical problems, what is a 'null distribution' in evolutionary biology (that question has since been reasonably well answered), in the days of the rows about the parsimony method - bootstrapping was well under radar. Hence bootstrapping is now mainstream, with some caveats.

I assume bootstrapping was originally supported by simulation studies, however it so established I don't even know it theoretical history.

Jack-knifing 'Jack-knifing' is named because it was deemed a rough and ready tool (a 'pen-knife') to assess a data set and a forerunner of the bootstrap. Its so old now I don't even remember the resampling strategy it uses, chopping the data set in half rings a bell - I honestly forget.

Today These days there is no real reason to know the method, other than using it and the thresholds commonly accepted. Very little statistical development is based around bootstrapping. The underlying tenant is that each site is independent of the others and that is a widespread assumption across phylogenetics. That assumptions has been assessed (obviously) but the solutions were not implemented. Continually reassessing this assumption remains very interesting and could be productive longterm, but it doesn't negate the basic approach to treat each alignment position as an independent event to every other alignment position because its a baseline used throughout molecular phylogenetics.