57

In the empirical sciences, there are a number of journals that publish 'negative' results. Negative or null results occur when researchers are unable to confirm the findings obtained from earlier published reports. In the applied sciences, they may also come about when a scientist aims to to show that a particular technology (e.g., CRISPR) could alleviate a problem (e.g., a particular virus that kills that kills a specific type of plant), only to find out that it does quite the opposite (e.g. the technology led to the evolution of viruses that were more resistant to CRISPR).

In the formal sciences, including mathematics and logic, experiments like these aren't conducted*. However, it does happen that mathematicians develop machinery to tackle a particular thorny problem, only to find out it doesn't work. A good example is John R. Stallings' false proof of the Poincaré Conjecture.

Publications like these are few and far between. It seems to me that one of the reasons this is the case, is that there aren't any journals that are specifically geared to these types of papers. They are predominantly focussed on publishing articles that obtain 'positive' results, i.e. actually prove theorems or refute conjectures.

Yet it also appears to me that papers like these can be very useful to researchers in mathematics, for the following reasons:

A. They may inspire someone to slightly tweak the failed approach, in order to make it work and actually prove the theorem(s);

B. They may allow someone to see what has already been tried, and what types of avenues of research are probably not worth pursuing;

C. They may provide a platform for approaches to tackling difficult problems in mathematics, even if the methods don't work so far. Thus, they provide a place to share ideas, rather than throwing away months of work.

My question is twofold:

  1. Are there already any journals that are devoted to papers containing negative results in the above sense?
  2. Would it be worthwhile to set up such a journal, from your perspective?

(*) I am aware that experimental mathematics is a thing. The focal point of this question isn't really the experimental nature of the mathematics research, but it's about offering a venue to the failed approaches to solving problems developed through research - formal, experimental or otherwise.

Max Muller
  • 4,505
  • 4
    "In the formal sciences, including mathematics and logic, experiments like these aren't conducted." - this isn't strictly speaking true, experimental mathematics is a thing, and there are journals dedicated to it. I vaguely recall seeing some papers in them which were of more "negative" nature too. This doesn't address the main points of your question though so I will keep it to a comment – Wojowu Oct 23 '21 at 15:32
  • @Wojowu Fair enough, they are a different type of experiments I guess, but still experiments. That's why I mentioned 'experiments like these'. Maybe I should phrase it differently – Max Muller Oct 23 '21 at 15:34
  • When you ask "Are there already any journals that are devoted to papers containing negative results in the above sense?" you really mean, only to such negative results? You don't seem to address at all the question on how such papers are currently considered, and seem to implicitly suggest that "negative papers" currently have no chance of publication at all. But this is very questionable, I believe that there are a lot of works whose contents can be read in the angle of "this approach to proving X doesn't work", even in encapsulated in a positive result. – YCor Oct 23 '21 at 16:09
  • @YCor Yes, I indeed mean journals devoted only to such negative results. I don't think such negative papers have no chance of publication at all - Stallings' article is a good example of a paper containing negative results that was published in a general journal. I do suspect many more papers of this type would be published if there would exist journals especially dedicated to them. – Max Muller Oct 23 '21 at 16:15
  • @YCor I agree with you that there are works whose general message is "this approach to problem X doesn't work", but like to you said, they contain positive results as well. The journal I have in mind would be dedicated to papers containing methods to solving problems that indeed never or almost never work for the type of problem the authors had in mind when they started working on them - they would contain no or perhaps very few positive results – Max Muller Oct 23 '21 at 16:21
  • 3
    I think one issue with your implicit proposal, as basically suggested by YCor, is that this is simply not how math journals are organized. It's like asking if there is a journal dedicated to proofs by contradiction. A failed (but interesting!) attempt to e.g. prove the Poincaré conjecture might be published in a journal dedicated to geometry/topology; a failed attempt to prove the Riemann hypothesis might be published in a number theory or an analysis journal. – Sam Hopkins Oct 23 '21 at 16:26
  • 2
    To make your question CW, just flag it for moderator attention. \ By the way, Stallings's paper appears to be the progenitor of the "How not to prove" title format. – LSpice Oct 23 '21 at 16:26
  • 13
    @SamHopkins, you say "might be published"; but are they? Without saying that they don't exist, I can't think of any papers in my field that are intentionally of the shape "we tried this proof technique, and it doesn't work." There are definitely papers found in retrospect to have errors, or papers describing only partially successful solutions, but I can't think of any of the form "this might seem like a good proof technique, but actually it isn't." – LSpice Oct 23 '21 at 16:28
  • 4
    To me, this idea sounds like it has potential. More than anything, I imagine that the composition of its editorial board would determine whether such a journal would succeed and be valuable. With some respected, responsible, and discerning mathematicians from a variety of fields on the editorial board, authors would likely be intrigued by the experiment, and ready to send some interesting and worthwhile papers to such a journal. Then I think you have the beginning of a successful journal. –  Oct 23 '21 at 16:31
  • 2
    I think a good idea is to post such a "negative paper" on arXiv with a title such that it is found by a suitably precise google search. It does not count as a publication but it will then perhaps be useful. – Roland Bacher Oct 23 '21 at 16:33
  • 12
    @LSpice: I think it might just be a matter of how papers are phrased. For example, suppose there are two important objects $X$ and $Y$ in some subfield, and there is a question of whether perhaps $X=Y$. A paper might compute some invariant of both $X$ and $Y$ and observe they are the same. This could be (but is usually not) thought of as a failed attempt to prove that $X\neq Y$. – Sam Hopkins Oct 23 '21 at 16:34
  • I do not think that this would need a journal, some wiki type page (similar maybe to the one for polymath) would be enough... – Nick S Oct 23 '21 at 19:12
  • 2
    I was mentally formulating an idea similar to what @Sam Hopkins said as I was reading through the comments. I doubt there are enough incorrect attempts to prove something (of sufficient interest) which can't be turned into a correct publishable-worthy proof of something else to merit such a journal. For example, you're trying to prove something for all positive integers, and realize your proof doesn't work. Does it prove the result for all but finitely many positive integers? For infinitely many positive integers? For positive integers having a positive density? (continued) – Dave L Renfro Oct 23 '21 at 19:42
  • What about slightly changing the the various assumptions? The possibilities are seemingly endless, and probably worth looking at if the incorrect proof has some reasonably deep ideas. – Dave L Renfro Oct 23 '21 at 19:42
  • 2
    I say such a journal is not worthwhile. For B and C there is the arxiv, and I doubt if A actually happens. Are there any examples where one mathematician labored for a long time with one approach, and achieved no noteworthy positive results, but some other mathematician used their detailed work to achieve the original goal? –  Oct 24 '21 at 04:55
  • @MattF. I think A rarely happens because mathematicians generally are reluctant to publish failed proofs or negative results - which in my view has to do with the fact that there are no journals dedicated to such results and there is a general, silent consensus that true proofs or positive results are more useful. There are very few purely negative results published in journals that actually go deep and have a plausible approach, and if they'd be more prevalent, then there is a bigger chance that other mathematicians pick up on them and turn them into correct ones – Max Muller Oct 24 '21 at 09:35
  • 3
    It's not exactly what you want, but perhaps it is worth mentioning that there used to be something called Rejecta Mathematica. It was mentioned in one of the answers to this question: Does any publication venue make rejected papers available for download? (It was posted on [academia.se].) – Martin Sleziak Oct 24 '21 at 11:12
  • @MaxMueller, people get information in less formal ways than publications too — if you don’t have any examples of A based on a talk or a draft or a Nachlass then I remain unconvinced. (In Proof, a young guy has success with notes taken from a math prof’s home after he dies, but that is fiction.) –  Oct 24 '21 at 13:12
  • I believe there are some data bases on the internet intended to collect data from experiments whose results were not considered publishable because they didn't find anything. This is quite important as a remedy to the "file-drawer problem." Suppose mega-doses of vitamin XYZ are given to 15 patients who have Caesarsmith's disease and 10 of them immediately recover. Suppose it is known that only one in five such patients recover so fast without this drug. Then the probability of this observed result, under the hypothesis that the treatment has no effect, is$,\ldots\qquad$ – Michael Hardy Oct 25 '21 at 19:59
  • $\ldots,$about one in ten-thousand. So publish it!! But what about the $9999$ earlier studies that found no such effect? They are stashed in "file drawers" rather than published. Knowledge of their existence is useful in assessing the publishability of this latest experiment. – Michael Hardy Oct 25 '21 at 20:00
  • PS: $\quad\uparrow\quad$ One in about $9924$ if you mean exactly ten patients; one in about $8832$ if you mean ten or more, which would be the traditionally used thing for this purpose. With the shift away from frequentism, this might be done differently now. – Michael Hardy Oct 25 '21 at 20:10
  • @MichaelHardy, the case for medicine is clear, but is any similar argument convincing for math? –  Oct 25 '21 at 23:36
  • 1
    Here's another nice example of a paper in this category: "Yet another failed attempt to prove the Irrationality of $\zeta(5)$" by Werner Horn: http://www.csun.edu/~hcmth017/zeta.pdf – Max Muller Dec 02 '21 at 17:00
  • "Would it be worthwhile to set up such a journal, from your perspective?" Isn't this the ultimate example of a opinion-based question, the kind of question we regularly close here? – Gerry Myerson Aug 25 '22 at 23:09
  • P. Cartier, Comment l'hypothèse de Riemann ne fut pas prouvée, Seminar on Number Theory, Paris 1980-81 (Paris, 1980/1981), pp. 35–48, Progr. Math., 22, Birkhäuser Boston, Boston, MA, 1982, MR0693308 (85f:11035). – Gerry Myerson Aug 25 '22 at 23:17
  • @MaxMuller: I am not too sure, but does the journal Experimental Mathematics fit the bill? – Jose Arnaldo Bebita Dris Apr 30 '23 at 11:56

5 Answers5

62

I don't really know what an answer is for a question like "what about ...?" but I have some thoughts.

In fact, way back around 2006-2007 (according to the dates on the ArXiv, see Multiplying Modular Forms, if you want). What happened was I had written what I thought was a really nice paper explaining how to multiply modular forms whose associated representations (of a real Lie group) belonged to the discrete series. It clarified (to me) some ideas of others, and seemed to extend to all sorts of other kinds of modular forms like those on the exceptional group $G_2$.

Well... I was all happy about this and about to speak at a conference. But the day before, Gordan Savin told me about a mistake in the paper. I spent a long evening kicking around the paper and then kicking myself about it. It was a really subtle thing to me -- a difference between $K$-fixed vectors and $K \times K$-fixed vectors -- but a "well-known" issue to experts, ultimately involving the failure of discrete decomposability.

Anyways, the next day at the conference (AMS Special Session, Jan 8, 2007), I didn't really know what to say, but I suggested that someone start the "Journal of Doomed Proofs". And I wasn't kidding. It would be refereed and everything. Criteria for acceptance would be the following:

  1. The paper contains an plausible approach to a problem of interest to the mathematical community.

  2. The approach is sufficiently motivated that many other people might try it.

  3. The approach is doomed, though not obviously from the beginning.

  4. The paper explains why the approach is doomed, identifying the obstacles which really stop things from working. Or at least have to be worked around in the future.

I still think this is a good idea, and not just for the usual "science should publish negative results" reason. Mathematicians have a sort of secret oral tradition of "well-known" things (doomed ideas, silly apocrypha, the largest rank of an elliptic curve over Q, etc.). But those of us who teach in redwood forests don't really have access to this tradition any more. And some were never granted access in the first place. A journal might go a little way to correct this.

If anyone knows how to pitch a new journal, count me in. If it's JDP (Journal of Doomed Proofs) or JNR (Journal of Negative Results) or whatever, it's fine with me. But not with Elsevier please.

David Roberts
  • 33,851
Marty
  • 13,104
  • 2
    Great story! Nice to see you've been thinking along the same lines. I also completely agree with the criteria for acceptance you've laid out – Max Muller Oct 24 '21 at 09:40
  • 10
    @Marty Did you try publishing your work anyway? Maybe what is needed isn't for the community to set up a journal, but for authors like you to recognize that their work is valuable and make the effort to write it up in a way that makes that value clear. – Timothy Chow Oct 24 '21 at 12:58
  • 10
    @TimothyChow Yes, in fact the article is published. I recall it getting rejected from a journal at first, because the results were more negative than positive. But then it ended up published by Cambridge Univ. Press in the volume "Modular Forms on Schiermonnikoog", in 2008. – Marty Oct 24 '21 at 21:24
27

There have been papers like the following in the American Mathematical Monthly

Guy, Richard K., "Unsolved Problems: Don't Try to Solve These Problems". Amer. Math. Monthly 90 (1983), no. 1, 35–38+39–41.

The suggestion being that graduate students and beginning mathematicians would likely waste a lot of time and not get anywhere if they attack these problems.


The original version of this post confused the above Monthly department with another one called "Research Problems", which ran from 1969 to about 1980. The first was on what P.J.Kelly called the "Spoke Problem".

Klee, Victor; Research Problems: "Can a Plane Convex Body have Two Equichordal Points?" Amer. Math. Monthly 76 (1969), no. 1, 54–55.

Gerald Edgar
  • 40,238
  • 9
    This is a blog post and not a proper article, but Terry Tao's "Why global regularity for Navier-Stokes is hard" (https://terrytao.wordpress.com/2007/03/18/why-global-regularity-for-navier-stokes-is-hard/) has sort of the same flavor of explaining why standard techniques have not worked for a notorious open problem. – Sam Hopkins Oct 24 '21 at 12:37
  • @SamHopkins Is this the same work that none mentioned? – Timothy Chow Oct 24 '21 at 13:00
  • @TimothyChow: Not exactly. none referenced a novel approach to Navier-Stokes that Tao proposed (see, e.g., https://terrytao.wordpress.com/2014/02/04/finite-time-blowup-for-an-averaged-three-dimensional-navier-stokes-equation/ for a blog discussion of that). What I linked to was Tao's discussion of why standard approaches are unlikely to succeed. – Sam Hopkins Oct 24 '21 at 13:09
  • 1
    @GeraldEdgar Do you remember the statement of the "Spoke Problem"? If you can provide a description, maybe someone can help locate the article. – Timothy Chow Oct 25 '21 at 08:00
  • Spoke problem is here: Klee, Victor; Research Problems: Can a Plane Convex Body have Two Equichordal Points? Amer. Math. Monthly 76 (1969), no. 1, 54–55. – Gerald Edgar Oct 25 '21 at 16:21
  • 2
    @SamHopkins, seven years later Tao went from “why global regularity for Napier-Stokes is hard” to “finite-time blowup for an averaged three-dimensional Navier-Stokes equation”, and that became a published article in the Journal of the AMS. –  Oct 26 '21 at 00:36
  • @TimothyChow the blowup paper stems from an attempt to show why energy conservation arguments (tried by some people) were likely to fail: I think this comment of Tao's refers to it. Only afterwards did he start to believe similar methods could show blowup in the real NS equations. – none Oct 29 '21 at 21:18
20

(Too long for a comment, but probably more opinionated and mildly off topic than a good answer. My apologies.)

It seems like there are two distinct issues at hand here suggesting why one should want such a journal. The first is where barriers or obstructions are subtle but known folklore in a specific community. The second is when a single person has tried a specific tactic and it doesn't work, and they can identify that it doesn't work for reasons $X$, $Y$, and $Z$. There may sometimes be overlap between these two issues but the first is part of a more general problem, of folklore in general, so I am almost inclined to think that there should be a journal of folklore results, but maybe the Arxiv is better suited for that.

At least or some specific problems, it seems like obstructions are themselves studied or are included in papers addressing related issues. For example, consider the very old open problem of whether there are any odd perfect numbers. Recently, Pace Nielsen and a collection of other researchers published a paper on a generalized version of "spoof odd perfect numbers" which are numbers which would be an odd perfect number if one pretends that a specific factor is prime. The classic example here is due to Descartes who looked at $$N=3^27^211^213^2 22021$$ where at a glance it seems to be an odd perfect number because if we ignore that $22021=19^2 61$, we would have $\sigma(N) = (3^2+3+1)(7^2+7+1)(11^2+11+1)(13^2+13+1)(22021+1) = 2N$. Here, $\sigma(N)$ should be the sum of the divisors of $N$, but we are applying the formula for it as if 22021 were prime. John Voight generalized this idea, allowing one to have negative prime factors in a spoof, and this was further generalized by Nielsen's group who allowed repeated prime factors where one doesn't recognize that the they are the same prime. Now, the interesting bit here is that many of the results proven about odd perfect numbers in the literature have proofs which essentially go through with only small modification to prove analogous statements about this broader class of numbers. This means that those results cannot by themselves hope to prove there are no odd perfect numbers. This is also useful if one is asked to referee papers claiming to prove that there are no odd perfect numbers or claiming very strong results about them. One can go through the entire argument using Descartes number or one of the other examples, and often the error will pop up that way.

In a different direction but also on the same problem, I recently introduced another obstruction in this paper. There, the primary idea was the following: Euler proved that any odd perfect number $N$ must satisfy that $N = q^e m^2$ where $q$ is a prime and $q \equiv e \equiv 1$ (mod 4), and $(q,m)=1$. (This is by modern standards pretty trivial. It isn't even really a statement about an odd perfect number, but a statement about any odd number $n$ where $\sigma(n) \equiv 2$ (mod 4).) It turns out that many of the statements about odd perfect numbers that have been proven are weak in the following sense. Given a set of positive integers $S$, we'll write $S(x)$ to be the number of elements in $S$ which are at most $x$. Write $E(x)$ to be the set of numbers satisfying Euler's restriction. Given a property $p$, we'll write $E_p$ to be the set of elements in $E$ which satisfy property $p$. Then most properties $p$ in the literature which have been proven about odd perfect numbers satisfy $$\lim_{x \rightarrow \infty} \frac{E_p(x)}{E(x)}=1.$$ That is, the property applies to almost all elements in the set $E$. Now, no finite collection of such properties can prove that no odd perfect numbers exist. Again, this is also pretty useful for trying to find holes in claimed proofs of the conjecture if one wants to do that sort of thing.

As far as I can tell, while there are some results which manage to partially evade the spoof obstruction, and some which manage to evade the density issue, there's nothing in the literature which seems to fully evade both.

But all of these are things which have managed to go in existing papers. Are there similar ideas which are not getting put in the literature to a large extent? I'm less certain of that.

JoshuaZ
  • 6,090
20

Failed attempts can be documented in technical reports which one can post online if so desired. A technical report is a way to communicate to your superiors, or a funding agency that you put in an effort, which unfortunately led nowhere. Also a technical report could be a note to yourself, a record of what you thought about at a given time.

Of course, failed attempts are common, as most of the things we try as mathematicians fail. I personally find it hard enough to keep up with ever-growing record of successful attempts. In my opinion there are very few people whose failed attempts are worth reading, and hence in most cases publishing failed attempts is a waste of resources. Posting them on one's homepage is certainly okay.

17

There was a workshop on "Barriers" in computational complexity a while back. A barrier result is one showing that a certain approach to solving a given problem (such as P vs NP) can't possibly work. I'm having trouble finding info about the workshop but here is an overview of the topic:

http://pi.math.cornell.edu/~takhmejanov/BarriersInComplexity.pdf

It would be interesting to have a collection of such articles from various areas of mathematics. Terence Tao's use of self-replicating fluid "automata" to show you can't prove Navier-Stokes regularity using energy conservation is another example. A whole journal might be hard to fill though.

none
  • 1,117
  • 2
    You're right, it's surprisingly hard to find info about the workshop. The Wayback Machine has archived some material from the first Barriers workshop in 2009, but for the second workshop in 2010, all I've been able to find is a list of talks here with a bunch of dead links. – Timothy Chow Oct 24 '21 at 20:01