4

ArXiv does not compile many of the (derivative) auxiliary files for its files, like the .bbl and .ind files, and these files need to be uploaded along with the original source.

The main problem with this is that these files can be pretty sensitive about having the wrong version. Thus, to upload a document to arXiv, I have to somehow get a downgraded version of LaTeX.

Is there a known portable LaTeX distribution, or live USB system with a robust selection of LaTeX packages preinstalled, in the versions suitable for (current) arXiv (which would not require a lot of work to set up, like installing another OS or downloading packages by hand)? Or is there some other way to avoid all this nuisance? Something I can just download, run and be done with it?

I know there are hacks which allow you to upload the .pdf without the source files, but I would rather not do that (although if there is no hassle-free solution, I think I will do just that --- I'm honestly sick of it by now).

TobiBS
  • 5,240
tomasz
  • 825
  • it seems surprising that you need to downgrade the entire latex setup, perhaps there is a specific package that you need an old version for? Clearly most people submitting to arXiv use whatever version of latex they have locally to prepare the manuscript. – David Carlisle Jun 20 '18 at 13:11
  • 1
    Distantly related: https://tex.stackexchange.com/q/428277/35864. If you don't use biblatex your bibliography should be fine. If you don't use cutting edge new features most packages should be OK too – moewe Jun 20 '18 at 13:11
  • 1
    If you want nearly the same tex distribution than arxiv, you can install texlive2016, it is available from ftp://tug.org/historic/systems/texlive/2016/ Even with the same version, you'll still have to play by their rules about which bibliography formats they accept. – samcarter_is_at_topanswers.xyz Jun 20 '18 at 13:17
  • @moewe: It's not about any cutting edge new features, it's just that the resulting .bbl files are incompatible. – tomasz Jun 20 '18 at 14:08
  • 1
    Like I say, they are quite probably for biblatex (that's why I said not to use it), but it is very unlikely that .bbls not produced for biblatex are going to be incompatible. Most have been stable for years and even if not .bbls rarely contain code that would cause incompatibilities. – moewe Jun 20 '18 at 14:12
  • @moewe: Dropping biblatex just to upload files to arXiv seems like a more silly solution than just uploading a .pdf file... Bibtex does not support the fields I need, and typing it up by hand would require ridiculous amount of work. – tomasz Jun 20 '18 at 14:15
  • @samcarter: Download of the ISO fails for me. Anyway, is there any guide here (or elsewhere) on how to do set up a separate texlive environment to use side-by-side (the OS I'm using right now is Ubuntu, I guess this sort of thing might be slightly easier in Windows...)? – tomasz Jun 20 '18 at 14:25
  • To use multiples installations side by side, just install them normally and you can switch between them by setting your path export PATH=/usr/local/texlive/2016/bin/x86_64-darwinlegacy:$PATH or whatever year you want to use – samcarter_is_at_topanswers.xyz Jun 20 '18 at 14:28
  • @DavidCarlisle: I need a couple of things. For .bbl specifically, I need to downgrade biblatex and biber. I would probably need to downgrade makeindex at some point (didn't get the error messages about that yet). I remember doing that some time ago, but now I tried and failed --- maybe I'm making a stupid mistake, I only have superficial knowledge how pdflatex works, but my point is to find a solution which would be viable and relatively hassle-free for a non-expert like me. – tomasz Jun 20 '18 at 14:30
  • makeindex hasn't changed for 20 years so that's not likely to be a problem. as moewe guessed above biblatex/biber is almost the only example that's likely to be an issue so if that is causing a problem I would use bibtex note that same is true of most journal submission journal classes typically don't support biblatex so require bibtex – David Carlisle Jun 20 '18 at 14:39
  • @DavidCarlisle: Well, I did not have that problem, thankfully. I always just supply the .bib file and whatever .bbl file gets generated, and have not had any problems so far... But then again, I've only actually submitted sources for two journals so far, so maybe I'm just lucky. – tomasz Jun 20 '18 at 14:45
  • @samcarter: So I tried once more, installed the texlive version from the ISO, and... the resulting .bbl version is too old. – tomasz Jun 21 '18 at 11:19
  • While I think it is true that arXiv are running TeX live 2016, a closer look suggests they run an updated version of it placing it somewhere between TeX live 2016 and TeX live 2017. If this is only about biblatex, have a look at https://tex.stackexchange.com/q/415703/35864 and https://tex.stackexchange.com/q/358831/35864. The long period between biblatex 3.7 and 3.8 means that biblatex/Biber-wise TeX live 2017 is closer to what is running on the arXiv. – moewe Jun 21 '18 at 14:15
  • 5
    They should fix arXiv rather than expecting us to work around their broken build system that does not generate auxiliary data properly. It would be so easy. Just make a checkbox "run bibtex", "run biber". Or auto-detect: if the user did not upload a .bbl, but a .bib, then you probably need to generate the .bbl... – Erich Schubert Oct 12 '18 at 19:18
  • @ErichSchubert: well, I agree. I've heard some people say that the issue is that some people have enormous .bib-s with all papers in existence (or something of that magnitude). The correct solution would be (I think) just limiting the filesize to something reasonable... But I'm not an arXiv maintainer, so there may be bigger issues I'm not aware of (but I doubt it). The worse issue here is, I think, that the .bbl files are going to expire just as soon as they upgrade their texlive, at which point the papers won't compile properly again... – tomasz Oct 13 '18 at 22:44
  • @tomasz yes, a size limit would be easiest for this problem (if it really is that much of an issue - how many such papers are uploaded per minute?)... as for the versions: apparently they intend to keep a snapshot of each TeX live they used forever; and if they rebuild old papers, they're going to use the old TeX live versions. So that part would be solved then. – Erich Schubert Oct 13 '18 at 23:21

3 Answers3

7

I've found a sort-of easy solution. The requirements are several gigabytes of disk space and sufficient privileges with the machine to run docker (I did have some trouble with that on one machine). This is based on the instructions found here. Special thanks are owed to makisyu for maintaining the texlive 2016 image used in this guide.

The steps are as follows (as of October 14, 2018, they will need to be amended when arXiv's texlive is upgraded):

  • Install Docker (Ubuntu has it in the standard repositories; I'm guessing so do all the major distros).
  • Navigate into the directory with your tex file.
  • Install the texlive 2016 image using the command: docker pull makisyu/texlive-2016
  • Run pdflatex the required number of times via the command docker run --rm --volume $PWD:/workdir --workdir /workdir makisyu/texlive-2016 pdflatex my_texfile.tex (replacing my_texfile.tex with the appropriate filename).
  • Run biber using the command docker run --rm --volume $PWD:/workdir --workdir /workdir makisyu/texlive-2016 biber my_texfile (replacing my_texfile with the appropriate filename, again).
  • Run any other commands necessary to generate the required auxiliary files.

Christian Clason suggested in the comments that the last three steps can be replaced by docker run --rm --volume $PWD:/workdir --workdir /workdir makisyu/texlive-2016 latexmk my_texflile.tex. I did not test that (nor can I right now), but it should make the process slightly easier.

tomasz
  • 825
  • 1
    I was wondering whether such a docker image was available for this very purpose -- thanks! A comment: The last three points can probably be combined into a single docker ... latexmk my_texfile.tex. – Christian Clason Oct 14 '18 at 08:26
  • If you are running this with sudo and have issues with the file owner, you can add --user $(id -u):$(id -g) to the command line (for running pdflatex and biber). – JPW Apr 09 '20 at 10:16
3

It's also possible to just install the 2016 version of texlive in parallel to the current one -- it can be found at ftp://tug.org/historic/systems/texlive/2016, which includes the latest version of all packages, including biblatex and biber, before the freeze -- and just change the environment variable before compiling. For example, under Linux or macOS using bash, you can open a shell, change to the appropriate directory, and

  1. Delete all auxiliary files (especially *.bbl etc. for biblatex!)
  2. Set the path (assuming you used the standard install location):
    • Linux: export PATH=/usr/local/texlive/2016/bin/x86_64-linux:$PATH
    • macOS: export PATH=/usr/local/texlive/2016/bin/x86_64-darwin:$PATH
    • Windows: set path=c:\texlive\2016\bin\win32;%path%
  3. latexmk my_file.tex
  4. latexmk -c
  5. zip everything and upload to arXiv (they will ignore all files they don't use such as *.pdf or *.bib).

This works for me (I've put this in a script called arxivmk).

EDIT: I just saw that @samcarter already made this suggestion in the comments; maybe the details are still helpful.

Christian Clason
  • 1,315
  • For windows users the path to the W32TeX or TeX Live binaries can vary widely but wherever the path is e.g. the recommended locations are respectively c:\W32TeX\bin or c:\texlive\2016\bin\win32 (for in this case its the 2016 distro) Having determined the correct syntax for the above step 3 would change accordingly to set path=c:\w32tex\bin;%path% or set path=c:\texlive\2016\bin\win32;%path% in either case this only lasts for the session so best used in a command file where it can be set and re set at end of the run. To set permanently throughout the system use setX (only ONCE) instead. –  Oct 14 '18 at 10:33
  • @KJO Thanks! I'll include this in my answer. (Since the whole point of the exercise is to mirror the TeX setup that arXiv is using, you'd want to stick to texlive.) That the setting is only for the session is by design; I usually work with an up-to-date texlive and switch to 2016 only for submission (to check if it'll (likely) works and generate the correct .bbl for biblatex). – Christian Clason Oct 14 '18 at 10:44
  • Note I specially used 2016 yours suggest 2018 ?also would update step 1 break the reason for backdating ? –  Oct 14 '18 at 10:52
  • "As far as biblatex and Biber are concerned, you should be good with an un-updated TeX live 2017" from https://tex.stackexchange.com/questions/429436/making-the-arxiv-accept-a-bibtex-bbl-may-2018 –  Oct 14 '18 at 12:57
  • @KJO D'oh! Thanks for pointing that out. You are right about biblatex and biber, but this will also give you updated binaries (as well as some packages that got updated during the freeze). So if you're installing an old version to match arXiv, you might as well pick the closest one to make sure. (If we knew which specific texlive revision arXiv updated to, and tlmgr allowed to update to a specific revision, we could get even closer.) – Christian Clason Oct 14 '18 at 13:33
  • Also, tlmgr will not update across years, so this should get you the final state of 2016. In fact, let me test that quickly. – Christian Clason Oct 14 '18 at 13:35
  • It will not even let you update, but the net installer should install the last revision of 2016 (including biblatex from April 2017) anyway. – Christian Clason Oct 14 '18 at 13:48
  • 1
    I have successfully used your suggestion where I have TexLive 2019 and didn't install Tex 2017 for that. I just downloaded the binaries. – user2679290 Jan 22 '20 at 23:32
0

I have just found out that you can use Overleaf to compile the requisite auxiliary files by using the compiler version select feature and the submit feature.

This seems like the most convenient way to obtain the compatible files for most purposes.

tomasz
  • 825