89

This question led to a new package:
pkgloader(CTAN, Github)

LaTeX package conflicts are a common source of frustration: (1) (2) (3) (4) (5) (6). To quote Freek Dijkstra:

Package conflicts in LaTeX are a hell.

I've been looking around, but I don't think anyone has worked on what would seem, to me, the obvious solution: a package which automates conflict resolution for LateX packages.

As it happens, my PhD research (Delta Modeling) matches this problem closely, so I've made a quick start on that package and I would like some feedback on the idea.

The current version is temporarily stored in an SVN branch for another package. It depends on the l3graph package I'm working on, which is not ready for CTAN yet. The l3graph package offers a graph datastructure for expl3 with facilities for detecting cycles and working with the transitive closure/reduction and topological order of graphs.

Package Description

The pkgloader package offers tools for resolving conflicts, and is backed by a database of known resolutions. For the average user, everything would just work. The user interface looks like this:

\documentclass{article}

\usepackage{pkgloader}

% List packages in any order you please
\usepackage{xltxtra}
\usepackage[xetex]{graphicx}

\LoadPackagesNow

\begin{document}
\end{document}

By redefining \usepackage (and friends), pkgloader gathers the list of packages (with options) that the user wants to load, then loads them in the right order at the \LoadPackagesNow command. The package order above (needs XeTeX) would immediately cause an error if not for pkgloader.

It is able to prevent the problem because I've already included a modest list of package-ordering rules:

\Load {fancyhdr} before {hyperref}
\Load {fncychap} before {hyperref}
\Load {float}    before {hyperref}
\Load {hyperref} before {algorithm}
\Load {amssymb}  before {xunicode}
\Load {amssymb}  before {xltxtra}
\Load {graphicx} before {xltxtra}
\Load {graphicx} before {fontspec}
\Load {caption}  before {subfig}

These can also be added manually.

Potential Features

Fixing the package loading order is only the first (and simplest) step. With a bit of extra work we could add features like the following:

  • A package ordering rule like: \Load {cleveref} late (or early)
  • Facilities for package authors to include their own resolutions in a uniform way
  • Automatic detection of cycles within the rules
  • Accumulating and combining package options, with specific rules per package
  • Adding extra 'glue code' to make two other packages behave (if they're both being loaded)
  • Automatically resolving pretty much every conflict listed on Freek's page.
  • For conflicts that cannot (yet) be resolved, at least show an error message rather than let things go wrong at a later stage.

Question: What prior work is there?

Specifically:

  • Is there already another package that does (something like) this?
  • Are there any specific pitfalls I should be aware of?
  • Other than Freek's page, is there any kind of comprehensive list of known conflicts and resolutions?

I think a package like this has the potential to be very useful to a lot of people, if I do this right. A package like this will have a very wide scope. I don't want to step on anyone's toes, so please help me do this right.

I'd be happy to collaborate with the community on this - any feedback is welcome, so contact me if you're interested.

mhelvens
  • 6,126
  • 5
    +1 (the system doesn't let me upvote more often) WOW! That would be amazing if you made it work! – Count Zero Jul 08 '13 at 22:09
  • 1
    @CountZero: Thanks! The loading-order stuff already works, actually (download the code and try). It just needs to be cleaned up (probably debugged) and documented. – mhelvens Jul 08 '13 at 23:00
  • 5
    KOMA-Script's scrlfile package does related stuff – cgnieder Jul 08 '13 at 23:47
  • @cgnieder Indeed, thanks! As it happens, scrlfile cannot replace pkgloader, but it's most definitely related, so it's important that we describe the differences and similarities in the documentation. – mhelvens Jul 08 '13 at 23:57
  • Packages load other packages. Thus you will eventually end up with a huge database, filled with package dependencies. – Heiko Oberdiek Jul 09 '13 at 00:31
  • 11
    The order of packages and code, specified by the user matters. Compare \usepackage{amsmath,wasysym} with \usepackage{wasysym}\let\iint\relax\iiint\relax\usepackage{amsmath}. The symbols \iint and \iiint are different. Also without the \let assignments amsmath will complain. (AFAIK there is also a package that with situations that packages define the same macro). – Heiko Oberdiek Jul 09 '13 at 00:37
  • 7
    Package load order is only part of the problem. Many commands, offered by packages, should only be used after another package has successfully patched it, e.g. \newfloat which should be used after loading float + hyperref first. Furthermore LaTeX itself is IMHO the main problem, it's not very modularized. One example: One need to patch \chapter to adjust the spacing between figures and tables in LoF/LoT, the so-called "chapter's gap". So writing such packages is only scratching on the surface, and IMHO not solving the true problems. –  Jul 09 '13 at 06:19
  • fresh & related: http://tex.stackexchange.com/questions/123234 – Count Zero Jul 09 '13 at 10:16
  • 1
    @HeikoOberdiek: The database does not need to contain all package dependencies. Only the known conflicts. It might keep 'tracking' \RequirePackage, though, so it can do the 'right thing'. --- The amsmath vs wasysym thing is a very good point! I see two possible ways to address this kind of thing. [1] It is a conflict, the resolution is to fix the order (amsmath before wasysym), but the poweruser can explicitly disable such resolutions in order to provide their own. [2] This is not registered as a conflict, but as a choice with a default. – mhelvens Jul 09 '13 at 11:04
  • 2
    @mhelvens: Example, class beamer or package bookmark load package hyperref. Therefore a package that conflicts with hyperref also conflicts with beamer or bookmark. – Heiko Oberdiek Jul 09 '13 at 11:07
  • 7
    @AxelSommerfeldt: Loading order is indeed only part of the problem. But I'm neither claiming that pkgloader will solve all the worlds problems, nor saying that it will stop at loading-order. For better or worse, we're stuck with LaTeX for a while and might as well fix what we can. I do believe that even fixing the loading order alone would solve many problems (thanks for the link, @CountZero). – mhelvens Jul 09 '13 at 11:14
  • 1
    @HeikoOberdiek: Ah! Now I see what you mean. Yes, indeed. In those cases the database will also need to contain the relevant package dependencies. We don't need to repeat the induced conflicts, though, e.g., the database would contain {bookmark} loads {hyperref} rather than \Load {fancyhdr} before {bookmark}. --- Also, when such a dependency was not 'known', pkgloader could still realize that hyperref was loaded prematurely, diagnose the problem and warn the user. – mhelvens Jul 09 '13 at 11:20
  • @mhelvens IMHO we are not stuck with LaTeX for a while. ConTeXt already exists ;-) –  Jul 09 '13 at 18:57
  • 3
    Don't forget the optional version specification in \usepackage[opts]{pkg}[yyyy/mm/dd]. (I'm not sure I'm a friend of a formulation that scans your preamble, I'd prefer something vaguely like \usepackages{[ngerman]babel,[T1]fontenc[2013/12/12],listings} in whatever varation is convenient to parse. – Ulrich Schwarz Jul 09 '13 at 19:35
  • @UlrichSchwarz: Good point about the version specification! Anyway, the package doesn't scan the preamble. It only 'takes possession' of \usepackage commands. The advantages are [1] that only minimal changes are necessary from your usual preamble and [2] that we can keep track of which packages are being loaded by other packages (outside of our control), so that we can at least show relevant error messages, probably even take advantage of this information at the next run! – mhelvens Jul 09 '13 at 20:46
  • 2
    I often thought about this, but I've only got as far as to imagine a "LaTeX Packages (In)compatibility Matrix", a 2D table version of the linked wiki list (Freek). I'd happily collaborate in the creation of such database. I think even while the package is being developed, the database should be online, up-to-date and browsable. – marczellm Aug 05 '13 at 21:17
  • Sounds good to me. What we'd probably need is a LaTeX file with the database in it, parsable by the package, as well as by the website (are you good with javascript?), so both draw from the same source. --- Thing is, I'm pretty busy these days —with a number of other packages (not to mention a PhD thesis)— so I can't be as active with this as I would like. --- If you like, you could start things off by setting up a webpage somewhere? – mhelvens Aug 05 '13 at 21:45
  • I vote for \usepackages. That's a good starting point IMHO. It would do a very clear job, and sort-of avoid messing with the problems like \iiint, amsmath and wasasym together, because you could simply \usepackages multiple times. – yo' Nov 28 '13 at 21:02
  • Hm. This seems to be purely a matter of syntax. Functionality-wise it doesn't matter much; you can also call \usepackage multiple times. Anyway, this syntax seems fine to me. But taking possession of the existing \usepackage-like commands has some other nifty advantages. For example, we can recursively keep track of what all packages are loading, and integrate that knowledge for the next run. If not that, we can at least give the user clear error messages when they mess up the order. – mhelvens Nov 28 '13 at 21:18
  • 2
    I don't understand what answers you are expecting to this question. Could you give a clear statement what I should write which would be an acceptable answer to you? – Stephan Lehmke Nov 30 '13 at 05:32
  • 3
    For your information, we (the LaTeX3 team) would strongly prefer that people avoid using names which start with l3, as this wrongly gives the impression that it is supported by us. Right now I don't have much time, but graphs are eventually on my todo list :-). – Bruno Le Floch Dec 06 '13 at 00:31
  • @BrunoLeFloch: Two questions: (1) Why duplicate the effort? Better to collaborate. (In fact, you can take over completely whenever you want. I just need this functionality and want to make it available.) (2) In any case, the package name should make it clear that it is an expl3 datastructure. What kind of name would you suggest? Perhaps you should come up with a prefix for user contributed expl3 libraries. --- @@StephenLehmke: Well, I'm trying to gather feedback. Trying to make sure I do this in a way that doesn't bother other package authors. – mhelvens Dec 06 '13 at 12:32
  • @mhelvens (1) I'm not necessarily planning on duplicating the effort, but I will have to read your code to decide whether this is the approach I want to take for graphs: there are very many notions of graphs, and which one to take as the most fundamental influences performance. (2) I've raised the question on the Team list of how third-party packages should be named, I'll let you know. (3) StephenLehmke won't be notified, you'll need to write a separate comment for that. – Bruno Le Floch Dec 07 '13 at 06:16
  • @BrunoLeFloch: I humbly suggest the prefix lt3. Different from l3 but not too obtrusive. --- And with all due respect for the bureaucratic process, I have already submitted the package to CTAN under the name lt3graph. I want to move on to the pkgloader package as soon as possible. The lt3graph documentation explains that the the package is not supported by the LaTeX3 team. I hope that is sufficient. – mhelvens Dec 07 '13 at 14:17
  • @StephanLehmke: Hi Stephan, I had hoped my original description was clear. Though I admit it's not the standard kind of question for TeX.SE. --- In short, I want to finish and release this package, but I want to make sure I gather feedback from the LaTeX community first. This package will be relatively invasive and I want to make sure everyone is happy with it. That way, I hope it may become a recommended standard. – mhelvens Dec 07 '13 at 14:25
  • @mhelvens This comment at least does nothing to clear things up for me, as you are talking about yourself and not explaining what kind of answer (written by me) you would mark "accepted" :-) To me it sounds as if you are asking for comments, which you have received plentiful, but what about answers? – Stephan Lehmke Dec 07 '13 at 14:44
  • @StephanLehmke: I don't think there is a single answer to this question. In that sense I guess I've abused the stackexchange system. --- Perhaps it should be a big-list kind of question, to which each answer should offer relevant contributions. – mhelvens Dec 07 '13 at 15:55
  • @mhelvens I'm currently thinking about making a big list question (maybe on meta) "What package are you developing" where authors could ask for comments. Otherwise we'll have a lot of highly voted unanswered questions soon. – Stephan Lehmke Dec 07 '13 at 16:02
  • To those who voted to put the question on hold: I removed the one question that could be seen as too broad: "Any other feedback?" The other (related) questions would seem to be quite specific. --- (PS: The package is almost ready to go on CTAN.) – mhelvens Jan 30 '14 at 11:06
  • Didn’t read all the comments but have you seen (and implemented) http://tex.stackexchange.com/q/1863/4918? – Tobi Mar 07 '14 at 01:39
  • @Tobi: Very useful, thanks! No, in fact, very few rules have been implemented so far. The package is still somewhat immature, and advice like this is exactly the kind of thing it needs. Cheers! – mhelvens Mar 08 '14 at 10:52
  • Is the package still in development? It seems there has been no development since last year. I really wish to implement this in my projects but do not want to make it part of my routine if it's not supported. :-) – Michael Andersen Sep 08 '15 at 16:09
  • 1
    @MichaelAndersen: Unfortunately I haven't had time to actively develop this package. The basic functionality is there, and I do handle all bug reports. But don't expect new features anytime soon. (If someone else wants to do work on this, I'd be happy to give all support I can.) – mhelvens Sep 08 '15 at 20:52

1 Answers1

18

The first version of the pkgloader package has been released! Based on all of the useful comments above, and my own (re)search, I feel safe to answer my own individual questions as follows:

  • Is there already another package that does (something like) this?

No. The closest that has been suggested is scrlfile, and, frankly, it's not all that close. Also, others have since asked for this kind of package, just like I did.

  • Are there any specific pitfalls I should be aware of?

The comments above name a couple of important issues that need to be taken into account.

  1. Both options and minimal version need to be tracked for each package request.
  2. There is not always one correct package loading order. Sometimes both choices are valid, even if they yield different results. This may just be a matter of user-preference.
  3. Sometimes ordering two packages is not enough, but some 'glue code' needs to be run to make two packages work together.

There are probably plenty of other issues. But as for specific pitfalls, I haven't found any that I haven't been able to work around. As always, I'm open to new information (which should go to the Github issue tracker, please).

  • Other than Freek's page, is there any kind of comprehensive list of known conflicts and resolutions?

It doesn't seem so. To equip pkgloader with a good set of rules, the following should be done:

  1. Encode the problems and solutions on Freek's page into pkgloader rules.
  2. Search tex.SE for conflict reports and encode those.
  3. Once pkgloader has been thoroughly tested and is trusted by the community, new conflicts (and resolutions) posted to tex.SE should be reported to the pkgloader issue tracker.

Eventually, the pkgloader developers (for now, just me) will need to maintain a comprehensive list.

As mentioned in the release-notes, I wouldn't yet trust the package blindly to handle my package loading needs, although I haven't yet found any bugs. A suite of unit-tests needs to be written, additional features need to be implemented and a useful set of rules needs to be defined.

Please feel free to contribute! Just use the Github issue tracker, or contact me personally.

mhelvens
  • 6,126
  • 4
    Is pkgloader dead? – Viridjan Dec 10 '16 at 17:18
  • 6
    @Viridjan: Not exactly, but it's not in active development. I firmly believe that this is the right approach to package loading. But I've only heard from one or two people who are using it, and I haven't had to use LaTeX myself in several years (new job). If there's a bug, I'll try to find time to fix it. If someone would like to take over development, I'd be happy to work with them to get them started. – mhelvens Dec 10 '16 at 22:23
  • 1
    @mhelvens I see it hasn't been updated in a couple of years... do you still maintain it passively though? – Noldorin Jan 15 '21 at 22:08
  • 1
    @Noldorin I wish I could say yes, but I haven't used LaTeX in years. I've toyed with the idea of keeping my packages up to date, but I never manage to actually get to it. What I said in my previous comment still holds though. If someone wants to pick it up, I'd be happy to help get them started. – mhelvens Jan 17 '21 at 22:23
  • 1
    @mhelvens Good to know, thanks. I might be up for that, if Joseph Wright isn't (I know he submitted a PR on GitHub, and is a LaTeX dev, so he's much more knowledgeable than me). – Noldorin Jan 17 '21 at 22:33