3

How can be LuaTeX callbacks used in ConTeXt? (with LuaTeX backend)

My main concern is the usage of process_input_buffer callback available in LuaTeX and LuaLaTeX. I am using it to altering the compiled source to reflect typografic rules customary in my country (maybe customary in general), to avoid one-letter prefixes ending the line.

Next MWEs should illustrate that:

One-letter prefix on the end of line (should be avoided):

\documentclass{article}

\usepackage{xcolor}


\begin{document}
Filling text filling text filling text filling text filling text filling text \textcolor{red}{filling V text}
\end{document}

And correct version:

\documentclass{article}

\usepackage{xcolor}
\usepackage{luacode}


\begin{luacode}
function pevnaMezera(s)
  s = string.gsub(s, " V ", " V~")
  return s
end
\end{luacode}


\AtBeginDocument{%
  \directlua{luatexbase.add_to_callback(
  "process_input_buffer", pevnaMezera , "pevnaMezera" )}
}


\begin{document}
Filling text filling text filling text filling text filling text filling text \textcolor{red}{filling V text}
\end{document}

I am aware of this question: How to register a callback in ConTeXt; Howerver, presented answer seems focused only on callback pre_linebreak_filter, which is in ConTeXt defined as finalizer.

I wasnt able to find anything about finalizers in general in any ConTeXt documentation or on ConTeXt Garden.

NOTE: In actual projects I am using lua regexes in function for process_input_buffer callback, for brevity and readability I made simplier definition in MWE illustrating the problem.

EDIT3:

MWE changed according to comment of @Noone:

\startluacode
userdata = userdata or {}

function userdata.pevnaMezera(s)
  s = string.gsub(s, " V ", " V~")
  return s
end


process_input_buffer = userdata.pevnaMezera
\stopluacode


\starttext
Filling text filling text filling text filling text filling text filling text filling text fil V text
\stoptext

Is not adding non-breakable space. Is it correct from syntactical point of view?? I tryed to move the "assignment" to luacode block, but it did not work either.

Tomáš Kruliš
  • 1,062
  • process_input_buffer = pevnaMezera should just work, as it is defined the engine itself, although I must recognize it isn't a very robust implementation. –  Jul 29 '20 at 11:20
  • Do not use process_input_buffer to search and replace text! It is close to impossible to write down a pattern that is both sufficient and exhaustive at the same time. Also the error handling scenarios are frightening, because you either get an extremely obscure error that is very hard to debug or no error at all and silently garbled output. – Henri Menke Jul 29 '20 at 11:37
  • That is true, sometimes I have hit a brick with this and spent some time finding the cause, but from my perspective it mostly works (but in some places it fails). Still, I would like to ask, what is ConTeXt equivalent of this callback. After having this as temporary workaround, I try to find how to achive the same in more cleaner, but also automated way. – Tomáš Kruliš Jul 29 '20 at 11:56
  • @Noone See please edited question. If I understood you correctly (from syntactical point of view), then your suggestion is not acting as expected. – Tomáš Kruliš Jul 29 '20 at 11:57
  • @TomášKruliš First, it's better if you use userdata = userdata or {} when you define Lua functions in ConTeXt, so you define your function as userdata.myfunction. Second, you could include the line directly in the luacode environment, if you decide to do so. But third, Henri is right when he says you shouldn't use process_input_buffer in the first place (AFAIK ConTeXt doesn't use it). –  Jul 29 '20 at 12:12
  • @Noone Thank you for your correction. However, now I am in turn getting error about end-of-file expected at end or in different places (if I try different syntax). I understand that this is very dirty solution. But in some cases it is very helpfull workaround, Ex: Friend as a self-publisher turned to TeX, writing novel, needed solution for prefixes staying on the end of line. For simple text, this is fast and good enough workaround. – Tomáš Kruliš Jul 29 '20 at 12:27
  • You're missing function before –  Jul 29 '20 at 12:34
  • @Noone Yes, noticed shortly after commenting, sorry for that. Still, the expected (hoped for) input buffer processing is not happenning (no non-breakable space is inserted). Could it be that the function argument is just ignored? – Tomáš Kruliš Jul 29 '20 at 12:37
  • I have answered this exact same question before using ConTeXt's processors mechanism. I have marked it as a duplicate. Let me know if that is not the case so that I can reopen it. – Henri Menke Jul 29 '20 at 23:03
  • @HenriMenke You have my sheer admiration Mr. Menke. I did not find solution to the original question, however, I am aware of luavlna and yesterday started to rewrite it for ConTeXt. Did not know that it is done already ... – Tomáš Kruliš Jul 30 '20 at 08:36
  • @TomášKruliš Honestly, I was searching for it myself for a good 10 minutes, even though I knew that I wrote the answer. That's because the title of that question is not unique enough for a useful search and I couldn't remember it anyway and the tags also don't really narrow it down. – Henri Menke Jul 30 '20 at 08:39
  • @HenriMenke I really need to work on my searching skills ... – Tomáš Kruliš Jul 30 '20 at 08:40
  • I've reopened this question because even though your problem is solved by other means, the original question remained unanswered, so I dug a bit through the Lua code and summarized my findings. – Henri Menke Aug 09 '20 at 09:37

1 Answers1

4

The actual question of how to suppress line breaks after single letter words is answered much better in this thread:

Hard spaces after one-letter words in ConTeXt

Before I begin to show how to register your own callbacks in ConTeXt, some words of warning (copied from my comment):

Do not use process_input_buffer to search and replace text! It is close to impossible to write down a pattern that is both sufficient and exhaustive at the same time. Also the error handling scenarios are frightening, because you either get an extremely obscure error that is very hard to debug or no error at all and silently garbled output.

With that out of the way, let's get to it. ConTeXt by default sets up its own callback mechanism called processors and finalizers and “freezes” the original LuaTeX callbacks, i.e. makes them unusable and issues an error if you attempt to register one. The reason for that is that ConTeXt registers own callbacks that are tightly coupled and breakage is expected if you tamper with them. However, it is possible to unfreeze the callbacks by adding

\enabledirectives[system.callbacks.permitoverloads]

to your document. After that you can register callbacks as usual, e.g.

\startluacode
callbacks.register("process_input_buffer", function(line)
    return line
end)
\stopluacode

Note that in LMTX the callbacks are quite different from MkIV and some callbacks have been removed and other have been added. The process_input_buffer callback for instance has been removed in LMTX.

After enabling the above directive, ConTeXt will greet you with this nice warning in the log:

system          >
system          > The callback system has been brought in an unprotected state. As a result of directly
system          > setting of callbacks subsystems of ConTeXt can stop working. There is no support for
system          > bugs resulting from this state. It's better to use the official extension mechanisms.
system          >

That should make it even more clear that this should not be used.

Henri Menke
  • 109,596