0

I have a bash pipeline. I would like to stop too much work being by the earlier workers before the later workers have processed it. Is there a way to decrease the amount data that can be stored in a pipe and I can I specify this limit in lines rather than bytes?

I'm aware that the "correct" way of doing this would be to use a message passing system -but that seems like a bunch of overhead.

Att Righ
  • 1,263
  • 1
  • 13
  • 21
  • 2
    That's up to the individual programs to configure for themselves, IIRC. They'll be given data as often as they ask for it – Alexander Mar 22 '22 at 14:57
  • Once the buffer fills up, the pipeline should block, right? Are you seeing a problem, or just thinking ahead to one that may not exist? If you are really concerned, does https://stackoverflow.com/questions/51636391/how-to-set-pipe-size-from-shell help? – Paul Hodges Mar 22 '22 at 16:12

0 Answers0