How can you use vim as a stream editor?

Question

Like most programmers, I perform a lot of repetitive tasks. In optimising my workflow, I'm taking some of those repetitive tasks, and refactoring them into shell scripts.

One thing that I'm trying to automate is the recreation of PostgreSQL views. I have the following view.

create or replace view person as
select
    1 as person_id
    , 'John'::text as first_name,
    'Doe'::text as last_name;

I can dump this view with psql -c "\\d+ person" and the output is as follows:

                   View "public.person"
   Column   |  Type   | Modifiers | Storage  | Description 
------------+---------+-----------+----------+-------------
 person_id  | integer |           | plain    | 
 first_name | text    |           | extended | 
 last_name  | text    |           | extended | 
View definition:
 SELECT 1 AS person_id,
    'John'::text AS first_name,
    'Doe'::text AS last_name;

I can reformat this text into a CREATE OR REPLACE VIEW statement with the following keystrokes in vim: gg0df"iCREATE OR REPLACE VIEW <ESC>$cl AS<ESC>j0d/^View definition:<CR>ddG$:wq (I've reformatted <ESC>, etc, in the above).

I've got the above working perfectly, except for the screen "flashes" that occur. For example, if at the shell I type a=$(psql -c "\\d+ person" | vim -s <vimscriptfile> -); echo "$a" then my screen "flashes" before outputting the nicely format SQL.

Is there any way to remove this flash? Or is there a better using-vim-in-a-pipeline approach than what I'm employing?

Answer 1

To avoid screen flashes when editing stream non-interactively, you need to start Vim in Ex mode by adding -e (Ex mode) or -E (improved Ex mode) into your command-line arguments.

Here is the simple example editing text from the standard output and printing the result into standard output:

$ echo "foo bar" | vim -E +%s/foo.// +%p -cq! /dev/stdin
bar

To parse your output, you may try the following syntax:

$ psql -c "\\d+ person" | \
  ex +'1s/\s\+View "\([^"]\+\)"/CREATE OR REPLACE VIEW \1 AS/g' +'2,/View.definition/d' +%j +%p -scq! /dev/stdin

The output should look like:

CREATE OR REPLACE VIEW public.person AS SELECT 1 AS person_id, 'John'::text AS first_name, 'Doe'::text AS last_name;

Explanation:

• Using ex command is equivalent to run Vim in Ex mode (vim -e).
• +'cmd'/-c {cmd} - Invokes Ex command.
• -cq! - Forcibly quit Vim as we're not saving any files.
• -s - Run in silent mode (prevents extra screen flashes).
• /dev/stdin - We use /dev/stdin as an input file, because using - doesn't work properly.
• 1s/pattern/something/g - Substitute pattern with something in the 1st line.
• 2,/pattern/d - Delete content starting from the 2nd line to the line with pattern.
• %j - Joins all the lines together.
• %p - Prints the buffer to the console output.

Answer 2

I was not able to reproduce your macro but here is a (far from ideal) approach to your problem: Register persistently your steps in a macro and then launch Vim and tell it to directly execute your macro and close again.

Ex: In your .vimrc, you register the macro:

" convert everything to uppercase
let @x = 'ggVGU'

Start Vim, call the macro from the cmd line and close Vim again:

$ cat test.txt
sdfas;dlfasdf
asdf
adf

$ vim -c ":norm! @x" -c ":x" test.txt

$ cat test.txt
SDFAS;DLFASDF
ASDF
ADF

This will obviously start vim and close it right a way. I guess there are some optimizations possible. Make a shell alias for the whole thing, start Vim without plugins/vimrc and only load your dummy vimrc (to make startup fast if that's a problem), probably other ideas as well.

This answer explains how to record macros and add them to your .vimrc.