Exclude start and end match from region

Question

I'm trying to establish syntax coloring for a certain lexical syntax: a "buffer literal". A buffer literal begins with #b' and ends with '. Between the single quotes, there can be undivided pairs of hex digits, with optional whitespace in between, that being spaces, tabs and newlines.

Valid buffer literals:

#b'' #b'FF' #b'Fe' #b'dead be
                      ef'

Invalid:

#b'z' #b'FFF'  #b'9 0'

Here is what I have:

syn match buferr "." contained
syn match bufok "[0-9A-Fa-f][0-9A-Fa-f]\|[\t\n ]" contained

syn region buflit start="#b'"rs=e+1 end="'"re=s-1 contains=bufok,buferr

hi def link buferr Error
hi def link bufok String
hi def link buflit String

The problem is that the bufok and buferr match items are being applied to the region delimiters. And thus the #b' start and ' end are matching buferr and being highlighted as Error.

I tried to compensate for this using the rs (region start) and re (region end) offset options on the region patterns, seen above, but it has no effect. The matches are not contained to occur between the indicated region start and end.

How can we tell Vim that we want this:

#b'....'
   rrrr  <- this is the region, in which bufok and buferr are contained!

It really looks like rs and re offsets should be doing this, since the documentation says:

The pattern can be followed by a character offset. This can be used to change the highlighted part, and to change the text area included in the match or region (which only matters when trying to match other items).

On the contrary, my above fiddling with rs and re doesn't seem to be having an effect on the matching of contained items.

I'm working with Vim as old as 7.3.429.

Answer 1

There are a couple of nuances that need to be grasped in order to make it work:

1. The "." in buferr matches ' and will eat it up before the pattern in buflit can see it and, as a result, the buflit region mismatch both delimiters. You could deal with this issue by adding keepend (see :help :syn-keepend) before the conteins argument, but you won't need it in this case.

2. The rs and re offsets only make sense when they're used along with matchgroup, and that happens to be very helpful because of this (from :help :syn-matchgroup):

   ...
   In a start or end pattern that is highlighted with "matchgroup" the
   contained items of the region are not used.  This can be used to avoid
   that a contained item matches in the start or end pattern match.
   ...
   

and, as you'll see below, you won't need the rsand re offsets in this case either.

syntax case ignore
syntax match bufok "\x\x\|\_s" contained
syntax region buferr matchgroup=buflit start="#b'" end="'" contains=bufok

highlight default link buferr Error
highlight default link bufok String
highlight default link buflit String

Here the region was redefined by highlighting:

• its content as a error by default
• its delimiters with the buflit highlighting group
• the pair of hexadecimal digits with bufok

I also:

• made the :syntax command case-insensitive with syntax case ignore (see :help :help :syn-case
• used \x to match hexadecimal digits (see :help /\x)
• used \_s to match whitespace characters, including the end-of-line (see :help /\s and :help /\_s)

Note: \x and \_s are faster than the [] form.

Bonus

I used the long form of each command because some experts around here recommend it (and I agree with them), for example:

• From :help usr_20.txt

  It is recommended that in Vim scripts you write the full command name.  That
  makes it easier to read back when you make later changes.  Except for some
  often used commands like ":w" (":write") and ":r" (":read").
  A particularly confusing one is ":end", which could stand for ":endif",
  ":endwhile" or ":endfunction".  Therefore, always use the full name.
  

• From romainl's answer:

  you should use the long form names of your settings as they are more readable than their short counterpart. Your future self will thank you.

• From Tommy A's comment:

  I consider the short commands as a command line convenience. I always use full command names in scripts. It's frustrating when the short commands are used in plugins. It makes grepping/searching scripts harder than it needs to be.

Answer 2

Supplementary answer: based on Jair López's clue about matchgroup, the following minimal change rescues the original concept:

syn match buferr "[^']" contained                                                                                                            
syn match bufok "[0-9A-Fa-f][0-9A-Fa-f]\|[\t\n ]" contained                                                                               

syn region buflit matchgroup=buflit start="#b'" end="'" contains=bufok,buferr                                                  
hi def link bufok String                                                                                                                  
hi def link buferr Error                                                                                                                  
hi def link buflit String                                                                                                     

The first key difference from the nonworking definition is that there is a matchgroup=buflit there which means that the start and end delimiters are treated as being bufok, and colorized as String via the hi def link.

The second key difference is that the error match is "any character but apostrophe" rather than "any character". This way the error match doesn't compete with the closing delimiter.

Also, the offset specifiers rs= and re= are removed; they are a red herring in this situation that make no difference.

This solution differs from the accepted solution in a small way: it doesn't require the region to be categorized as a one big Error in which the good pieces of syntax are the exceptions to the error. The errors are recognized explicitly within the region as positive matches. The default behavior of "everything else that is not good token material is an error" is implemented by a single character match routed to Error: flag and drop the bad character.

The end effect appears identical as far as I can tell with interactive testing.

So the exact differences relative to the accepted solution:

• the region is called buflit rather than buferr and highlighted as a String.
• the buferr name is used for a one character bad match rather than the region.
• the region contains buferr also, not only bufok.

It's good to keep both approaches in mind: error highlights can be identified as explicit matches of erroneous figures on a good background, or as implicitly unmatched gaps between good figures on an error background.