Is there a Python class/enum for flag/bit mask operations?

Question

I know of base classes Enum and IntEnum. Both are very helpful but I miss features for flag operations. I don't expect that these two classes implement my wished feature.

Let's construct an example:

class NetlistKind(IntEnum):
  Unknown = 0
  LatticeNetlist = 1
  QuartusNetlist = 2
  XSTNetlist = 4
  CoreGenNetlist = 8
  All = 15

As you can see, I'm already using IntEnum to get arithmetic features for this enum. It would be nice to have something like @unique to ensure all values are a power of two. I can do this by forking enum.unique for my needs. (I'm aware that All is an exception from that rule.)

How is such an enum used?

filter = NetlistKind.LatticeNetlist | NetlistKind.QuartusNetlist

Thanks to the underlaying int bit operations are possible and filter has an internal value of 3.

If would be nice to have a "is flag X set in filter Y" function or even better an operator. I add a magic function for x in y:

@unique
class NetlistKind(IntEnum):
  Unknown = 0
  LatticeNetlist = 1
  QuartusNetlist = 2
  XSTNetlist = 4
  CoreGenNetlist = 8
  All = 15

  def __contains__(self, item):
    return  (self.value & item.value) == item.value

Usage example:

....
  def GetNetlists(self, filter=NetlistKind.All):
    for entity in self._entities:
      for nl in entity.GetNetlists():
        if (nl.kind in filter):
          yield nl

  def GetXilinxNetlists(self):
    return self.GetNetlists(NetlistKind.XSTNetlist | NetlistKind.CoreGenNetlist)

So the questions are:

• Are there better ways to implement bit fields?
• Are thete better ways to implement such an 1-D filter? I don't want to use lamdas for such a simple filter condition?
• Is such solution already included in the Python standard library?
• How to add this enum extension to the next Python version? :)

Open features:

• return a list of all active flags in __str__
• ...?

Answer 1

Python 3.6 has added Flag and IntFlag which support the usual bit-wise operations. As a bonus, the resulting values from the bit-wise operations are still members of the original flag class, and are singletons [1].

The aenum library also has this addition and is usable back to Python 2.7.

[1] A bug exists in 3.6.0: if the psuedo-flag members are being created in threads then there may end up being duplicates; this is fixed in 3.6.1 (and never existed in aenum).

Answer 2

I've recently published an opensource package py-flags that aims this problem. That library has exactly this functionality and its design is heavily influenced by the python3 enum module.

There are debates about whether it is pythonic enough to implement such a flags class because its functionality has huge overlaps with other methods provided by the language (collection of bool variables, sets, objects with bool attributes or dicts with bool items, ...). For this reason I feel a flags class to be too narrow purpose and/or redundant to make its way to the standard library but in some cases it is much better than the previously listed solutions so having a "pip install"-able library can come in handy.

Your example would look like the following using the py-flags module:

from flags import Flags

class NetlistKind(Flags):
    Unknown = 0
    LatticeNetlist = 1
    QuartusNetlist = 2
    XSTNetlist = 4
    CoreGenNetlist = 8
    All = 15

The above things could be tweaked a bit further because a flags class declared with the library automatically provides two "virtual" flags: NetlistKind.no_flags and NetlistKind.all_flags. These make the already declared NetlistKind.Unknown and NetlistKind.All redundant so we could leave them out from the declaration but the problem is that no_flags and all_flags don't match your naming convention. To aid this we declare a flags base class in your project instead of flags.Flags and you will have to use that in the rest of your project:

from flags import Flags

class BaseFlags(Flags):
    __no_flags_name__ = 'Unknown'
    __all_flags_name__ = 'All'

Based on the previously declared base class that can be subclassed by any of your flags in your project we could change your flag declaration to:

class NetlistKind(BaseFlags):
    LatticeNetlist = 1
    QuartusNetlist = 2
    XSTNetlist = 4
    CoreGenNetlist = 8

This way NetlistKind.Unknown is automatically declared with a value of zero. NetlistKind.All is also there and it is automatically the combination of all of your declared flags. It is possible to iterate enum members with/without these virtual flags. You can also declare aliases (flags that have the same value as another previously declared flag).

As an alternative declaration using the "function-call style" (also provided by the standard enum module):

NetlistKind = BaseFlags('NetlistKind', ['LatticeNetlist', 'QuartusNetlist',
                                        'XSTNetlist', 'CoreGenNetlist'])

If a flags class declares some members then it is considered to be final. Trying to subclass it will result in error. It is semantically undesired to allow subclassing a flag class for the purpose of adding new members or change functionality.

Besides this the flags class provides the operators your listed (bool operators, in, iteration, etc...) in a type-safe way. I'm going to finish the README.rst along with a little plumbing on the package interface in the next few days but the basic functionality is already there and tested with quite good coverage.