For this case you can use the Cross-Classification and Tabulation as was mentioned above. To use it, first you need to have raster layer with int values. Depending on you usage layer you would need to perform some reclass or recode to have your raster ready.

The other premise is that your polygons(your buffer zones) must have some int value. If not, you have to create it (with the calculator formula $id+1).

Rasterize your vector layer using the int code as raster value. Here you have to put some conditions: use same pixel value and same extension as the raster that you want to extract the information.

Now use the Cross-Classification and Tabulation from SAGA tools. You have to consider that this is going to create a xy table with the count of pixels of each usage category into each polygon. Here you have to insert the "max number of classes" what means that, for example, if you have a raster with 10 categories and a 25 polygon classes, your max will be 25.

Here is the final result: a table where you have the count of pixels of each category in each polygon. If you need to convert it to area, just multiply the number of cells by the resolution of you raster.
