remove text glyph from legend in ggplot

Question

Normally I would remove these kind of things with a call to guide_...(override.aes = list(<text-label> = c(NA, NA, NA)), but I do not know how to do in the case of these a's appearing in the legend that come from the geom_text geom.

The code to produce the below figure is the following:

ggplot(mtcars) +
  geom_point(aes(x = mpg,
                 y = disp,
                 color = as.factor(cyl))) +
  geom_text(aes(
    x = mpg,
    y = disp,
    label = wt,
    color = as.factor(cyl)
  ))

Which produces this figure:

And I want to remove the a's that are overlayed in the legend in top of the colored points.