The residuals from an ANOVA can be handled like any linear regression in R, in that once you run a saved object from an ANOVA or regression, you can typically obtain the residuals fairly easy and thus run a QQ Plot. Here is a quick way I borrowed from this site, though I skipped a lot of the other stuff since it wasn't relevant:
#### Save Data as Object ####
my_data <- ToothGrowth
Create Second Factor
my_data$dose <- factor(my_data$dose,
levels = c(0.5, 1, 2),
labels = c("D0.5",
"D1",
"D2"))
Run Two-Way ANOVA
res.aov <- aov(len
~ supp
+ dose,
data = my_data)
summary(res.aov)
Plot Residuals
plot(res.aov, 2)
Which should give you this QQ plot:

As someone else noted here, you can also plot a histogram of the residuals and see if they are normal as well.