Creating a simulation to find a p-value for a slope, is this valid?

Question

I plot Score (y variable) against Year (x variable) and I want to be able to show that there is an increase in the score as the year increases. The scatter plot indeed indicates this but I would like to get a p-value for this by way of an appropriate hypothesis test on this slope. I believe that a linear regression t Test is not valid as my data Year is not normally distributed (it being discrete and uniform).

I thought to then create a simulation and I want to know if what I have done is a valid technique.

1. I find the linear regression slope of the actual data, which I called slopeActual.
2. I did a simulation (say 1000 times) whereby at each loop I permuted the y values (Score) and calculated a regression slope. These values I stored in a list I called slopeList.
3. I calculate: p-value = P(slopeActual>0 | there is no association) = proportion of values in slopeList greater than slopeActual.

When I did this for the data below I got a p-value of 0.0087.

So, the question again: Is this method valid?

data1 <-
structure(list(Year = c(1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 3L, 3L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 5L, 5L, 5L, 5L, 
5L, 5L, 6L, 6L, 6L), Score = c(-5.2, -2, -1, -3, -5, 3.8, 0, 
-3.2, 1.2, 2.2, 11.5, -4, 10.2, 2, 12, 6.5, 6, 6, 9.2, 4.2, 13, 
0.8, 8.5, 4.5, 6, 6, 2.7, 8, -3.8, 6.7, 4.5)), .Names = c("Year", 
"Score"), class = "data.frame", row.names = c(NA, -31L))

Answer 1

1. This premise of the question is misplaced: "I believe that a linear regression t Test is not valid as my data Year is not normally distributed (it being discrete and uniform)."

   ... there's no such assumption in regression. Your x-variables are not assumed to have any particular distribution (indeed, neither is there an assumption about the marginal distribution of the y-variable)

2. "p-value [...] = proportion of values in slopeList greater than slopeActual"

   Very nearly correct.

   You should include the original sample in your list, and then count the cases greater than or equal to that one.

   I'd also suggest doing it more than 1000 times; your estimate of the p-value will be pretty variable and if it happens to come out close to a boundary you may want it to be reasonably precise (it won't make much difference in this case unless you're doing a 1% test).

On your data with the above modifications and 10000 simulations (each such took about 17 or 18 seconds on my laptop) I got p-values just below 0.0084, 0.0083, and 0.0092 across three trials.

(The one-tailed p-value under the ordinary regression assumptions is a bit below 0.0078; doing a lot more permutations seems to be getting us quite close to that -- after 60000 permutations I have a p-value of about 0.00797.)