3

I'm wondering what should be the minimum sample size to perform ridge, lasso or elastic net regression. I have a binomial outcome that I want to relate with a set of features (18 features in some cases and 36 in some others).

I'm performing a repeated analysis on subgroups of a population (penalized regression). Some of this subgroups have a moderate-large sample size (103 individuals). Some others have a total number of observations of 1, 4, 5 ,7 ,9 , 19 and so on.

From which sample size should I start consider the use of ridge/lasso/elastic net?

Thanks in advance!

Frank Harrell
  • 91,879
  • 6
  • 178
  • 397
A. Nicolás López L.
  • 73
  • I don't think there's really a magic number. It's a factor of other parameters of your model and data, for instance the amount of variance and the number of regressors. For the record, most people and journals would consider 103 be very small sample size. You can't do a regression with 1 observation and -- while there is no true magic number -- the rule of thumb I was taught is that 30 is typically the minimum sample size to do any statistical analysis. – Hack-R Mar 06 '16 at 16:50
  • 1
    Could you please say a bit more about the structure of your data? In particular, how does the yes/no nature of the outcome correspond to the "repeated analysis on subgroups": could the same individual have a "yes" outcome sometimes and a "no" response other times? And what is the number in the smallest of the 2 outcome categories for eachof your analyses? That's what's important in this type of model, not just the total number of cases. – EdM Mar 06 '16 at 17:45
  • Hi, thanks for your interest. Each observation corresponds to the number of plants who develop a disease on a plants field. The plants field are compatible only if they have the same age and ground material, which partition the dataset into several categories. Some of this have several individuals and some others just have very few. I would like to relate this outcome (incidence of sick plants) with chemical related characteristics of each field (18 continuous variables). – A. Nicolás López L. Mar 06 '16 at 18:08

2 Answers2

5

First, a comment on you overall design. You will almost certainly be better off if you analyze all your fields together. Even if "the plants field[s] are compatible only if they have the same age and ground material," that doesn't mean that you have to partition the data for analysis. You could include variables for age, ground material, etc in your model to account for these differences. That is typically much more powerful (in the statistical sense) than is analyzing separate subgroups.

Second, it's not completely clear how you intend to structure the form of the regression model. However you do this, make sure that you include information about the total number of plants in each field, not just the number or proportion that got the disease.

Third, to your question about sample sizes for penalized regression. I find it helpful to start with how many variables you can consider with a standard regression model. In your case, the number of plants that got the disease would be a guide: about 1 variable per 15 diseased plants if less than half of plants got the disease. (If you insist on doing separate subgroup analyses, then this limit would apply to each separate analysis. That's another reason to do a combined analysis in which you examine all fields together and include variables like age and ground material that characterize different fields.) So if you have 18 continuous variables plus a variable for the age of fields and one for each of a small number of ground types for the fields, something on the order of 300 diseased plants might be sufficient to consider even a standard non-penalized regression (300/15 = 20 variables), if the relations of the variables to outcome (in the appropriate scale) are linear and there are no interactions among predictors with respect to outcome.

One way to think about penalized regression methods is that they reduce the effective degrees of freedom used by the model. For the non-penalized regression, these degrees of freedom equal the number of predictor variables, including any dummy variables for multi-level categorical variables, interaction terms, and so forth. (The 15 diseased plants per variable guideline above is more precisely per degree of freedom.) Increasing the penalty used for ridge, LASSO, or their hybrid elastic net decreases the effective degrees of freedom, allowing consideration of more variables for the same data set.

If you set the penalty high enough, you can go as low as you want in terms of effective degrees of freedom. So from that perspective there really is no minimum number of samples you need before you apply these penalized methods. The problem is that with small numbers of samples the amount of penalization that you need to get a reliable model may be so great that you have only limited predictive power. After cross-validation to choose the penalty value, bootstrapping to evaluate the final model will provide information on how well the model might be expected to generalize outside your data sample.

EdM
  • 92,183
  • 10
  • 92
  • 267
  • Thanks so much for your comments. I totally agree with analyzing the complete dataset. I wanted to show the differences on both approaches. But as I noticed the small sample size for the partition approach in some cases, I was wondering how to handle it. According with your comments, it seems that there is no use on doing such thing, I'll go just for the model on the complete dataset.

    My final logistic model will consider the 18 chemical related features of each field, the ground material and the age, also the total number of plants for each field. It may not need regularization.

     – A. Nicolás López L. Mar 07 '16 at 02:01
  • If your interest is in prediction, a penalized regression might still improve generalizability. Also, if you suspect that the effects of some chemicals may differ based on the ground material or age, then you would need to include interaction terms also, increasing the effective number of variables. Penalized regression might make that less likely to lead to overfitting. – EdM Mar 07 '16 at 03:13
  • My interest is not in prediction. And I do not expect significant interaction effects, thanks for the warnings though. There is one thing still worries me, is the fact that the disease is very rare among the fields, so I think I'll have to consider the following http://gking.harvard.edu/files/gking/files/baby0s.pdf. And I may be limited by the use of penalized regression explaining rare events, right? – A. Nicolás López L. Mar 07 '16 at 03:22
  • Explaining rare events is always a challenge; even defining the word "rare" can be. Although you do not expect interaction effects, you still should consider evaluating whether they are present, as for any regression model. Depending on how your diseased plants were distributed among the fields, there may still be a danger of overfitting with a standard regression model and also issues of correlations among predictors, dangers that ridge regression may help minimize. – EdM Mar 07 '16 at 14:37
  • +1 For non-partitioned analysis – John Vandivier Oct 20 '21 at 15:02
  • The rule of thumb of 15/10 events per covariate can be improved further if you follow this approach: https://onlinelibrary.wiley.com/doi/full/10.1002/sim.7992 This is implemented in the R package pmsampsize. – The Doctor Oct 17 '23 at 09:18
5

Penalized estimation can be used for very small sample sizes; it's just that the estimates will be shrunk to the mean very heavily, so the estimates will be conservative. But the intercept in the model is not shrunk, and for binary $Y$ the minimum sample size needed just to estimate the intercept in a logistic model is $n=96$ yielding a margin of error of $\pm 0.1$ in estimating the probability. If you want to make subgroup-specific estimates that translates to 96 observations per subgroup.

I assume your outcome is Bernoulli, not binomial.

Frank Harrell
  • 91,879
  • 6
  • 178
  • 397