After calculation of log partial hazards, how do I calculate the survival function for prediction of remaining [useful] life [RUL] for unknown/test dataset with the time-varying covariate values?
The method in the linked post has some problems.
First, it depends on having complete survival times for a set of identical turbofan engines, each run until failure. That way you know the actual RUL for each fan at each time point. That's unlikely in many real-life situations.
Second, one step raises red flags for those experienced with survival analysis:
... we’ll clip any RUL values above 125 ...
That's precisely one type of thing that survival analysis is intended to avoid: arbitrarily cutting off long survival times. You want to use all of the available data.
Third, although the log partial hazards are evaluated from a survival model, the RUL values are not. Instead, a calibration curve is used to estimate the association between log partial hazards and the (known) RUL. Effectively, you calculate a log partial hazard and read the corresponding RUL (determined separately) from a curve based on the data set.
Fourth, you have to be really careful with time-varying covariate values in Cox models. At each event time, a Cox model only evaluates the covariate values that are in place for those at risk at that time. If covariate values are constant over time, it's relatively easy to think about their overall associations with outcome. If they are changing over time (as in the sensor values in the linked post) then there's a risk of circular reasoning: if you have a covariate value at some survival time, then you already know survival has been at least that long. The author of lifelines thus doesn't allow for predictions based on time-varying covariates.
Fifth, the two prior points mean that adjustment of the baseline survival curve (which you should be able to extract from the model) for covariates, which is what you want to accomplish, is tricky in this situation. With time-constant covariates, you specify sets of covariate values, calculate corresponding log partial hazards (aka linear-predictor values), and simply estimate new sets of complete survival curves as in this answer. If covariate values are changing in time you have to re-adjust the survival function for each new set of values/times, conditional on surviving already to each time. The R survival package (unlike lifelines) will allow for such calculations, but you have to make sure that the time trajectories of new covariate values make sense. Furthermore, any estimates of survival beyond the last change of covariate values will be based on that last set of values.
Sixth, what's probably needed here is a joint model of longitudinal covariate values (sensor readings) over time and survival. That's beyond my expertise. The R Survival Task View provides some suggestions.
