Fitting lifelines CoxTimeVaryingFitter() to large training set?

Since Cox’s time varying proportional hazard model doesn’t appear to be implemented in dask_ml, and the lifelines package doesn’t appear to offer integration with dask, is there any way I can leverage dask to fit lifelines’ CoxTimeVaryingFitter() to a large training set?

Hi @MarkPundurs, welcome to this forum.

If you want to fit a model on a dataset larger than memory using Dask, as stated in

https://ml.dask.org/incremental.html

the model must at least implement the partial_fit API from Sickit-Learn.

By looking at lifelines package documentation, it seems it does not support it.

I guess there is no magic wand here, either the algorithm is compatible with this and someone needs to implement this, either this is not possible to fit this model on a dataset larger then memory using Dask.

Hope that helps.