Various recent efforts of mine on modelling some data through logistic regression have been… not successful. While there is still more data to look at, I’ve been wanting to explore nonlinear dependencies in the data too.
Are there techniques out there that have better fitting capabilities than logistic regression (or linear regressions in general) that still retain easy interpretability and explanatory power (i.e. are not neural networks or other ‘black box’ techniques)?
My initial intuition is to apply transforms to the individual variables and combinations thereof. For the variables $ x_1…x_n $, possibilities include taking $log(x_i)$, $x_i^m$, $sqrt[m]x_i $ or $x_i * x_j$ etc. Third order relationships (i.e. $x_i *x_j *x_k$) would be omitted as they aren’t easy to visualise or graph.
Needless to say, even for a smallish number of variables, the number of these transformed variables could easily balloon to unwieldy sizes, and the number of combinations of these variables to systematically try would be similarly huge. Furthermore, there would be issues of multicolinearity between the various transforms of a given variable $x_i$.