Revisiting the MLP prior
Taking a closer look at the vanilla MLP's prior, which turns out to be poorly suited for tabular data. Rotational invariance, spectral bias, and (non)-resilience to irrelevant-feature are the culprits.
Taking a closer look at the vanilla MLP's prior, which turns out to be poorly suited for tabular data. Rotational invariance, spectral bias, and (non)-resilience to irrelevant-feature are the culprits.
Gradient-boosted decision trees dominated the 2010s on tabular data, and are still extremely prevalent today. We examine the reasons behind their success, and continued relevance in the 2020s.