Nietzsche on Tuning Deep Neural Nets

I’ve been spending quite some time over the summer designing networks architecture tailored for our beloved macroeconomic data. Needless to say, if you are used to work with trees ensembles — like yours truly — you’ll find deep NNs quite taxing tuning-wise. Surprise, surprise. It’s very hard to build a bad random forest, and spectacularly easy to construct an abysmal neural net. Speaking of abysses, this made me realize our friend Nietzsche, well before his time, had some prophetic thoughts on existential dread born out of hyperparameters tuning in NNs:

“Whoever fights with monsters should see to it that he does not become a monster in the process.

And when you gaze long into an abyss the abyss also gazes into you.”