"In this paper, we focus on the following four patterns that appear to us to be trending in ML scholarship: (i) failure to distinguish between explanation and speculation; (ii) failure to identify the sources of empirical gains, e.g., emphasizing unnecessary modifications to neural architectures when gains actually stem from hyper-parameter tuning; (iii) mathiness: the use of mathematics that obfuscates or impresses rather than clarifies, e.g., by confusing technical and non-technical concepts; and (iv) misuse of language, e.g., by choosing terms of art with colloquial connotations or by overloading established technical terms. "
To their credit, the authors actually own-up to doing this themselves in various papers. It seems like a way to describe the situation is that neural nets have become such computational monsters that talking about them exactly becomes very difficult with the language opaque and ambiguous.
I'd say a lack of a proper fundamental understanding of trained neural networks is the main cause. People throw NNs at any problem they can think of, get good results and when they want to publish, they come up with an explanation that is more esoteric than founded in solid theory because the monster they generated is so inscrutable.
We'll have to see where this all leads, to ver the next few years/decades. Maybe someone will manage to combine "a proper fundamental understanding of trained neural networks," and good results. That'll lead to (perhaps) good theories, to explain the good results.
If "good results" continue to outpace our understanding of wtf the useful NN is up to... It'll have to be studied expirementaly, like the way we study biology.
Ie, we might see CS theory adapt from "mathematical," to "scientific" to in its methods and theories.
The current trajectory seems to be heading here. There is tremendous interest and resources in NNs. As they become more commercially important, interest and resources dedicated to developing them increases. They only need the NNs to work, not to be scriptable.
Scientists are not just going to give up though. They'll study NNs expirementaly as black boxes if that's all they have.
What you're saying is a bit tautological in a way you may not intend.
What the paper describes is those research papers which aim at, that, giving a fundamental understanding of a trained neural network. That the papers are satisfied with "it works" stands in the way of anyone having this fundamental understanding.
It’s questionable how “immature” ML really is. Most methods that get used were initially designed 50+ years ago, with various improvements over time. E.g., neural networks were invented in the 1950s, backprop was introduced in the 80s, architectures like LSTM and CNN in the 90s, etc.
The only thing that’s really new is the amount of computational power at our hands. That has allowed us to shift from relatively simpler methods to more powerful but opaque methods like NNs. They just don’t lend themselves to easy analysis because it’s a lot harder to explain why inputs to these ML systems map to their respective outputs. Hence, attempts at drawing the connection between inputs and outputs become more speculative.
The people who make a paper have to know where the border is for their particular paper. That is, which things in the paper are claims with evidence to back it up - and which things are speculations about what might be an explanation.
That some things are speculative is not such a big deal, as long as it is clearly marked as such. Then someone else can investigate it properly in another paper. Or people can use it in another work, by treating it as an assumption that they can verify whether holds, and then make use of.
To their credit, the authors actually own-up to doing this themselves in various papers. It seems like a way to describe the situation is that neural nets have become such computational monsters that talking about them exactly becomes very difficult with the language opaque and ambiguous.