Member-only story
Academia & Industry: Pursuits of Novelty vs. Finessing through Reinventing-the-Wheel
Over the past few months, I came across a few PhD students who encountered ongoing projects in the industry and asked,
“This problem has been solved. Why is XXX reinventing the wheel?”
A few years ago, I would have had similar thoughts. After working for a while in the industry where I no longer do pure research, my perspective has changed.
What is “solved”?
The definition of “solved problem” in academia is different from the industry. Solutions in academic publications usually work well under a few assumptions. These assumptions were carefully chosen to make sure the scope covers the common cases while avoiding the rare edge cases (a.k.a. leaving them for future work). Roughly we can say that it works for 90% of the time. The publication systems also give little incentive for researchers to work on the remaining 10%. Oftentimes, paper submissions are rejected due to “incremental” contributions. Also, one of the PhD students’ worst nightmares is learning that somebody just publish something close to their dear dissertation topic. Therefore, researchers tend to focus more on looking for new frontiers and exploring new open areas that can be fully claimed, viewing the explored areas as “solved”.
Industry, on the other hand, adopts these “solutions” to solve problems that are important to business and provide values to the organization. The original ideas are then applied to many new situations, with new edge cases which original assumptions may not hold any more and keep pushing the limits. The remaining 10% work may become blockers that prevent the system from reaching its full potential and has to be addressed. A lot of engineering work are also required to transform research prototype into production software. While it may appear that many companies are working on the same “solved” problem, the problem is not completely solved, and people spend more time on it to expand and finesse the solution from different angles.
“The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.” (a.k.a. Ninety-ninety rule)
