A couple weeks ago I was at the New York Philharmonic. The conductor, critically-acclaimed Alan Gilbert, and the piano soloist, Emanuel Ax, “broke the fourth wall” and explained Schoenberg’s Piano Concerto to the audience before playing it. They described Schoenberg’s 12-tone technique for composing music as: the composer selects a range of 12 notes and must use each note at least once before being allowed to repeat. Gilbert described the 12-tone technique as as an “organizational rule” (“Algorithm!” I thought).
Then Gilbert went on to say (and I wrote this down) “The 12-tone technique has been mis-applied by lesser composers…Great composers are in control of technique.” (“Lesser data scientists!” I thought.)
I took this all as an analogy for using machine learning algorithms. Don’t be incompetent! In the hands of lesser data scientists, the results will be unpleasant–“unpleasant” in the context of Data Science means anywhere from “meaningless” to “disastrous”. The heart of the problem of course is: no one thinks they’re incompetent.