In 2012, the ATLAS detector recorded over 20 fb-1 of 8 TeV proton-proton collisions produced by the LHC. These were analyzed to search for Higgs boson decays to tau pairs. An excess over background of 4.1 (3.2) standard deviations is observed (expected). For the most recent iteration of this search, the strategy was changed dramatically to take advantage of the power of boosted decision trees (BDT). This new approach resulted in a substantial gain in sensitivity over the previous iteration and was instrumental in taking the expected sensitivity above the threshold of 3 standard deviations from the background-only hypothesis. This talk will describe how the search for the Higgs boson in the di-tau final state was a challenge perfectly suited for machine learning algorithms like the BDT, as well as what the main pitfalls were, how they were avoided, and how the power provided by the technique was fully exploited to yield this remarkable new result.