Search for Higgs bosons decaying to aa in the μμττ final state in pp collisions at s√= 8 TeV with the ATLAS experiment

A search for the decay to a pair of new particles of either the 125 GeV Higgs boson (h) or a second CP-even Higgs boson (H) is presented. The dataset correspods to an integrated luminosity of 20.3 fb−1 of pp collisions at s√= 8 TeV recorded by the ATLAS experiment at the LHC in 2012. The search was done in the context of the next-to-minimal supersymmetric standard model, in which the new particles are the lightest neutral pseudoscalar Higgs bosons (a). One of the two a bosons is required to decay to two muons while the other is required to decay to two τ-leptons. No significant excess is observed above the expected backgrounds in the dimuon invariant mass range from 3.7 GeV to 50 GeV. Upper limits are placed on the production of h→aa relative to the Standard Model gg→h production, assuming no coupling of the a boson to quarks. The most stringent limit is placed at 3.5% for ma= 3.75 GeV. Upper limits are also placed on the production cross section of H→aa from 2.33 pb to 0.72 pb, for fixed ma = 5 GeV with mH ranging from 100 GeV to 500 GeV.