In this work, we explore the potential of lepton-flavor violating (LFV) Higgs decays as a probe for new physics, specifically through the mediation of an ultralight gauge boson, denoted as $\chi$. Our study bridges the gap between a model generating the LFV interaction $\ell_i \bar{\ell}_j \chi$ at tree level and an effective field theory framework that preserves the light mass of $\chi$. We utilize stringent constraints from recent CMS and ATLAS data on $H \to \ell_i \ell_j$ decays to infer an upper bound on the exotic decay channel $H \to \ell_i \ell_j \chi$. The talk will delve into various kinematic observables, including the lepton energy spectrum, Dalitz plot distribution, and asymmetries in lepton charge and forward-backward distributions, offering a comprehensive perspective on the implications of our findings.