A model with fermion and scalar fields charged under a Peccei-Queen~(PQ) symmetry is proposed. The PQ charges are chosen in such a way that they can reproduce mass matrices with five texture zeros, {which can generate} the fermion masses, the CKM matrix, and the PMNS matrix of the Standard Model~(SM). To obtain this result, at least 4~Higgs doublets are needed. As we will see in the manuscript this is a highly non-trivial result since the texture zeros of the mass matrices impose a large number of restrictions. This model shows a route to understand the different scales of the SM by extending it with a multi-Higgs sector and an additional PQ symmetry. Since the PQ charges are not universal, the model presents flavor-changing neutral currents~(FCNC) at the tree level, a feature that constitutes the main source of restrictions on the parameter space. We report the allowed regions by lepton decays and compare them with those coming from the semileptonic decays K±⟶πν¯ν. We also show the excluded regions and the projected bounds of future experiments for the axion-photon coupling as a function of the axion mass and compare it with the parameter space of our model.