A topological theory of the diabolical points (degeneracies) of quantum magnets is presented. Diabolical points are characterized by their diabolicity index, for which topological sum rules are derived. The paradox of the missing diabolical points for Fe₈ molecular magnets is clarified. A new method is also developed to provide a simple interpretation, in terms of destructive interferences due to the Berry phase, of the complete set of diabolical points found in biaxial systems such as Fe₈.