Empirical explorations of the limitations of the Equality of Opportunity framework