In other cases, Israel was called to keep their ill-advised covenents such as the one made with the Gibeonites. Why would it be preferred to break marriage covenents in this case? They even make a covenent with God to put away their foreign wives, although it would seem the prior covenent of marriage should hold. Were the foreign "wives" not really wives, but concubines of some sort? Or was this a special dispensation for preserving the purity of the newly-rebuilt city?
The law clearly states for most of the nations mentioned in Ezra 9 and 10 that the marriage is illegal (compare the law in Deuteronomy 7:3 to Ezra's own description of the transgression in Ezra 9:12). The response in Ezra 10:3 states explicitly that they will break the marriage covenants "by the law". The marriages never had a legal footing, and are therefore void ab initio.
This is not the same as the covenant made with the Gibeonites, which was not illegal, but on the contrary, legally mandatory. Hebrew law also holds that before conquering a city you must make a diplomatic peace offering (Deuteronomy 20:10). In this case the Gibeonites, despite going about it in a deceptive way because they were unfamiliar with the law, essentially accepted the peace offering under the terms given, making the covenant perfectly legal, and not at all ill-advised.