Affirmative action has always been a politically fraught issue. Some polls show a majority of Americans oppose universities using race as a factor to increase diversity, but others indicate that a majority favors affirmative action more generally.

Most people recognize that racism remains a barrier to equal opportunity, but many see the college admissions process as a zero-sum game and want to be sure any remedy does not affect their own or their children’s prospects.

Wherever you fall, the decision of the six Republican-appointed justices last week to eliminate affirmative action in Students for Fair Admissions, Inc. v. Harvard should be troubling. Once again, a mere change in the composition of the court has completely flipped the judicial view of the Constitution and the law.

To promote diversity, Harvard considers “plus factors,” including race, that can tip “a significant percentage of all admitted African Americans and Hispanic applicants” into the class. The University of North Carolina offers some applicants a “‘plus’ based on their race which may be significant in an individual case.”

In University of California Regents v. Bakke 45 years ago, and then again 20 years ago in Grutter v. Bollinger and seven years ago in Fisher v. University of Texas, the Supreme Court permitted universities to do exactly that. The court repeatedly ruled that universities that want their students to obtain “the educational benefits that flow from a racially diverse student body” can consider race as a factor in deciding which applicants to admit, as long as the universities do so in a manner “flexible enough to consider all pertinent elements of diversity in light of the particular qualifications of each applicant.” The Supreme Court found that the universities have a “compelling” interest in doing so that is allowed by the equal protection clause of the 14th Amendment, which applies to public universities, and Title VI of the Civil Rights Act, which applies to private universities that receive government funding.

The current Supreme Court majority does not like those rulings, but its arguments against them are not persuasive. According to Chief Justice John Roberts’s opinion for the court, “the interests they [Harvard and UNC] view as compelling” — such as training future leaders, preparing graduates for an increasingly pluralistic society, better educating students through diversity and producing new knowledge from diverse outlooks — “cannot be subject to meaningful judicial review” and are “not sufficiently coherent.” You might well ask how that happened. Forty-five years ago, and then again 20 years ago, the court not only agreed those interests were “compelling” and sufficiently coherent, but it expressly cited “the Harvard admissions program” as an example of what is acceptable.

Roberts writes that considering race for these purposes amounts to “impermissible racial stereotyp[ing]” that “members of the same racial group…think alike.” But nothing that Harvard or UNC is doing is based on any such assumption, just as affirmative action was not stereotyping 45, 20 and seven years ago when the court found it was permissible.

Roberts says considering race as a factor means applicants are not being considered based on their merit and other qualities. That also does not follow now, any more than it did before.

Roberts faults the Harvard and UNC affirmative action programs for not having a logical endpoint, but that was also true of the programs the court previously upheld. It is due to a larger failure in this country to arrive at a colorblind society in which opportunities are more equal.

It is frustrating, but not surprising, that more time is needed for the transition. From its founding, this country allowed the enslavement of African Americans. As Roberts recognizes, even after the enactment of the 13th and 14th amendments, the Supreme Court and this country “quickly failed to live up to the [Equal Protection] Clause’s core commitments.…The Court played its own role in that ignoble history.”

Justice Clarence Thomas’s concurrence asserts that Harvard’s admissions program likely discriminates against Asian Americans, Jews “and those who are not part of the white elite.” Of course, any such discrimination should not be tolerated. But undoing impermissible discriminatory quotas for these groups does not require the elimination of affirmative action.

The court’s decision also ignores the important principle of stare decisis — literally, “to stand by things decided.” While this court’s willingness to reject its own precedents is by now familiar, that does not make it any less disturbing or undemocratic. Only a month ago, in Allen v. Milligan, Justice Brett Kavanaugh explained that the stare decisis standard for the court to overrule a statutory precedent, such as Title VI, “is comparatively strict.”

Thomas concedes the court’s decision in the Harvard and UNC cases overrules Grutter “for all intents and purposes.” But Roberts appears to try to avoid that obvious conclusion by arguing that Grutter set a 25-year time limit on its decision, and that time limit is about to run out. That too is incorrect. What Grutter actually said is: “The court takes the law school at its word that it would like nothing better than to find a race-neutral admissions formula and will terminate its use of racial preferences as soon as practicable. The court expects that 25 years from now the racial preferences will no longer be necessary to further the interests approved today.”

That “expect[ation]” is very different from a time limit. And it is beyond dispute that the expectation has proved to be overly optimistic. As Kavanaugh acknowledges in his concurrence, “although progress has been made since Bakke and Grutter, racial discrimination still occurs and the effects of past racial discrimination still persist.”

University admission, including at our most elite institutions, makes an enormous difference to opportunity in this society. It is important that African Americans and Hispanics are included. Just look at the composition of the Supreme Court itself — all but one of the Justices attended law school at Harvard or Yale, and the ninth attended Notre Dame.

As the Supreme Court explained in the 1880 decision Ex Parte Virginia, “One great purpose of the Thirteenth and Fourteenth Amendments was to raise the colored race from the condition of inferiority and servitude in which most of them had previously stood into perfect equality of civil rights with all other persons.”

The 14th Amendment should not now be interpreted to do the opposite, by preventing African Americans from achieving equality in education.

Michael J. Dell is a New York lawyer who writes and litigates about constitutional issues.