It would be tough to find anyone whose public standing has seesawed the way former FBI director James Comey’s has over the last five years. After taking up leadership of the bureau with a near saint-like aura in 2013, Comey became liberal enemy number one during the 2016 presidential campaign for his investigation of Hillary Clinton’s email server — and then earned conservative enmity for deciding not to charge her. His standing fell to a new low after November 2016 when he was widely derided by liberals, but cheered by conservatives, for supposedly tipping the election to Trump, only to rebound in the eyes of those same partisan detractors once Trump fired him last year. With a new book out and a full schedule of media appearances, Comey’s image is once again being rehabilitated, though some liberals continue to hold a grudge against him over the email shenanigans.

But despite what some of his detractors will tell you, the worst blemishes on Comey’s record have nothing to do with Hillary Clinton. A few high-profile acts of resistance to some Bush-era excesses have papered over his much longer record as a privacy-shredding extremist who has repeatedly advocated trampling basic rights for the sake of national security. Despite his much-hyped acknowledgement of the bureau’s past misdeeds, Comey’s FBI continued its long-standing pattern of racial profiling and criminalizing lawful activism. And completely forgotten today is Comey’s record in the private sector, where he did the political bidding of his corporate employers.

Until the 2016 debacle, Comey always shrewdly managed his public image, aided by an army of journalists who readily repeated favorable talking points. Whether it will work again remains to be seen.