I would unite with anybody to do right and with nobody to do wrong.
Attribution checked
Quothly found supporting evidence for this attribution.
The National Park Service page on Frederick Douglass quotes him saying, “I would unite with anybody to do right and with nobody to do wrong.”
Frederick Douglass National Historic Site (National Park Service)