The black and white radical abolitionists of the North are the most well known figures working for racial justice in America in the nineteenth century. But within that movement, blacks grew disillusioned with the reluctance of radical whites to deal with the problems of black poverty and racism on their doorstep. Abolitionists agitated for the immediate end of slavery in the South, but they did not build schools or other institutions for blacks in the North who experienced severe racism that limited their economic and political opportunities. While researching my book, In the Shadow of Slavery: African-Americans in New York City, 1626-1863, I discovered detailed records of an institution that tried to address this gap. The Colored Orphans Asylum was established in New York City in 1837 by two Quaker women, Anna Shotwell and Mary Murray. They kept the records of the children housed in the orphanage and of the children’s families.

The women who founded the Colored Orphan Asylum were not radical abolitionists, but they made concrete changes to address problems related to black poverty that the radical abolitionists did not. Over twelve hundred children passed through the Colored Orphan Asylum in the decades between its founding and the Civil War. This same group of women also constructed the Colored Home for the care of elderly indigent blacks in New York.

Before the establishment of this institution in 1837, other private orphanages built at the time refused to receive black children. Black orphans were taken in by extended family or neighbors, and when these networks failed, they were sent to the almshouse with adults, where they lived in the disease-plagued basement, without access to education. Although the asylum began as an institution for children who had lost both parents, the children of single mothers formed the majority of the clients. Some of the children were runaway slaves from the South, but most were children born after emancipation in New York and brought to the orphanage by black family members or neighbors who saw it as safe place.

The orphanage also offered some economic independence for the white female charity workers who ran the institution. The asylum served as a source of employment and income for many single women over the years, including the managers themselves. But the women also realized that the orphanage’s existence depended on the support of their black clients. Shotwell and Murray knew that their clients were not just passive recipients and that they would put pressure on the institution to meet their needs. As a result, the goals and practices of the orphanage changed over time in response to their clients’ concerns. The orphanage’s acceptance of the children of single parents was a response to the requests of black adults. Relatives of some of the children negotiated for more education than just the basic skills the women initially offered at the institution. And in response to New York City’s black community, the women hired the black doctor and radical abolitionist James McCune Smith to tend the orphans, replacing the white doctors associated with the American Colonization Society, an organization that many blacks deeply distrusted.

The alliance between the Quaker reformers and the black community fueled the growing hostility of some whites towards blacks in antebellum New York. Many whites greatly feared alliances between blacks and middle-class and elite whites that would lead to greater economic and political success for blacks. With the approach of the Civil War, those fears intensified and took gruesome shape in the Civil War Draft Riots that erupted in July 1863.

Poor whites resented being drafted into the Army while members of the elite could pay for substitutes to take their place. Though many blacks were willing to go South and fight to end slavery, the Army refused to enlist blacks at this time. So, groups of poor whites—German, Irish, and other immigrants—rioted, initially targeting symbols of the draft, like the Draft Office. But they then turned against blacks in general, as representatives of why they had to fight this war and as people whom they believed really were not part of the nation. Over five days of rioting, angry white mobs lynched eleven black men and drove hundreds of blacks out of the city.

By this time, the asylum had grown into a large and very visible institution, near the present site of the New York Public Library on Fifth Avenue. On July 13, 1863, a large group of men and women looted the four-story asylum and burned it to the ground. Miraculously, none of the 233 children there were killed. They escaped to a police station and eventually were sent to the Alms House—the very fate the orphanage originally was designed to prevent.

When the women attempted to rebuild after the riots, white neighboring property owners asked them to leave. They relocated first to 51st Street, and then to 143rd Street between Broadway and Amsterdam, in the heart of present-day Harlem. The asylum continued its work through the Civil War and into the twentieth century, following the trajectory of social welfare practices. As the idea of orphanages fell away, it became a child-welfare and foster care organization and exists today in the Bronx as the Harlem Dowling–West Side Center for Children and Family Services.

The history of this institution is similar to the way I think about community. It doesn’t just happen, and it’s not something you retreat to. It’s something you have to create and recreate. That’s what white reformers and black parents and children did with the orphanage. The institution was created and sustained as both groups continually renegotiated the meaning of asylum and safety for black children.

If this history can inform contemporary conversations and conflicts about race, including the current controversies here at Emory, the connecting thread is the significance of dialogue and negotiation within interracial communities. Shotwell, Murray, McCune Smith, and the asylum’s clients were only at the beginning of a long history of interracial dialogue about race and racism in America. Now, as then, whites must seek to understand blacks’ and other minorities’ definitions and experiences of racism and racial inequality. Currently, many whites seek to define what constitutes racism at Emory and in America by selective listening to “trusted” blacks and minorities, and by undermining minority group statements when they do not conform to whites’ ideas of what constitutes racial inequality. But genuine listening, negotiation, and responsiveness between whites and minorities demand first a willingness by whites to recognize their unequal privilege, and then a relinquishing of their dominance of the discourse on race in this country.