It's not necessary to ALWAYS reiterate that you're not a racist, or that not every white person is racist. This is a larger issue called "structural racism," which has been perpetuated throughout history and is present in education, politics, public safety, culture, health, and also social relations.
One way to begin to understand this problem is by trying to understand white identity, and why many people who identify as white feel superior to black people. What does it mean to be white? How does being white feel? What privileges does it afford you?
This field of study is called "whiteness," and it begins to explore and answer some of these questions. So, when someone says that "white people are racist," they are referring to whiteness or white identity. It's not necessarily about you as an individual.