Wood said that the toxicity scoring model was put together by asking internet users to rate internet comments on a scale from “very toxic” to “very healthy”.
He said: “Toxic is defined as a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion.
“The data has been tested and validated by external researchers. We leverage this model to rank social posts by these scores.
“The more training data the model consumes the closer it gets to the average perceived toxicity for any given post.
“Over time, at Vanilla, we will be regularly checking our training models and ensuring that we limit the impact of bias in our data sets in detecting and identifying what is ‘toxic’.”