Harmful AI content definitions vary by jurisdiction but coalesce around four categories: safety risks (self-harm, violence, illegal activity), fairness harms (discrimination, bias-driven denial of service), privacy violations (unauthorized data collection, unauthorized synthetic content), and misinformation (false claims presented as fact). Washington’s HB 2225 defines harm narrowly: chatbots cannot “encourage or provide information on suicide, self-harm, or eating disorders.” This is behavior-based—it’s not about having information, but about the AI system’s intent to encourage harm. Oklahoma’s SB 1521 uses a broader standard: companies violate law if building systems with “reckless disregard” that users could solicit minors into “sexual conduct simulation, violence, or self-harm.” This is about foreseeable misuse, not actual harm.
The EU AI Act takes a risks-based approach: harm includes discrimination, manipulation (e.g., exploiting cognitive vulnerabilities), surveillance overreach, and autonomy violations. This broader definition recognizes that harm isn’t always obvious—a hiring system that learns to discriminate against protected classes does harm even if no individual job seeker sues. The framework assumes that without regulation, AI systems will amplify existing biases and create new mechanisms for harm.
For developers, this definitional fragmentation creates compliance complexity. You can’t have one “harm detection” layer; you need jurisdiction-specific classifiers. If your system serves Washington users, you need self-harm detection. For EU users, you need discrimination detection. For Oklahoma, age-gating prevents minors from seeing potentially harmful content. Using Milvus, you can implement this by partitioning your content embeddings by harm category: store self-harm content vectors in a restricted partition, discrimination-risk vectors in another, age-inappropriate content in a third. Then, at query time, filter which partitions are searchable based on user jurisdiction and age. This architecture makes compliance auditable—regulators see that you’ve separated harmful content and restricted access by user profile. For open-source setups, metadata fields in Milvus become compliance proof: tag embeddings with harm categories and enforcement rules, creating a machine-readable compliance policy.