Many of us are not meant to consume milk.
Milk by itself, theorectically, is not a bad staple product, though it's not a superfood. Issues are from somewhere else usually.
IIRC, well more than 50% of adults globally cannot really digest lactose due to reduced lactase production in adulthood, IIRC it's about 70%. Not that you cannot drink milk, but that you cannot drink that much before it causes issues.
Except breast milk