The small intestinal mucosa-associated microbiota (MAM) can potentially impact the etiology of primary biliary cholangitis (PBC). Herein, we investigate the MAM profile to determine its association with liver pathology in patients with PBC. Thirty-four patients with PBC and 21 healthy controls who underwent colonoscopy at our hospital were enrolled in our study. We performed 16S ribosomal RNA gene sequencing of MAM samples obtained from the mucosa of the terminal ileum and examined the relationship between the abundance of ileal MAM and chronic nonsuppurative destructive cholangitis using liver specimens from patients with PBC. There was a significant reduction in microbial diversity within individuals with PBC (
P
 = 0.039). Dysbiosis of ileal MAM was observed in patients with PBC, with a characteristic overgrowth of
Sphingomonadaceae
and
Pseudomonas
. Multivariate analysis showed that the overgrowth of
Sphingomonadaceae
and
Pseudomonas
is an independent association factor for PBC (
P
 = 0.0429,
P
 = 0.026). Moreover, the abundance of
Sphingomonadaceae
was associated with chronic nonsuppurative destructive cholangitis in PBC (
P
 = 0.00981). The overgrowth of
Sphingomonadaceae
and
Pseudomonas
in ileal MAM was found in patients with PBC.
Sphingomonadaceae
may be associated with the pathological development of PBC.