BackgroundThe scoring system for bile cytology (SSBC) aims to improve bile cytology diagnostic accuracy. Here, the practicality of SSBC was verified by multiple cytotechnologists.MethodsBile cytological specimens were evaluated by 24 cytotechnologists using SSBC. The samples were assessed before using the SSBC (first‐time assessment) according to three categories: benign, indeterminate, and malignant. A first scoring evaluation (FSE) was then performed using SSBC; each item in the scoring system was classified as present or absent. After distributing an instruction sheet with diagnostic criteria, a second scoring evaluation (SSE) was performed using SSBC. Each method was evaluated using diagnostic accuracy and interobserver and intraobserver agreement.ResultsSeveral samples were assessed as indeterminate in the first‐time assessment. Although the specificity of the SSE improved, the sensitivity and accuracy decreased compared with those of the FSE. The overall interobserver agreement was fair for all parameters, including abnormal chromatin, irregular internuclear distances, irregularly overlapped nuclei, irregular cluster margins, and final evaluation in the FSE and SSE. The final evaluation by histological type exhibited slight agreement for well‐differentiated tubular adenocarcinoma and almost perfect agreement for poorly differentiated tubular adenocarcinoma in the FSE and SSE. For moderately differentiated tubular adenocarcinoma, agreement was moderate in the FSE and fair in the SSE. For cholangitis, a slight agreement was observed in the FSE, which improved to fair in the SSE.ConclusionsAlthough the SSBC is expected to improve specificity, there exists ambiguity regarding SSBC criteria and interindividual assessment differences. Therefore, the objective assessment method should be revised.