|Year : 2015 | Volume
| Issue : 4 | Page : 208-212
|The impact of Clostridum difficile on surgical rate among ulcerative colitis patients: A systemic review and meta-analysis
Jiang-Chen Peng, Jun Shen, Qi Zhu, Zhi-Hua Ran
Department of Gastroenterology, Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai Institute of Digestive Disease, Shanghai Inflammatory Bowel Disease Research Center, 160# Pu Jian Avenue, 200127 Shanghai, China
Click here for correspondence address and email
|Date of Submission||10-Oct-2014|
|Date of Acceptance||20-Nov-2014|
|Date of Web Publication||29-Jul-2015|
| Abstract|| |
There is growing recognition of the impact of Clostridum difficile infection (CDI) on patients with inflammatory bowel disease. Clostridium difficile infection causes greater morbidity and mortality. This study aimed to evaluate the impact of C. difficile on surgical risk among ulcerative colitis (UC) patients. We searched the following databases: MEDLINE, EMBASE, the Cochrane Central Register of Controlled Trials, ACP Journal Club, DARE, CMR, and HTA. Studies were included if fulfilled the following criteria: (1) Cohort or case–control studies, which involved a comparison group that lacked CDI, (2) Patients were given a primary diagnosis of UC, (3) Comorbidity of CDI was evaluated by enzyme immunoassay of stool for C. difficile toxin A and B or C. difficile stool culture, (4) Studies evaluated surgical rate, and (5) Studies reported an estimate of odds ratio, accompanied by a corresponding measure of uncertainty. Five studies with 2380 patients fulfilled the inclusion criteria. Overall, meta-analysis showed that UC with CDI patients had a significant higher surgical rate than patients with UC alone. (OR=1.76, 95% CI=1.36–2.28). C. difficile infection increased the surgical rate in UC patients. However, results should be interpreted with caution, given the limitations of this stud.
Keywords: C. difficile infection, surgical risk, ulcerative colitis
|How to cite this article:|
Peng JC, Shen J, Zhu Q, Ran ZH. The impact of Clostridum difficile on surgical rate among ulcerative colitis patients: A systemic review and meta-analysis. Saudi J Gastroenterol 2015;21:208-12
|How to cite this URL:|
Peng JC, Shen J, Zhu Q, Ran ZH. The impact of Clostridum difficile on surgical rate among ulcerative colitis patients: A systemic review and meta-analysis. Saudi J Gastroenterol [serial online] 2015 [cited 2022 Jan 21];21:208-12. Available from: https://www.saudijgastro.com/text.asp?2015/21/4/208/161644
There is a growing recognition of the impact of Clostridum difficile infection (CDI) on patients with inflammatory bowel disease. There has been a 2 to 3 fold increase in the proportion of ulcerative colitis (UC) and Crohn’s disease (CD) hospitalizations complicated by C. difficile. Clostridium difficile infection causes greater morbidity and mortality. Although a growing body of knowledge on the epidemiology, pathogenesis, risk factors, and management of CDI has been obtained over the last decade, the increased incidence and severity of CDI continue to pose challenges to the medical community., The expected health care costs due to CDI alone are estimated to be about 3.2 billion dollars per year in the United States.
The past decade has seen a change in the epidemiology of C. difficile infection. Some of the original risk factors such as antibiotic use or health care exposure are no longer considered essential to entertain the diagnosis of C. difficile. There is a greater recognition of community-acquired C. difficile infection, where a significant proportion has a positive test result within 2 days of hospitalization.
Recently, debate in the literature has centered on whether infection with C. difficile is related to the adverse clinical outcomes among IBD patients; some studies reported that IBD patients with C. difficile had worse clinical outcomes than those without C. difficile infection. However, other studies found that CDI was not associated with any adverse clinical outcomes., Understanding this potential risk for IBD is important. as it will allow clinicians to evaluate the cost-effectiveness of medical versus non-medical interventions in management strategies and plan health care resource utilization. Patients with UC appeared to be at a higher risk for CDI than CD. Therefore, we conducted a meta-analysis and systemic review to know whether CDI is related to increased colectomy among UC patients.
| Materials and Methods|| |
This review was performed according to the standard guidelines for meta-analyses and systematic reviews of observational studies. To find relevant articles for this review, we searched the following databases (from inception to December 2013): MEDLINE, EMBASE, the Cochrane Central Register of Controlled Trials, ACP Journal Club, DARE, CMR, and HTA. The search strategy used free-text words and MeSH terms to increase the sensitivity of the search. The following search terms were used: Inflammatory bowel disease, UC, C. difficile, clinical outcome, colectomy, and surgery. Boolean operators (AND, OR, NOT) were used to narrow and widen the search results. A comprehensive search of reference lists of all review articles and original studies retrieved by this method was performed to identify additional reports. Furthermore, we hand searched abstracts of major gastroenterological meetings, such as the Digestive Disease Week of the American Gastroenterological Association and the World Congress of Gastroenterology. No language restrictions were made. Authors of some identified trials were asked whether they knew of additional studies, including unpublished randomized ones.
Inclusion and exclusion criteria
For inclusion in the systematic review, a study had to meet the following criteria established by the study team: (1) Cohort or case–control studies involved a comparison group that lacked CDI, (2) patients were given a primary diagnosis of UC, (3) diagnosis of CDI was evaluated by enzyme immunoassay of stool for C. difficile toxin A and B or C. difficile stool culture, (4) studies evaluated surgical rate, and (5) studies reported an estimate of relative risk or odds ratio, accompanied by a corresponding measure of uncertainty [ie, 95% confidence interval (CI), standard error, variance, or P value].
Studies were excluded if (1) Age was younger than 18 years, (2) patients had no known history of IBD, (3) outcome of interest was not reported, or (4) incomplete data.
To reduce reporting bias and error in data collection, all papers were examined independently for eligibility by two reviewers (Peng and Shen). Disagreement was resolved by consulting a third reviewer (Ran). Standardized data extraction form created by the study team was used. This form included the authors, location, year of publication, study design, number of CDI-UC patients, number of non-CDI-UC patients, outcome, and the Newcastle–Ottawa Scale.
The primary outcome of this analysis was the odds ratio (OR) of surgical rate in CDI-UC versus controls. We calculated the OR with a 95% confidence interval (CI) based on a fixed-effects model using the methods of DerSimonian and Laird. Heterogeneity between the studies was assessed by Chi-square test and the I2-statistic. P values <0.05 were considered statistically significant. If significant heterogeneity exists, it would be inappropriate to combine the data for further analysis using a fixed-effects model, whereas the random model was used for calculations. Publication bias was assessed by constructing a funnel plot. Any heterogeneity identified would prompt subgroup analysis in an attempt to explain these findings. Statistical analysis was performed with the software REVMAN X6 from the Cochrane Collaboration.
| Results|| |
Characteristics and description of the studies
Our initial search strategy yielded 124 potential articles for inclusion. After detailed analysis of selected articles, 24 articles were reviewed in detail. Subsequently, 19 articles,, did not meet the inclusion criteria. The reasons for exclusion included the following: five articles,, were review, four studies,,, focused on children, six studies,,,,, did not report the outcome of interest, one article was case report, and three studies,, did not give complete data. Therefore, five studies, with 2380 patients fulfilled the inclusion criteria [Figure 1].
The characteristics of the included studies are listed in [Table 1]. Three studies were performed in the United States, and the other two were from Japan and Canada. The patients in the study group were diagnosed as CDI and UC, whereas the patients in the control group were diagnosed as UC alone. The diagnosis of CDI in most studies was according to the following criteria: enzyme immunoassay of stool for C. difficile toxin A and B, C. difficile stool culture, pathological evidence of C. difficile. Two studies, measured 1-year risks of colectomy between CDI-UC patients and UC patients. One study measured a 5-year outcome of CDI-UC patients compared with UC patients. One study performed a 3-month impact of C. difficile on UC patients. The other one focused on the 3-year impact of C. difficile on acute UC. Three studies,, reported increased surgical risks among CDI-UC patients, whereas another two, showed that there was no significant difference between CDI-UC and UC patients.
Methodological quality of included studies was evaluated using the Newcastle–Ottawa scale. It uses a “star” rating system to judge quality on the basis of three aspects of the study: Selection of study groups, comparability of study groups, and assessment of the exposure. This scale awards a maximum of 9 stars to each study: up to 4 for selection of participants, 2 for comparability of participants on the basis of the design or analysis, and 3 for ascertainment of exposure. We assigned scores of 0–3, 4–6, and 7–9 for low, moderate and high quality studies, respectively [Table 1]. All included studies had scores ≥7, which were considered as high quality.
Impact of C. difficile on surgical rate among patients with IBD
Overall, a total of 41% of CDI-UC patients and 31.6% of UC patients had colectomy. The OR of surgical rate in UC-CDI patients compared with controls was 1.76 (95% CI = 1.36–2.28). There was no significant heterogeneity among the examined studies, (P = 0.09) [Figure 2] so a fixed effect model was used for assessment of the pooled OR. The result showed that C. difficile infection was associated with increased risks of colectomy among UC patients.
|Figure 2: Risk of surgery between ulcerative colitis (UC)-Clostridium difficile infection patients and UC patients|
Click here to view
Kariv et al. also reported that subjects who did not use 5-ASA were 3.3 times more likely to have UC-related surgery within 3-months of C. difficile testing than those treated with 5-ASA, (95% CI 1.2–9.4, P = 0.03), which to some extent explained the role of immunosuppressants on clinical outcome in patients who were infected with C. difficile. Jodorkovsky et al. found that UC patients with C. difficile infection had significantly higher rates of colectomy in the long-term compared with those not infected, regardless of age and gender, which suggested an association between C. difficile infection and a need for eventual colectomy.
[Figure 3] shows a funnel plot of studies included in this meta-analysis. On visual inspection, the funnel plot was symmetrical in distribution. So, there was no significant publication bias in this analysis.
| Discussion|| |
CDI has become the leading identifiable infective cause of antibiotic-associated diarrhea in general population. Studies have shown that there was increased risk of CDI with use of antibiotics disrupting the bacterial flora, intensive care or prolonged hospital stay and more recently IBD., Multiple studies from tertiary-care institutions as well as large nationwide inpatient databases have demonstrated an increase in the incidence and severity of CDI in patients with IBD when compared with general population. In a study of risk factors for CDI in IBD patients, maintenance immunomodulator use and colonic involvement were independently associated with the risk of CDI. In addition, 61% of IBD patients who developed CDI had antibiotic exposure up to 2 months prior to development of CDI. A subsequent study also reported that increasing age and colonic involvement were independently associated with CDI in IBD. One study reported that the use of antibiotics up to 30 days prior to C. difficile testing, was independently associated with the development of CDI.
Pathogenicity of C. difficile infection begins with colonization of this Gram-positive anaerobic bacterium. Disruption of the normal intestinal flora allowing C. difficile to flourish is usually associated with antibiotic use and hospitalization. Inflammation may be an additional risk factor that disrupts the normal flora. C. difficile toxins were speculated to complicate IBD and contributed to relapse of IBD in some patients. With increasing incidence of CDI, superimposed CDI in patients with IBD has been increasingly reported. CDI can even present with enteritis, particularly in patients with IBD who had undergone bowel-diverting surgery or colectomy or as chronic antibiotic refractory pouchitis in patients with ileal pouch anal anastomosis for underlying IBD.
In our study, we evaluated the association between CDI and surgical rate among UC patients. We found UC-CDI patients had statistically significant higher risks of surgery when compared with UC patients (OR = 1.76, 95% CI = 1.36–2.28). There was no statistical significance of heterogeneity among the included studies (P = 0.09).
The explanation could be that UC patients were more susceptible to CDI. In recent years, the reported incidence of CDI has increased, and the OR of acquiring CDI in UC patients was found to be four times as many as in non-IBD patients., The exact reason for this phenomenon was unclear. The universal colonic involvement in UC compared with less frequent involvement in CD may contribute to a higher susceptibility to C. difficile infection. Second, C. difficile spores and vegetative organisms in some patients could theoretically alter the natural history of UC by activating abnormal mucosal immune response, or CDI might be a sign reflecting an aggressive form of UC course, which resulted in more ER visits, more hospitalization, escalation of medical treatment and ultimately resulting in colectomy., Use of immunosuppressive therapy, in addition to antibiotics, to treat C. difficile in the setting of UC may have also increased the duration or severity of colitis in many individuals. Besides, sicker patients might get CDI. This might play a major role in colectomy because, on endoscopy, it was found that these patients had a significantly active or severe disease when compared with patients without CDI.
There were also some limitations to this analysis. First, the number of included studies was small. Second, the unadjusted differences in severity of underlying IBD, comorbidity burden, or antibiotic use between infected and uninfected patients may influence the observed association in this analysis. Possibly, patients with severe manifestations of IBD and more comorbidities were at a greater risk of both acquiring C. difficile infection and worse clinical outcome. However, such information was not available in most of the included studies. Although one study evaluated the role of immunosuppressants on the clinical outcome of UC-CDI patients, more studies are needed to classify whether the use of biologics and immunosuppressants could affect clinical outcome in patients infected with C. difficile. Furthermore, C. difficile exposure status may have been misclassified in a proportion of patients, as a result of both misreporting on hospital discharge abstracts and inadequate stool testing for C. difficile. No studies have evaluated the diagnostic accuracy of CDI among IBD patients. Last but not the least, most of the studies were at an academic center with multiple primary physicians, treatment strategy was subjective and unlikely to be uniform. Specifically, decisions about inpatient therapy and need for surgery is highly individualized. These factors may affect the role of C. difficile in UC patients.
In summary, our analysis suggested an association between C. difficile infection and surgical risks among UC patients. Although no significant heterogeneity was found among the included studies, the number of included studies was small. Therefore, the result should be interpreted with caution and further clinical studies investigating the effect of C. difficile infection on UC patients are warranted. In our analysis, we found CDI increased the surgical rate in UC patients. If it is found that C. difficile does indeed exacerbate UC, this will have profound influence not only on the way we approach C. difficile testing, but also on the way we approach the treatment of UC patients complicated with C. difficile.
| References|| |
Ananthakrishnan AN, Issa M, Binion DG. Clostridium difficile
and inflammatory bowel disease. Gastroenterol Clin North Am 2009;38:711-28.
Ricciardi R, Ogilvie JW Jr, Roberts PL, Marcello PW, Concannon TW, Baxter NN. Epidemiology of Clostridium difficile
colitis in hospitalized patients with inflammatory bowel diseases. Dis Colon Rectum 2009;52:40-5.
Bartlett JG, Chang TW, Gurwith M, Gorbach SL, Onderdonk AB. Antibiotic-associated pseudomembranous colitis due to toxin-producing clostridia. N Engl J Med 1978;298:531-4.
McDonald LC, Killgore GE, Thompson A, Owens RC Jr, Kazakova SV, Sambol SP, et al
. An epidemic, toxin gene-variant strain of Clostridium difficile
. N Engl J Med 2005;353:2433-41.
Nguyen GC, Kaplan GG, Harris ML, Brant SR. A national survey of the prevalence and impact of Clostridium difficile
infection among hospitalized inflammatory bowel disease patients. Am J Gastroenterol 2008;103:1443-50.
O’Brien JA, Lahue BJ, Caro JJ, Davidson DM. The emerging infectious challenge of clostridium difficile
-associated disease in Massachusetts hospitals: Clinical and economic consequences. Infect Control Hosp Epidemiol 2007;28:1219-27.
Rodemann JF, Dubberke ER, Reske KA, Seo da H, Stone CD. Incidence of Clostridium difficile
infection in inflammatory bowel disease. Clin Gastroenterol Hepatol 2007;5:339-44.
Ananthakrishnan AN, McGinley EL, Saeian K, Binion DG. Temporal trends in disease outcomes related to Clostridium difficile
infection in patients with inflammatory bowel disease. Inflamm Bowel Dis 2011;17:976-83.
Navaneethan U, Mukewar S, Venkatesh PG, Lopez R, Shen B. Clostridium difficile
infection is associated with worse long term outcome in patients with ulcerative colitis. J Crohns Colitis 2012;6:330-6.
Murthy SK, Steinhart AH, Tinmouth J, Austin PC, Daneman N, Nguyen GC. Impact of Clostridium difficile
colitis on 5-year health outcomes in patients with ulcerative colitis. Aliment Pharmacol Ther 2012;36:1032-9.
Kariv R, Navaneethan U, Venkatesh PG, Lopez R, Shen B. Impact of Clostridium difficile
infection in patients with ulcerative colitis. J Crohns Colitis 2011;5:34-40.
Kaneko T, Matsuda R, Taguri M, Inamori M, Ogura A, Miyajima E, et al
. Clostridium difficile
infection in patients with ulcerative colitis: Investigations of risk factors and efficacy of antibiotics for steroid refractory patients. Clin Res Hepatol Gastroenterol 2011;35:315-20.
Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al
. Meta-analysis of observational studies in epidemiology: A proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008-12.
DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:177-88.
Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002;21:1539-58.
Reddy SS, Brandt LJ. Clostridium difficile
Infection and Inflammatory Bowel Disease. J Clin Gastroenterol 2013;47:666-71.
Sammons JS, Toltzis P, Zaoutis TE. Clostridium difficile
Infection in children. JAMA Pediatr 2013;167:567-73.
Campbell R, Dean B, Nathanson B, Haidar T, Strauss M, Thomas S. Length of stay and hospital costs among high-risk patients with hospital-origin Clostridium difficile
-associated diarrhea. J Med Econ 2013;16:440-8.
Hsu CH, Jeng YM, Ni YH. Clostridium difficile
infection in a patient with Crohn disease. J Formos Med Assoc 2012;111:347-9.
Berg AM, Kelly CP, Farraye FA. Clostridium difficile
infection in the inflammatory bowel disease patient. Inflamm Bowel Dis 2013;19:194-204.
Ananthakrishnan AN, Guzman-Perez R, Gainer V, Cai T, Churchill S, Kohane I, et al
. Predictors of severe outcomes associated with Clostridium difficile
infection in patients with inflammatory bowel disease. Aliment Pharmacol Ther 2012;35:789-95.
Mezoff E, Mann EA, Hart KW, Lindsell CJ, Cohen MB. Clostridium difficile
infection and treatment in the pediatric inflammatory bowel disease population. J Pediatr Gastroenterol Nutr 2011;52:437-41.
Ott C, Girlich C, Klebl F, Plentz A, Iesalnieks I, Scholmerich J, et al
. Low risk of Clostridium difficile
infections in hospitalized patients with inflammatory bowel disease in a German tertiary referral center. Digestion 2011;84:187-92.
Sinh P, Barrett TA, Yun L. Clostridium difficile
Infection and Inflammatory Bowel Disease: A Review. Gastroenterol Res Pract 2011;2011:136064.
Ananthakrishnan AN, Issa M, Binion DG. Clostridium difficile
and inflammatory bowel disease. Med Clin North Am 2010;94:135-53.
Musa S, Thomson S, Cowan M, Rahman T. Clostridium difficile
infection and inflammatory bowel disease. Scand J Gastroenterol 2010;45:261-72.
Navaneethan U, Venkatesh PG, Shen B. Clostridium difficile
infection and inflammatory bowel disease: Understanding the evolving relationship. World J Gastroenterol 2010;16:4892-904.
Sonnenberg A. Similar geographic variations of mortality and hospitalization associated with IBD and Clostridium difficile
colitis. Inflamm Bowel Dis 2010;16:487-93.
Wultanska D, Banaszkiewicz A, Radzikowski A, Obuch-Woszczatynski P, Mlynarczyk G, Brazier JS, et al
. Clostridium difficile
infection in Polish pediatric outpatients with inflammatory bowel disease. Eur J Clin Microbiol Infect Dis 2010;29:1265-70.
Dial S, Kezouh A, Dascal A, Barkun A, Suissa S. Patterns of antibiotic use and risk of hospital admission because of Clostridium difficile
infection. CMAJ 2008;179:767-72.
Pascarella F, Martinelli M, Miele E, Del Pezzo M, Roscetto E, Staiano A. Impact of Clostridium difficile
infection on pediatric inflammatory bowel disease. J Pediatr 2009;154:854-8.
Ananthakrishnan AN, McGinley EL. Infection-related hospitalizations are associated with increased mortality in patients with inflammatory bowel diseases. J Crohns Colitis 2013;7:107-12.
Issa M, Vijayapal A, Graham MB, Beaulieu DB, Otterson MF, Lundeen S, et al
. Impact of Clostridium difficile
on inflammatory bowel disease. Clin Gastroenterol Hepatol 2007;5:345-51.
Jen MH, Saxena S, Bottle A, Aylin P, Pollok RC. Increased health burden associated with Clostridium difficile
diarrhoea in patients with inflammatory bowel disease. Aliment Pharmacol Ther 2011;33:1322-31.
Ananthakrishnan AN, McGinley EL, Binion DG. Excess hospitalisation burden associated with Clostridium difficile
in patients with inflammatory bowel disease. Gut 2008;57:205-10.
Jodorkovsky D, Young Y, Abreu MT. Clinical outcomes of patients with ulcerative colitis and co-existing Clostridium difficile
infection. Dig Dis Sci 2010;55:415-20.
Navaneethan U, Giannella RA. Thinking beyond the colon-small bowel involvement in Clostridium difficile
infection. Gut Pathog 2009;1:7.
Asha NJ, Tompkins D, Wilcox MH. Comparative analysis of prevalence, risk factors, and molecular epidemiology of antibiotic-associated diarrhea due to Clostridium difficile
, Clostridium perfringens, and Staphylococcus aureus
. J Clin Microbiol 2006;44:2785-91.
LaMont JT, Trnka YM. Therapeutic implications of Clostridium difficile
toxin during relapse of chronic inflammatory bowel disease. Lancet 1980;1:381-3.
Shen BO, Jiang ZD, Fazio VW, Remzi FH, Rodriguez L, Bennett AE, et al
. Clostridium difficile
infection in patients with ileal pouch-anal anastomosis. Clin Gastroenterol Hepatol 2008;6:782-8.
Lundeen SJ, Otterson MF, Binion DG, Carman ET, Peppard WJ. Clostridium difficile
enteritis: An early postoperative complication in inflammatory bowel disease patients after colectomy. J Gastrointest Surg 2007;11:138-42.
Ben-Horin S, Margalit M, Bossuyt P, Maul J, Shapira Y, Bojic D, et al
. Combination immunomodulator and antibiotic treatment in patients with inflammatory bowel disease and Clostridium difficile
infection. Clin Gastroenterol Hepatol 2009;7:981-7.
Department of Gastroenterology, Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai Institute of Digestive Disease, Shanghai Inflammatory Bowel Disease Research Center, 160# Pu Jian Avenue, 200127 Shanghai
Source of Support: This work was supported by the National Natural Science Foundation of China (No. 81000161 and No.81170362) (No. 81000161 and No.81170362), Conflict of Interest: None declared.
[Figure 1], [Figure 2], [Figure 3]
|This article has been cited by|
||Intestinal Microbiota and Perspectives of the Use of Meta-Analysis for Comparison of Ulcerative Colitis Studies
| ||Ivan Kushkevych, Kristýna Martínková, Monika Vítezová, Simon K.-M. R. Rittmann |
| ||Journal of Clinical Medicine. 2021; 10(3): 462 |
|[Pubmed] | [DOI]|
||Review of updated clinical practice guidelines of the Infectious Diseases Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA) for Clostridium difficile infection in adults and children (2017)
| ||Roman S. Kozlov, Yu.A. Shelygin, Alexander V. Veselov, Andrey V. Dekhnich, N.A. Zubareva, O.N. Ershova, V.V. Kulabukhov, Dmitriy A. Popov, G.K. Reshedko, Sergey B. Yakushin |
| ||Clinical Microbiology and Antimicrobial Chemotherapy. 2018; 20(2): 76 |
|[Pubmed] | [DOI]|
||Clostridium difficile infection in hospitalized patients with inflammatory bowel disease
| ||Nitsan Maharshak, Idan Barzilay, Hasya Zinger, Keren Hod, Iris Dotan |
| ||Medicine. 2018; 97(5): e9772 |
|[Pubmed] | [DOI]|
| Article Access Statistics|
| Viewed||2623 |
| Printed||53 |
| Emailed||0 |
| PDF Downloaded||451 |
| Comments ||[Add] |
| Cited by others ||3 |