| Abstract|| |
Background: Ponseti method is the gold standard treatment for idiopathic congenital clubfoot. Pirani clubfoot severity score is a vital tool in assessing treatment of clubfoot. This study determines whether, after a short training in the Pirani scoring, a nonmedical personnel can be as accurate as a doctor in assessing the degree of deformity in clubfoot. Materials and Methods: This was a prospective observational study from January to September 2016. Pirani scoring of all children ≤6 months was done by the counsellor, an orthopedic resident, and a consultant separately. All the three members of the team were blinded of the other's score. The Pirani scoring of the consultant was taken as the most correct. The data were analyzed for interobserver reliability using the kappa statistic and point-by-point interobserver agreements. Results: One hundred and fifteen clubfeet in 75 children (48 males and 27 females) with an average age of 96 days were included in the study. Differences between the means of scores for each severity component of the deformity including the sum of midfoot scores, hindfoot scores, and total foot scores were <0.1. There was fair-to-substantial interobserver reliability of all the subcomponents when scores from the three independent observers were analyzed. Conclusion: Our study successfully demonstrates that after a short training, Pirani score can be successfully used in assessing clubfoot severity by a nonmedical counsellor.
Keywords: Clubfoot, counsellor, interobserver reliability, Pirani score
MeSH terms: Ankle joint, clubfoot, congenital, reliability and validity
|How to cite this article:|
Sharma P, Verma R, Gaur S. Interobserver reliability of pirani clubfoot severity scoring between an orthopedic surgeon, a resident doctor, and a nonmedical counsellor at a clubfoot clinic. Indian J Orthop 2018;52:645-50
|How to cite this URL:|
Sharma P, Verma R, Gaur S. Interobserver reliability of pirani clubfoot severity scoring between an orthopedic surgeon, a resident doctor, and a nonmedical counsellor at a clubfoot clinic. Indian J Orthop [serial online] 2018 [cited 2019 May 23];52:645-50. Available from: http://www.ijoonline.com/text.asp?2018/52/6/645/244970
| Introduction|| |
Ponseti method of manipulation and serial plaster casting is the gold standard treatment for idiopathic congenital clubfoot., It is a simple and cost-effective method which can be executed by any trained medical or paramedical personnel. It is particularly suitable for health-care settings with scarce resources and limited professionals.
Our institute is a tertiary referral center for patients from all across the region. We run a clubfoot specialty clinic at our institute which operates twice a week. Our clubfoot clinic team which comprises an orthopedic consultant, two postgraduate students on a 6-monthly rotation, and a nonmedical counsellor caters to almost 50 patients in a month. All our team members form an integral and essential part of the clinic. In an effort to streamline the working of our clinic and increase the number of patients to whom we can efficiently cater, we trained our counsellors in assessing the severity of clubfoot by Pirani scoring.
Pirani scoring is a vital tool for the initial assessment and followup of clubfoot patients., The counsellor is a key figure in managing the clubfoot program; from counseling the parents to carrying out field visits in assigned communities, they extend services in remote health centers and district hospitals where trained human power may be lacking. This study was carried out with the purpose of assessing the efficiency of a trained counsellor in Pirani scoring so that their services can be taken more efficiently in situations where trained medical personnel are not readily available. There are studies comparing the reliability of Pirani scoring by paramedical personnel,, but no such study has been undertaken to review the reliability of a nonmedical professional.
This study determines whether, after a short training in the Pirani scoring, a nonmedical personnel can be as accurate as a doctor in assessing the degree of deformity in clubfoot.
| Materials and Methods|| |
We conducted a prospective observational study at our clubfoot clinic from January to September 2016. All idiopathic clubfoot children with age ≤6 months were included in the study. Syndromic, neurogenic, neglected, and clubfoot children of >6 months' age were excluded from the study.
A written informed consent was taken from the parents of all children enrolled in the study, in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments.
Pirani clubfoot severity score
Pirani et al. devised a scoring system for clubfoot based on clinical signs of contracture. The Pirani scoring system includes six clinical signs; three in the midfoot and three in hindfoot [Table 1].
Each clinical sign is scored either 0, 0.5, or 1 depending on its severity. Thus, each foot can receive a HFS between 0 and 3, a MFS between 0 and 3, and a total score (TS) between 0 and 6.
Severity scoring of all children included in the study was first done by the counsellor using the Pirani method, along with their regular record keeping and counseling sessions. Thereafter, the child was sent to the consultant in outpatient department where a regular checkup of the feet was done. The consultant did his independent Pirani scoring on the patient. Finally, at the time of casting, the resident doctor in the clubfoot team did the Pirani scoring before applying the corrective casts. All the three members of the team were blinded of the other's score. The Pirani score of the consultant was taken as the gold standard.
The data were analyzed for interobserver reliability using the kappa statistic and point-by-point interobserver agreement. Kappa statistic is a chance-corrected measure of agreement for particular data. It compares the observed agreement with the level of agreement expected by chance alone. The maximum value of 1.0 means that every assessor agrees on the point. A value of 0 indicates no more agreement than expected by chance alone. The kappa statistic for interobserver reliability (strength of agreement) was judged, values ≤0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement.
| Results|| |
Pirani severity score of 115 clubfeet in 75 children (48 males and 27 females) were measured independently by three different observers at the clubfoot clinic. The average age of children was 96 days with a range from 12 days to 6 months. Difference between the mean of scores for each severity component of the deformity including the sum of MFS, HFS, and total foot scores was <0.1 [Table 2].
The scores were analyzed by the kappa statistic for reliability between any two independent observers [Table 3]; interobserver reliability among all the three observers combined was also analyzed [Table 4]. PC had the highest kappa value (0.763) and EH had the lowest value (0.493) when the scores of the consultant and the counsellor were compared. CLB showed substantial agreement with a kappa value of 0.691 and EH had moderate agreement with a score of 0.468, between consultant and residents. When the resident and the counsellor scores were analyzed, EH had a perfect agreement, whereas LHT had the lowest score and had only fair agreement. There was fair-to-substantial interobserver reliability of all the subcomponents when scores from the three independent observers were analyzed together.
|Table 3: Agreement between consultant, counsellor, and resident in assessment of various parameters|
Click here to view
|Table 4: Overall agreement between three raters (i.e., consultant, counsellor, and resident) in assessing different parameters|
Click here to view
Percentages of point-by-point agreement between consultant and the resident doctor had a mean value of 84.9%. The mean percentage was 83.8% between consultant and counsellor whereas it was 85.6% between resident and the counsellor [Table 5].
|Table 5: Percentage of point-to-point agreement for each Pirani score parameter between both observers|
Click here to view
| Discussion|| |
Congenital talipes equinovarus or clubfoot is one of the most common congenital orthopedic conditions with an incidence of approximately 1:1000 live births.,, Grading the severity of deformity and assessing its response to treatment, form an important component in clubfoot management. Many classification systems have been proposed,,,,,,, for this purpose. The reliability of these systems has been evaluated in several studies. Wainwright et al. assessed the reliability of four classification systems, namely, Ponseti and Smoley, Harrold and Walker, Catterall, and Diméglio et al. Nine children (13 clubfeet) were assessed by four examiners at different stages in the first 6 months of life (=180 examinations). The results showed kappa values varying between 0.14 and 0.77. Even though Dimeglio classification system had the greatest reliability, none of the systems were found to be entirely satisfactory. Ghanem et al. evaluated the results of treatment of 35 patients with unilateral clubfoot and analyzed them according to 13 different rating scores available in literature. They concluded that none of the rating systems were ideal and suggested a new and more objective classification system should be developed. Cummings et al., Catterall, and Steck andRobertson also had concerns about the lack of consistent results in clubfoot severity scoring.
Munsh et al. evaluated fifty idiopathic clubfeet by the scoring systems of Laaveg and Ponseti, McKay, Magone et al., and Ghanem and Seringe. Although there was a good correlation between the scores, there was very poor agreement between the grading of feet by these different systems.
Pirani et al. developed a method of clinically evaluating the degree of deformity in clubfoot. This scoring system has become widely accepted and popular among the health personnel owing to its simplicity, reliability, and ease of application. A foot can be assessed in less than a minute and no technical equipment is required.,,, In addition, there is a significant correlation between the initial score and the number of casts required for correction. A higher score also correlates with a higher chance of the patient requiring tenotomy for complete correction.
Several studies,,,,,,,,,, have been undertaken to study the validity and interobserver reliability of Pirani scoring system among trained health personnel [Table 6]. Cosma andVasilescu in a study to determine the reproducibility of Pirani and Dimeglio classification systems found a high Pearson's coefficient correlation (>0.85) for both the systems. They even suggested a simultaneous use of both systems as they are different and complemented each other. Flynn et al. performed an independent assessment of Pirani and Dimeglio classification systems. The correlation coefficients were found to be 0.90 (P = 0.0001) for the Pirani classification and 0.83 (P = 0.0001) for the Dimeglio classification. Both classification systems had a good interobserver reliability after an initial learning phase.
|Table 6: Studies on interobserver reliability of Pirani scoring system for clubfoot severity|
Click here to view
Pirani et al. found the interobserver strength of agreement in Pirani scoring to be substantial or almost perfect among three independent observers, with kappa score of TS, MFS, and HFS to be 0.92, 0.91, and 0.86, respectively. In a similar study, Jain et al. measured the interobserver reliability of Pirani score between five orthopedic surgeons at a foot deformity correction camp. Interobserver variability was calculated using kappa statistic for each of the signs. There was substantial agreement for TS (TS kappa-0.71) and also for MFS (0.68) and HFS (0.66) separately.
Majority of the clubfoot cases (approximately 80%) are reported from developing countries. There is a significant shortage of trained health-care personnel in developing countries to manage the clubfoot program. This shortage is mostly compensated by taking help from various nongovernmental organizations (NGOs), paramedical staff, and allied health-care personnel. Pirani scoring, which has become an integral part of the clubfoot management, has a proven substantial interobserver reliability among trained health-care personnel, but its efficiency in the hands of nonmedical and paramedical staff is still a matter of debate. Shaheen et al. measured the interobserver reliability of Pirani score between a physiotherapy assistant and an orthopedic surgeon in 91 virgin clubfeet in 54 infants at the Sudan clubfoot clinic. Overall, there was a moderate-to-substantial interobserver reliability for the Pirani clubfoot severity score and all its subcomponents.
Jillani et al. compared the reliability of Pirani score between an orthopedic surgeon and a plaster technician who had 2-year operation theater technician diploma. They found interobserver reliability to be fair to substantial with point-to-point interobserver agreement for all components of deformity to be 82%.
Counsellors form an essential part of the clubfoot treatment program. These are nonmedical personnel usually associated with an NGO who lend their support to the working of a clubfoot clinic. Their efficiency in effective and accurate Pirani scoring can be of great advantage to the clubfoot team. The present study was undertaken to determine whether, after a short training in the Pirani scoring, a nonmedical personnel can be as accurate as a doctor in assessing the degree of deformity in clubfoot.
The mean scores for each parameter were comparable (difference <0.1 of each other) [Table 2]. Overall, there were substantial agreement for PC and CLB and fair agreement for EH and MC when scores from the three independent observers were analyzed [Table 3] and [Table 4].
The interobserver reliability of a nonmedical counsellor was found to be comparable to a resident doctor keeping the scores of the consultant as gold standard. The nonmedical counsellor included in our study had no previous training in medical or paramedical branches. They had been a part of our team from the past 4 years and had been actively assisting the doctors in managing clubfoot patients as far as their record keeping and counseling is concerned. The counsellor as a part of the job was aware of the various deformities and scoring system of clubfoot. It was an integral part of their duty to fill the Pirani scores given by the doctors in their records and to follow them up in next visits. We provided a short training in Pirani scoring and they developed the scoring ability equal to a trained medical personnel.
A counsellor's efficiency in effective scoring of clubfoot children can help the program in a big way. The counsellors are required to make frequent field visits in nearby areas for creating awareness regarding clubfoot and followup the patients under treatment. During these visits, they can easily counsel the parents about the approximate number of casts that will be required the probability of tenotomy in the patient and can easily detect any relapse at the earliest, all this with a simple knowledge of Pirani scoring. In the clinic, the treating doctor develops a confidence in their ability to assess the degree of deformity in a clubfoot. Therefore, the trained counsellor can assist in performing this aspect of care in the clubfoot clinic, relieving the doctor of the time burden associated with the assessment.
There are a few key questions which may arise regarding the appropriateness of the research. There may be doubts whether the counsellors will apply the instruments in practice, whether study patients represent the population that will be rated and whether data have been analyzed using appropriate statistics. In this study, all these were taken into consideration. There were a few limitations of the study, the sample size was small, and it is difficult to generalize the efficiency of a counsellor which may vary from center to center, person to person, and upon the duration of association with the clubfoot team. Intraobserver reliability was not analyzed as it would have required multiple examinations of the same child, which logistically would have been difficult, and also, it would not have contributed to the research question as the researcher would not have known whether the scoring has been done correctly or not; further multicentric trials on this research questions may be needed to validate the findings.
| Conclusion|| |
Our study suggests that after a short training, Pirani score can be effectively used in assessing clubfoot severity by a nonmedical counsellor. This is particularly useful in developing countries, where orthopedic surgeons are few. Effective use of Pirani scoring in the hands of counsellors will go a long way in increasing the number of children which a clinic can cater and also improve the quality of service provided.
Declaration of patient consent
The authors certify that they have obtained all appropriate patient consent forms. In the form the patient(s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Lehman WB, Mohaideen A, Madan S, Scher DM, Van Bosse HJ, Iannacone M, et al.
A method for the early evaluation of the ponseti (Iowa) technique for the treatment of idiopathic clubfoot. J Pediatr Orthop B 2003;12:133-40.
Sud A, Tiwari A, Sharma D, Kapoor S. Ponseti's vs. Kite's method in the treatment of clubfoot – A prospective randomised study. Int Orthop 2008;32:409-13.
Grimes CE, Holmer H, Maraka J, Ayana B, Hansen L, Lavy CBD, et al.
Cost-effectiveness of club-foot treatment in low-income and middle-income countries by the ponseti method. BMJ Glob Health 2016;1:e000023.
Dyer PJ, Davis N. The role of the pirani scoring system in the management of club foot by the ponseti method. J Bone Joint Surg Br 2006;88:1082-4.
Fixsen J. Associate editor, data supplement. J Bone Joint Surg Br 2006;88:1082-4.
Shaheen S, Jaiballa H, Pirani S. Interobserver reliability in Pirani clubfoot severity scoring between a paediatric orthopaedic surgeon and a physiotherapy assistant. J Pediatr Orthop B 2012;21:366-8.
Jillani SA, Aslam MZ, Chinoy MA, Khan MA, Saleem A, Ahmed SK, et al.
A comparison between orthopedic surgeon and allied health worker in pirani score. J Pak Med Assoc 2014;64:S127-30.
Pirani S, Outerbridge H, Moran M, Sawatsky BJ. A Method of Evaluating the Virgin Clubfoot with Substantial Interobserver Reliability. Vol. 71. Miami, Florida: POSNA; 1995.
Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37-46.
Ponseti IV, Smoley EN. Congenital club foot: The results of treatment. J Bone Joint Surg 1963;45-A: 261-344.
Harrold AJ, Walker CJ. Treatment and prognosis in congenital club foot. J Bone Joint Surg Br 1983;65:8-11.
Catterall A. A method of assessment of the clubfoot deformity. Clin Orthop Relat Res 1991;(264):48-53.
Diméglio A, Bensahel H, Souchet P, Mazeau P, Bonnet F. Classification of clubfoot. J Pediatr Orthop B 1995;4:129-36.
Ghanem I, Seringe R. Comparison of evaluation methods of the results of congenital clubfoot treatment. Rev Chir Orthop Reparatrice Appar Mot 1995;81:615-21.
Laaveg SJ, Ponseti IV. Long term results of treatment of congenital club foot. J Bone Joint Surg Am 1980;62:23-31.
McKay DW. New concept of and approach to clubfoot treatment: Section III – Evaluation and results. J Pediatr Orthop 1983;3:141-8.
Magone JB, Torch MA, Clark RN, Kean JR. Comparative review of surgical treatment of the idiopathic clubfoot by three different procedures at Columbus Children's Hospital. J Pediatr Orthop 1989;9:49-58.
Wainwright AM, Auld T, Benson MK, Theologis TN. The classification of congenital talipes equinovarus. J Bone Joint Surg Br 2002;84:1020-4.
Cummings RJ, Davidson RS, Armstrong PF, Lehman WB. Congenital clubfoot. J Bone Joint Surg Am 2002;84-A: 290-308.
Steck H, Robertson A. The inter-observer reliability of the Pirani Score of Clubfoot. Bone Joint J 2014;96-B Suppl 13:21.
Munshi S, Varghese RA, Joseph B. Evaluation of outcome of treatment of congenital clubfoot. J Pediatr Orthop 2006;26:664-72.
Pirani S, Hodges D, Sekeramyi F. A reliable and valid method of assessing the amount of deformity in the congenital clubfoot deformity. J Bone Joint Surg Br 2008;90 (Suppl I):53.
Awang M, Sulaiman AR, Munajat I, Fazliq ME. Influence of age, weight, and pirani score on the number of castings in the early phase of clubfoot treatment using ponseti method. Malays J Med Sci 2014;21:40-3.
Cosma D, Vasilescu DE. A clinical evaluation of the pirani and dimeglio idiopathic clubfoot classifications. J Foot Ankle Surg 2015;54:582-5.
Flynn JM, Donohoe M, Mackenzie WG. An independent assessment of two clubfoot-classification systems. J Pediatr Orthop 1998;18:323-7.
Jain S, Ajmera A, Solanki M, Verma A. Interobserver variability in pirani clubfoot severity scoring system between the orthopedic surgeons. Indian J Orthop 2017;51:81-5.
] [Full text]
Pulak S, Swamy M. Treatment of idiopathic clubfoot by ponseti technique of manipulation and serial plaster casting and its critical evaluation. Ethiop J Health Sci 2012;22:77-84.
Sharma P, Verma R, Gohiya A, Gaur S. Analysis of the effectiveness of a non-governmental organization in supporting clubfoot clinic at a tertiary care center. SICOT J 2015;1:8.
Karanicolas PJ, Bhandari M, Kreder H, Moroni A, Richardson M, Walter SD, et al.
Evaluating agreement: Conducting a reliability study. J Bone Joint Surg Am 2009;91 Suppl 3:99-106.
Dr. Rahul Verma
Department of Orthopaedics, Gandhi Medical College, Bhopal - 462 001, Madhya Pradesh
Source of Support: None, Conflict of Interest: None
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6]