Development and Applicability Verification of a Competency Evaluation Index System for General Practice Team Leaders
The general practice team leader is not only a community health practitioner, but also a manager, commander and coordinator of a general practice team. So choosing an eligible person for the post of general practice team leader is of primary importance.
To develop and verify a competency evaluation system for general practice team leaders, providing a reference for scientifically assessing the competencies of a general practice team leader, and for selecting an eligible person as a general practice team leader.
By use of literature review and semi-structured reviews, we collected data regarding competencies for an eligible general practice team leader, and used them to initially develop a competency evaluation system for general practice team leaders. Then from July to October 2020, we conducted a three-round Delphi survey with a purposive sample of 20 experts (in the field of general medicine, medical education, public health management, or administrative management) to improve the system, and assess the indicators of the system using the analytic hierarchy process, and test the logic consistency among indicators, then the final system was formed. To assess values of the three-level indicators in the system, from November 1 to 30, 2020, we carried out a survey on wjx.cn (an online questionnaire survey platform) using a self-administered questionnaire developed by us, and selected a stratified random sample of 32 cases (16 directors of community health centers, and 16 general practice team leaders from 8 central urban districts of Shanghai) to anonymously complete the survey via scanning the WeChat RQ code using a mobile phone or computer. The survey data were collected to input into a database, and analyzed for understanding the inter-rater differences in the importance of the top 15 indicators.
The response rate, authority coefficient, and Kendall's W were 90.0%, 0.912, and 0.183 (χ2=42.516, P<0.001) respectively, for the first round of survey, 95.0%, 0.933, and 0.359 (χ2=68.937, P<0.001) respectively, for the second round of survey, and 100.0%, 0.940, and 0.516 (χ2=87.329, P<0.001) , respectively, for the third round of survey. The final system is composed of 5 first-level indicators, 17 second-level indicators and 43 third-level indicators. The weights for the 5 first-level indicators were 0.344, 0.222, 0.192, 0.137 and 0.105, respectively. The consistency ratios for hierarchical arrangement of indicators were all <0.1. Among the top 15 third-level indicators in terms of importance, except for the importance of "the ability to provide referral services", and "the ability to assist in the handling and management of public health events", the importance of other 13 indicators rated by directors of community health centers and general practice team leaders showed no significant differences (P>0.05) .
The system developed by us has high scientificity and practicability with a rational structure and well-targeted indictors, which may be used as a tool for the selection and training of general practice team leader.
Development，Reliability and Validity of the Health Management Service Skills Training Evaluation Scale for Rural Doctors
Few studies have evaluated the effectiveness of health management services sills trainings for rural doctors, and there is a lack of effective evaluation toolsin China.
To develop the Health Management Service Skills Training Evaluation Scale for Rural Doctors (HMSSTESRD) , and test its reliability and validity.
The initial version of HMSSTESRD was developed using the framework of Kirkpatrick model, then was revised using expert consultation and group discussion, and was pretested using a survey with a sample of 100 cases conducted between May to June, 2021. The scale was further revised according to the survey, and analyzed using reliability analysis and exploratory factor analysis, then the formal version was determined. In July 2021, the formal scale was retested in a sample (n=306) , and was revised according to the test results, and was subjected to confirmatory factor analysis, homogeneous reliability analysis, composite reliability analysis, discriminant validity analysis, and structural validity analysis.
The formal scale consists of 15 entries in 4 dimensions: reaction, learning, behavior, and results. The KMO index was 0.847, Bartlett's test of sphericity was significant (χ2=1 862.826, P<0.05) , and the percentage of explained total variance in exploratory factor analysis was 87.575%. The results of the confirmatory factor analysis were χ2/df=3.906, RMSEA=0.098, CFI=0.961, IFI=0.961, and NFI=0.948, indicating that the model fit was basically good. When it comes to the formal scale, the Cronbach's α was 0.967. The Cronbach's α for each dimension (reaction, learning, behavior, and results) was 0.962, 0.958, 0.971, and 0.919, respectively. And the CR values for these dimensions were 0.963, 0.958, 0.971, and 0.919, respectively. The arithmetic square root of AVE was greater than the correlation coefficient between the factors. Theinter-dimension correlation coefficients were 0.538 to 0.842, and the dimension-scalecorrelation coefficients were 0.754 to 0.934.
The HMSSTESRD developed by us has been proved to have good reliability and validity, which could be used as an effective tool for assessingtheeffectiveness of health management service skills trainingsfor rural doctors.
Development，Reliability and Validity of a Concise，Prediction Model-based Nutritional Risk Assessment Scale for Nursing Home-dwelling Older People
The Malnutrition Risk Assessment for Elderly Adults (WS/T 552-2017) , a malnutrition risk assessment scale issued by the National Health Commission has proven to have unsatisfied reliability and validity, with some inappropriate items in nursing home-dwelling older people. There is a lack of nutritional risk assessment scale for nursing home-dwelling Chinese older people.
To establish a reliable, concise, prediction model-based nutritional risk assessment scale applicable for nursing home-dwelling older people, and test its reliability and validity.
A survey using a questionnaire consisting of general demographic information and the Malnutrition Risk Assessment for Elderly Adults (WS/T 552-2017) was conducted with a convenience sample of 1 411 elderly people in 12 nursing homes of 6 cities, from November 2019 to January 2020. Variables screened by item analysis were included in an ordinal, multinominal Logistic regression model, and the statistically significant ones of them were then incorporated into a decision tree model. After that, ROC analysis was used to estimate the AUC of Logistic regression model and decision tree model in predicting nutrition status to select a better model to develop a concise nutritional risk assessment scale, and to determine the diagnostic threshold for nutrition status. Cronbach's α, exploratory factor analysis, estimation of AUC, sensitivity, specificity, Youden index and Kappa coefficient were used to evaluate the reliability and validity of the scale.
For predicting good nutrition, malnutritional risk, and malnutrition, the AUC of Logistic regression model was 0.962, 0.942, 0.989, respectively, and that of the decision tree model was 0.914, 0.868, and 0.968, respectively, indicating that the Logistic regression model was better, and suitable for developing the nutritional risk assessment scale. The final concise Nutritional Risk Assessment Scale for Nursing Home-dwelling Older People is composed of 10 items: BMI, changes in weight in recent 3 months, ability of engaging in daily activities, dental status, nervous and mental diseases, number of illnesses, types of drugs used, time spent on doing outdoor activities independently, eating ability, and the circumference of the shin. The total score of the scale for nursing home-dwelling older people can be 0-14.5 points, with 0-3.0 stands for good nutrition, 3.5-7.5 for nutritional risk, and 8.0-14.5 for malnutrition. The Cronbach's α of the scale was 0.463. Exploratory factor analysis obtained five common factors with eigenvalues greater than 1, explaining 69.9% of the total variance. When predicting the malnutritional risk, the AUC of the scale was 0.902, with 0.799 sensitivity, 0.870 specificity, and 0.670 Youden index. When predicting malnutrition, the AUC of the scale was 0.976, with 0.809 sensitivity, 0.953 specificity, and 0.761 Youden index. The Kappa coefficient for the scale was 0.627. The nutritional status of the 1 411 participants assessed by the scale was: 634 (44.93%) had good nutrition, 639 (45.29%) had malnutritional risk, and 138 (9.78%) had malnutrition.
The concise, Nutritional Risk Assessment Scale for Nursing Home-dwelling Older People developed using a Logistic regression model has proven to have good reliability and validity, which could be used as a tool to identify malnutrition risk or malnutrition in nursing home-dwelling older people.
Reliability and Validity of the Chinese Version of the Partners in Health Scale in Patients with Chronic Heart Failure
Chronic heart failure (CHF) is a common cardiovascular disease. Improving the self-management ability of CHF patients will contribute to quality of life improvement and reduction of rehospitalization and mortality rates. The Partners in Health (PIH) Scale is a measure designed by Flinders University, Australia, to assess the generic knowledge, attitudes, behaviors, and impacts of self-management in chronic disease patients, and is mainly used to assess the implementation effect of self-management projects in chronic disease patients.
To translate the PIH Scale into Chinese, then test the reliability and validity of the Chinese version in CHF patients, providing CHF patients with a tool for precisely assessing their self-management abilities.
The PIH was translated into Chinese with the guidance of the Brislin's translation model, then was revised according to the results of the review of a panel of experts, and a pre-test, and then the Chinese version of PIH (C-PIH) was developed. The demographic questionnaire, C-PIH, and Minnesota Living with Heart Failure Questionnaire (MLHFQ) were used in two surveys (one was conducted between April and June 2010, and another between April and June 2011) with 410 CHF patients selected from two grade A tertiary hospitals in Beijing using convenience sampling. Measurement of ceiling and floor effects, and item-total correlation were used for item analysis. Expert evaluation was used to evaluate the content validity analysis. Spearman's rank correlation coefficient was used to measure the criterion-related validity. KMO test, Bartlett's test of sphericity, exploratory factor analysis and confirmatory factor analysis were used for construct validity analysis. Monofactor analysis was used for validity analysis of known-groups. Reliability analysis was estimated by using the Cronbach's α.
Item analysis indicated that only item 3 (level of adhering to medication) of the C-PIH showed ceiling effect. Item-total correlation coefficients of the scale ranged from 0.424 to 0.761 (P<0.001) . The scale-level content validity index of the scale was 0.966. Item-level content validity indices ranged from 0.800 to 1.000. C-PIH was positively correlated with MLHFQ in terms of total score (rs=0.200, P<0.05) . The KMO value was 0.872 and Bartlett's test of sphericity was χ2=1 139.142 (P<0.001) , indicating that the sample size was appropriate for factor analysis. By exploratory factor analysis, 3 factors with an eigenvalue greater than 1.000 were extracted, including knowledge (7 items) , coping (3 items) and adherence (2 items) , explaining 66.514% of the total variance. The loadings of items on each factor ranged from 0.571 to 0.869. The original model fit indices did not reach the critical value. After adding the suggested covariance correlation between errors-in-variables e1 and e2, e6 and e7, the fitting indices of the modified model were acceptable (χ2/df=2.393, RMSEA=0.0851, CFI=0.968, NFI=0.953, NNFI=0.963, GFI=0.905, AGFI=0.854, RFI=0.932, IFI=0.966) . Known-groups analysis demonstrated that the C-PIH total score varied significantly by level of education, economic income, NYHA class, and treatment (inpatient or outpatient) in CHF patients (P<0.001) . Good internal consistency was indicated with a scale Cronbach's α of 0.890, and three factors' (knowledge, coping and adherence) Cronbach's α of 0.894, 0.807, and 0.511.
The C-PIH exhibited good reliability and validity, which may be used as a general self-management assessment tool in patients with CHF.
