Is Mandatory Nonfinancial Performance Measurement Beneficial? Susanna Gallani

Is Mandatory Nonfinancial Performance Measurement Beneficial?





Susanna Gallani Ranjani Krishnan

Takehisa Kajiwara



Working Paper 16-018



Is Mandatory Nonfinancial Performance Measurement Beneficial? Susanna Gallani Harvard Business School

Takehisa Kajiwara Kobe University - Japan

Ranjani Krishnan Michigan State University

Working Paper 16-018

Copyright © 2015 by Susanna Gallani, Takehisa Kajiwara, and Ranjani Krishnan Working papers are in draft form. This working paper is distributed for purposes of comment and discussion only. It may not be reproduced without permission of the copyright holder. Copies of working papers are available from the author.

Is Mandatory Nonfinancial Performance Measurement Beneficial?

SUSANNA GALLANI Michigan State University [email protected] TAKEHISA KAJIWARA Kobe University - Japan [email protected] RANJANI KRISHNAN* Michigan State University [email protected]

Acknowledgments: We thank Jeff Biddle, Clara Chen, Leslie Eldenburg, Matthias Mahlendorf, Melissa Martin, Pam Murphy, Steve Salterio, Greg Sabin, Daniel Thornton, Stephanie Tsui, Jeff Wooldridge, and workshop participants at the 2013 Canadian Accounting Association Meeting at Montreal, 2014 AAA Management Accounting Section Meeting at Orlando, University of Arizona, Erasmus University, Michigan State University, Queen’s School of Business, and Wilfrid Laurier University for their valuable comments and suggestions. We thank Kenji Yasukata, Yoshinobu Shima and Chiyuki Kurisu for their support in the collection of the data used in this study. * Corresponding author address: N207, North Business Complex, Broad College of Business, Michigan State University, East Lansing, MI 48824, Ph: 517-353-4687, Fax: 517-432-1101. Data availability: data are available from the authors upon request.

Is Mandatory Nonfinancial Performance Measurement Beneficial? ABSTRACT We use value of information theory and examine the effect of regulation requiring mandatory measurement and peer disclosure of nonfinancial performance information in the hospital industry. We posit that mandatory nonfinancial performance measurement has an information effect and a referent performance effect. The information (referent performance) effect arises because the new performance signals induce more precise posterior beliefs about individual (relative) performance. Using panel data from the Japanese National Hospital Organization, we analyze performance improvements following regulation requiring standardized measurement and peer disclosure of absolute and relative patient satisfaction performance. After controlling for ceiling effects, bounded dependent variables, and regression to the mean, results show that mandatory nonfinancial performance information measurement and peer disclosure improves overall performance (information effect) with larger improvements for poorly performing hospitals (referent performance effect). These effects are found even in the absence of any compensation-based incentives to improve performance. Keywords: Value of information, Patient Satisfaction, Mandatory performance measurement, Health care.

2

Is Mandatory Nonfinancial Performance Measurement Beneficial? 1. Introduction Improving health care outcomes, quality, and cost are topics of high policy interest. Systematic collection and disclosure of reliable, consistent, and comparable health outcome information is a popular policy tool that has been proposed in many countries. For example, in the US, the Center for Medicare and Medicaid Services (CMS) and the Agency for Health Care Research and Quality (AHRQ) developed a patient satisfaction survey titled Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) with an aim to produce and publicly disseminate “comparable data on patient's perspective on care that allows objective and meaningful comparisons between hospitals on domains that are important to consumers”, to “create incentives for hospitals to improve their quality of care”, and “enhance public accountability in health care by increasing the transparency of the quality of hospital care provided in return for the public investment” (http://www.hcahpsonline.org/home.aspx). Such regulations generate information that was previously unavailable to the decision maker and should therefore improve decisions. However, the requirement for public disclosure may cloud the value of the new information for improved internal decision making. For example, decision makers may become so highly focused on the public’s response to the information that they could exert more effort to manage the information rather than use the insights provided by the new information. Additionally, incentive mechanisms may further encourage managers to improve reported performance rather than focus on the actual drivers of performance (Dranove and Jin 2010). Mandatory requirements for new information that are bundled with public disclosure therefore do not allow for the assessment of the value of the new information to the decision maker absent any rewards or penalties arising from the public disclosure. An interesting question that arises is whether the mere availability of new information to decision makers has an independent effect on performance; that is, absent public disclosure or obvious monetary benefits or penalties, will information affect performance? Most economists and decision theorists would agree that information is largely beneficial to a decision maker. For example, value of information (VOI) is commonly defined using Bayesian methods

3

as the difference between the expected utility of an action based on the posterior probability given a new information set, and the expected utility of the action given only the prior information set (Pratt et al. 1995). However, as stated earlier, outside of a laboratory, one rarely encounters situations where access to new information is not confounded with other factors that impinge on the decision maker’s choices with the new information. In this study, we examine the value of information using a unique, quasiexperimental, Japanese hospital setting where new regulation requiring standardized measurement and peer disclosure (as opposed to public disclosure) of patient satisfaction was imposed. The new information was not previously collected by the hospital or available from other agencies. Further, performance on the new information metrics was not tied to incentive compensation or other pecuniary payoffs. We posit that mandatory performance measurement has two effects: an information effect that arises from the value of the new information about the firm’s own performance, and a referent performance effect that arises from the value of the new information signal about the firm’s performance relative to the peer group. Our empirical settings allow us to disentangle these effects without the confounding influences of performance pressures that could arise from incentive contracting or public disclosures. The Japanese hospital industry introduced regulation in 2004 requiring hospitals belonging to the Japanese National Health Organization (NHO) to be subject to an annual patient satisfaction survey using a standardized questionnaire. A neutral external agency surveys inpatients and outpatients about their satisfaction with a number of aspects of their hospital experience, including medical treatments and procedures, physician and staff behavior and attitudes, and hospital infrastructure. The results of the survey containing performance information on the level as well as relative rank of individual member hospitals are disseminated to all hospitals within the NHO. We analyze patient satisfaction panel data from all 145 NHO-member hospitals over a period of seven years (2004-2010). We first conduct a factor analysis of the survey responses and identify two satisfaction constructs, which we label as staff /treatment, and logistics/infrastructure. We then examine whether the patient satisfaction information results in an improvement in performance on each construct

4

in subsequent years, and whether there are differences in the extent of improvement based on initial relative performance. Our data are right- and left-bounded, which is typical of surveys that use a Likert-type scale. That is, the patient satisfaction score is constrained by both a floor and a ceiling, corresponding to the minimum and maximum values of the scale used. The boundary observations create problems in econometric estimation if standard models are used (Papke and Wooldridge 1996). Further, poorly performing hospitals have greater available range for potential improvement than highly performing ones. Consequently, performance improvements are likely to be non-linear. To address these issues, we use a fractional response econometric model, which accounts for non-linearity in the data and corrects for floor and ceiling effects. Our analyses control for regression to the mean, i.e., hospitals that have low (high) performance on patient satisfaction in a particular year could have high (low) performance in future years simply because of the nature of the distribution rather than actual changes in performance (Cook and Campbell 1979). Results indicate that mandatory measurement of patient satisfaction has an information effect. Firm-fixed effects using the fractional response method indicate an overall increase in inpatient as well as outpatient satisfaction. This increase is of larger magnitude in the year immediately following the release when the information is new compared to subsequent years. Trend analysis indicates that this effect is not driven by ceiling effects or diminishing marginal utility of effort. The improvement is found for all hospitals and not merely the poorly performing hospitals (i.e., hospitals in the lowest quartile of performance in 2004). We conclude that the patient satisfaction information has value. We also find evidence of the referent performance effect - hospitals that were performing poorly during the first year of the information release (i.e., in the lowest quartile of performance in 2004) have greater improvement in performance following the release. We conclude that the new information about relative performance facilitates improved goal-directed effort. Our study contributes to the literature in several ways. First, our unique, quasi-experimental design enables the assessment of the value of new information, which to our knowledge has not been

5

studied in an archival health care setting. Second, we study the value of information in the absence of confounding factors such as incentive compensation or performance pressure arising from public disclosures. Prior research finds that public disclosures of health care quality can motivate hospitals and providers to improve quality. For example, Evans et al. (1997) finds that mandated public disclosures of hospital mortality performance lead to subsequent improvements in mortality for hospitals that were performing poorly during the initial period. Lamb et al. (2013) finds that voluntary reporting of quality of ambulatory care motivated physician groups to improve quality. In both these aforementioned studies, the hospitals or physicians were already collecting the quality information and the only change was to make the disclosure public. That is, the information was not new to the decision maker but only the disclosure was new. Third, our empirical method attempts to disentangle the value of information about individual performance and the value of information about relative performance. Fourth, research in accounting has explored the importance of nonfinancial measures such as satisfaction in driving future financial performance (e.g. Ittner and Larcker 1998; Nagar and Rajan 2005). These studies have stressed the importance of nonfinancial information due to its role as a lead indicator of future financial performance, rather than the value of such information in improving decision making. Finally, our study has policy implications. Recently, there has been an increasing recognition that engaging patients in their own care is a cornerstone of successful health system reform (Hibbard and Greene 2013). From October 2012, the Patient Protection and Affordable Care Act of 2010 requires that hospitals collect and report patient satisfaction or face a reduction in reimbursement by up to 2.0 percentage points. The results of this study indicate that such regulation requiring mandatory collection and of satisfaction information has the potential to improve hospital performance on the reported measures, even in the absence of public disclosures. The following section summarizes the theory and extant literature. Next, we present a description of sample characteristics and the methods. A discussion of the results follows. The last section concludes. 2. Prior literature and hypotheses development 2.1 Institutional background of Japanese NHO

6

The Japanese National Health Organization (NHO) is the oversight agency for Japanese government hospitals. Headquartered in Tokyo, it comprises of 145 hospitals, which represent 3.5% of the total number of hospitals in Japan. Like other Japanese Independent Administrative Institutions (IAI), the NHO is the result of the separation between political and operational responsibilities for public services. NHO hospitals are grouped into two categories: general hospitals, which have discretion on the type of service they provide, and sanatoriums, which, in addition to offering services similar to general hospitals, are required to supply particularly expensive and risky medical services. Both types of hospitals provide inpatient as well as outpatient treatment. Funding for NHO hospitals comes from three sources: patients’ copayments for service rendered, reimbursements by the National Health Insurance or Employees’ Health Insurance, and public funding through government grants and subsidies.1 Patient copayments are received directly by the hospital, while insurance reimbursements are received through a claims process (Figure 1). Public funding allocation to each hospital within the NHO is dependent upon periodic performance evaluations of medical outcomes (e.g. mortality and morbidity) and assessment of the hospital’s need for resources. Prices for healthcare services are determined by the Japanese government. Price is, therefore, not a driver of patients’ choice of healthcare provider. --- Insert Figure 1 here --In 2004 the NHO introduced an annual patient satisfaction survey for every hospital within the NHO. Patients treated in NHO hospitals are required to complete a standardized questionnaire, which assesses their satisfaction with many aspects of their hospital experience, including medical treatment, the behavior of the staff and the quality of the infrastructure. The survey is conducted by an independent university research agency, which is unrelated to the NHO or its hospitals. The university research agency compiles and analyzes the results of the annual survey, and feedback reports are disseminated to all

1

Health Insurance in Japan is compulsory for all citizens and can be obtained either through the employer (Employees’ Health Insurance) or, in the case of self-employed individuals and students, through the National Health Insurance system. Special insurance programs are in place for elderly citizens (over 75 years). Patients pay for 30% of the cost of medical services, with the remaining 70% being reimbursed to the hospital by the insurer or the government. Medical costs exceeding the equivalent of $600 in a month are fully reimbursed by the insurer or the government.

7

member hospitals. The feedback reports include the average scores of each hospital on the various categories and each hospital’s ranking within the NHO. There is no explicit monetary incentive tied to patient satisfaction performance.2 Consistent with standard models of health economics beginning with the seminal research of Arrow (1963) and the more recent model of Kolstad (2013), we assume that health care providers obtain utility from non-financial outcomes such as patient satisfaction, independent of monetary compensation. We further partition the utility from patient satisfaction into two parts: one that arises from the hospital’s absolute performance level, and the other that arises from performance relative to peers. We then test whether the information provided by the patient satisfaction survey has value, in that it affects the choice of actions leading to improvements in the reported performance. 2.2 Information effect of Patient Satisfaction Information Adopting an expected value of information framework (Demski 1972; Bromwich 2007), suppose an organization is considering an action choice from a vector of potential actions given by A{a}, and a potential set of uncertain states S={s}. The expected utility of a particular action is U(s, a) and the organization maximizes its expected utility, that is

|

∗



∈

,

∅

. The relationship

between the action a and the expected outcome is based on subjective probability distributions based on past events (Feltham 1968). Suppose the organization obtains an additional information signal y from an information system ƞ. The information signal y allows for an improved assessment of the state and appropriate action choices, and it has value if it affects the decision that the organization would have made without the signal, thus leading to improved utility. The organization’s expected utility including the new information signal is

, ƞ,

∗



∈

,

∅ | , ƞ . The expected value of the new

2

Physicians and medical staff at the NHO are compensated on a fixed wage basis and are not provided performance-contingent bonuses. Physicians and staff obtain raises based on general macro-economic conditions. Section 3 examines physician compensation at NHO hospitals in greater detail.

8

information signal y is the difference between

, ƞ,

∗

and

|

∗

. 3 If this expected value is

positive, then the new information has value to the decision maker. We first assess whether the patient satisfaction measure provides new information. NHO sources indicated that before 2004, neither the NHO nor individual member hospitals were collecting information on patient satisfaction. Without this information, hospitals only had noisy priors about their own performance. Because of the tendency for individuals to be overconfident about their ability and overestimate their effort levels (Camerer and Lovallo 1999; Benoit and Dubra 2011; Kruger and Dunning 1999, 2002), health care providers likely concluded that their patient satisfaction performance was above average. Additionally, hospitals were unable to accurately assess the payoffs from their efforts to increase patient satisfaction. Thus, on average, hospitals are likely to have exerted less than optimal effort toward increasing patient satisfaction. With the introduction of the mandatory patient satisfaction survey, hospitals received an additional information system (corresponding to ƞ referred to earlier). ƞ contains two new signals - absolute patient satisfaction level (y1) and patient satisfaction relative to peer hospitals (y2). These two new signals are based on data collected systematically by an independent universitybased research center using a scientific, standardized, survey instrument. The new signals enable hospitals to revise their priors and obtain more accurate posterior beliefs about the relation between their actions and patient satisfaction, which influence future effort allocations and decisions, i.e. to move from utility function

|

∗

to

, ƞ,

∗ 4

The information on individual performance level (y1) provides a

more precise signal of performance, which allows for guided effort that is better suited to the circumstances (Morris and Shin 2002; Bandura and Jourden 1991; Ederer 2010). The positive weight associated with patient satisfaction within the healthcare provider’s utility function drives the extent to

3

Although VOI is sometimes interpreted rather narrowly as the amount a decision maker would be willing to pay for higher quality information, the analytical models of VOI are generic and refer to “value” in a flexible sense that allows for non-pecuniary interpretations (Bromwich 2007; Demski 1972; Raiffa 1968). 4 This assumes that hospitals are Bayesian, i.e., they use new information to update their prior beliefs, which is a standard assumption in decision theory (Pratt et al. 1995).

9

which the information is internalized and used in the subsequent decision making process. This leads to the following prediction about the information effect of nonfinancial performance information: H1. Introduction of mandatory nonfinancial performance measurement improves subsequent performance. 2.3 Referent performance effect of Patient Satisfaction Information The second signal contained in the new information system ƞ is patient satisfaction relative to peer hospitals (y2). Economic theory recognizes referent performance as an important driver of individuals’ and firms’ utility functions (Kolstad 2013; Sugden 2003).5 Referent performance is particularly important for strategically interdependent competitive firms (Lant and Hewlin 2002). If a valuable relative performance signal indicates poorer performance relative to a referent group, it prompts organizations to increase effort as well as search for new strategies that can enhance relative performance (Bandura and Jourden 1991) especially if decision makers have flexibility to respond to the new information (Abernethy and Bouwens 2005). Organizational theory posits that decision makers pay more attention to activities that fail to meet targets compared to those that succeed (Levitt and March 1988). Evidence indicates that poor relative performance is a higher motivator of performance than good performance. For example, Ramanarayanan and Snyder (2012) find that information disclosure in the dialysis industry is associated with reduction in mortality for poorly performing firms, but do not find comparable effects for highly performing firms. The relative performance signal y2 generated by the new information system ƞ eliminates idiosyncratic uncertainty creating a level field to assess performance. The new information signal y2 increases the accuracy of the posterior belief function about the mapping between effort and output relative to the organization’s peer group. The noise reduction value of relative feedback is higher for poorly performing hospitals because they likely expected to be above average in the pre-regulatory

5

In a reference-dependent utility model, preferences between decisions are influenced both by the expected outcome of the decision and by a reference point, which could be performance of a peer or competitor (Sugden 2003).

10

period, and therefore the relative information represents an unpleasant surprise.6 This serves as a motivation for poorly performing hospitals to increase effort to improve performance (Ederer 2010). Based on the above, we predict that firms with lower initial relative performance on patient satisfaction will have greater future performance improvements. H2. Lower initial performance on nonfinancial performance measures is associated with higher magnitude of subsequent improvements. 3. Data and analyses 3.1 Data and descriptive statistics The sample used in this study includes patient satisfaction data from 145 NHO hospitals for the period 2004 to 2010. Appendix A provides the list of the survey questions. The standardized patient satisfaction survey is administered every year during the months of June and July in all NHO general hospitals and sanatoriums. Separate surveys are administered to inpatients and outpatients and contain 15 questions (outpatients) and 19 questions (inpatients) respectively. The survey questionnaires contain multiple items related to quality of medical treatment, behavior of the staff, quality of infrastructure and facilities, waiting periods, etc. Ten additional questions capture the comprehensive assessment of the patient’s overall satisfaction. All the questions use a 5-point Likert-type scale, where 1 indicates “strong dissatisfaction” and 5 indicates “strong satisfaction.” Data are collected and processed by a university research center, which is unrelated to the NHO. Feedback reports are subsequently distributed to each member hospital. These reports indicate the average score for each of the questions included in the questionnaires and the ranking of the hospital within the NHO based on the overall satisfaction results. Table 1, Panel A contains the descriptive statistics. Hospital size is measured as the number of staffed beds. A dummy variable is used to capture whether the hospital is a general hospital (value = 1) or a sanatorium (value = 0). The sample includes 58 general hospitals and 87 sanatoriums. The number of

6

Prior literature finds that in the absence of information, individuals and firms tend to hold optimistic beliefs about their ability and therefore are overconfident about their performance relative to competitors (Kahnemann et al. 1982).

11

government and private hospitals per 100,000 inhabitants in the hospital’s prefecture (a geographical unit equivalent to a county) is used as a proxy for competition. Panel B provides the descriptive statistics for the overall satisfaction scores for each of the seven years included in the analysis. --- Insert Table 1 here --3.2 Variable reduction Patient satisfaction is a multidimensional construct (Chen 2009). To obtain a measure of the underlying dimensions, we conduct a factor analysis using principal component analysis with oblique rotation.7 Untabulated results show that for inpatients and outpatients alike, the items load on two factors (factor loadings > 0.6) and no variable cross-loads on more than one factor. Based on the items’ loadings we identify two constructs: staff /treatment, and logistics/infrastructure. Cronbach’s alpha values are greater than 0.9 for both inpatients and outpatients. Overall patient satisfaction is computed as the average of the scores reported for the ten overall satisfaction questions for inpatients and outpatients.8 3.3 Analysis of inpatients and outpatients We conjecture that inpatients and outpatients may differentially weigh the importance of each factor in formulating their assessments of overall satisfaction with the hospital.9 Therefore, we examine the extent to which each of the factors influences overall inpatient and outpatient satisfaction (Krishnan et al. 1999). We analyze the relationship between the aggregate overall patient satisfaction and each of the two satisfaction factors using OLS regressions with heteroskedasticity-robust standard errors clustered by firm. Separate regressions are performed for inpatients and outpatients. The results of this analysis are

7

The oblique rotation method allows for the possibility that the factors are correlated. The factor analysis on the ten questions related to overall satisfaction resulted in all questions loading onto one factor (factor loadings > 0.9 and Cronbach’s alpha > 0.97). 9 A recent survey conducted by the Japanese Ministry of Health, Labor and Welfare explored the major drivers of hospital choice for inpatients and outpatients. The sample consisted of more than 150,000 respondents, randomly selected from the patient population of all Japanese Hospitals. Overall, outpatients (inpatients) identified the following drivers of hospital choice: 38% (34.9%) prior experience at the same hospital, 37.6% (29.9%) physical closeness to their residence, school or place of work, 33.2% (49%) recommendation by doctors, 31.4% (34.7%) kindness of doctors and nurses, and 28.7% (25.5%) size/technology of the hospital. (Japanese Ministry of Health, Labour and Welfare. (2011). Patients Behavior Survey. from http://www.mhlw.go.jp/english/new-info/2012.html). 8

12

reported in Table 1, Panel C. The results indicate that while both factors are significant drivers of overall inpatient satisfaction, satisfaction with staff and treatment is a bigger driver (coefficient = 0.172) than is logistics and infrastructure (coefficient = 0.075, p-value of difference in coefficients < 0.10). For outpatients, only the staff and treatment factor contributes to overall satisfaction (coefficient = 0.185) while the logistics and infrastructure factor (coefficient = -0.004) is not significantly different from zero (p-value > 0.50). These results suggest that the relationship with the staff and the experience during treatment has a primary role in the patient’s satisfaction assessment. Therefore, for the remainder of the analysis, we do not examine the drivers of logistics and infrastructure outpatient satisfaction. 3.4 Estimation issues: Regression to the mean and fractional response model The hypotheses relate to the change in patient satisfaction subsequent to the introduction of the new regulation. Unobserved hospital-level characteristics may bias the results. That is, hospitals whose initial patient satisfaction measure is high are likely to keep performing well due to characteristics other than the patient satisfaction. A regression analysis of the satisfaction scores for each year on the satisfaction measure of the previous year confirms the presence of significant first-order firm fixed effects (untabulated). Consequently, all the subsequent analyses in this study control for firm fixed effects and estimate robust standard errors clustered by firm. Because of correlations between repeated measures, and standard deviations that decrease over time, it is necessary to consider the extent to which results may be a manifestation of regression towards the mean rather than actual improvements in patient satisfaction. That is, poorly performing hospitals may improve in performance simply because of the nature of the behavior of extreme values in a statistical distribution rather than an actual improvement.10 We are able to reject the hypothesis of regression to the mean based on the following. First, Table 1, Panel B shows that the overall mean patient satisfaction

10

Note that regression to the mean is primarily an issue when the analysis consists of only two observations, such as two variables measured on one occasion (e.g. control and treatment group in an experiment) or one variable measured on two occasions (e.g. pre-test post-test comparison after an experimental intervention). Regression to the mean is not a phenomenon that is relevant to multiple observations over time (Nesselroade et al. 1980), such as our longitudinal panel data.

13

measure increases over time, which rules out aggregate mean stability, a necessary condition for regression to the mean (Cook and Campbell 1979; Zhang and Tomblin 2003). Second, the overall satisfaction never decreases significantly below the initial (2004) levels for hospitals in the highest quartile of performance, which is further evidence against the aggregate mean stability condition. Third, we estimate the correlation between subsequent satisfaction measures after controlling for firm fixed effects. Untabulated results show non-significant correlations coefficients, thus rejecting the possibility that the improvement over time is purely a result of regression towards the mean (Cook and Campbell 1979; Zhang and Tomblin 2003). Finally, all analyses of satisfaction change include the initial (2004) satisfaction rates, which is standard practice for controlling for regression to the mean (Evans et al. 1997; Kolstad 2013). Consequently, we conclude that regression towards the mean has a non-significant influence on the results of this study. Patient satisfaction measures in our sample are bounded at both extremes. A patient who is extremely unhappy with the medical services provided by the hospital can, at most, assign a score of 1. Similarly, an extremely satisfied patient cannot assign a score higher than 5. With bounded data, traditional linear models, like OLS regressions, may misrepresent important characteristics of the relationship between outcomes and explanatory variables. For example, use of standard linear regression models could lead to predicted values outside of the response scale interval. Perusal of our data reveals that a non-trivial number of observations are at the extremes of the scale. Models involving non-linear transformations of the dependent variable, like the log-odds ratio, are likely to fail in the presence of response variables that take values at the extremes (Papke and Wooldridge 2008). Additionally, traditional linear functional models do not consider the diminished scope for improvement available to the firm when customer satisfaction is already at a high level. A linear model assumes that the distances between two response scale items are constant. However, an improvement from 4 to 5 on a five point scale is much harder to obtain than an improvement from 3 to 4. Transformations such as log-odds or beta distributions have been used in similar circumstances. However,

14

the problem with each of these alternative distributions is that they are not defined at the extremes of the scale unless an additional ad-hoc transformation is applied to the extreme values. A fractional response model circumvents the above issues with the application of Bernoulli quasilikelihood parameters estimation methods within generalized linear model settings. A fractional response model does not require special data adjustments for observed variables at the extremes and performs robust and efficient estimations of parameters using the assumptions of generalized linear models (Papke and Wooldridge 1996). To account for time-constant unobserved effects that could be correlated with explanatory variables (Wagner 2003), we specify our model as an unconditional fixed-effects fractional response model by adding indicator independent variables for each of the hospitals included in our sample. Our fractional response model takes the following form (Papke and Wooldridge 1996) | where

∙ is a cumulative distribution function (cdf) and



(1)

⁄1

satisfies 0

1 for all . The predicted values of G need to lie on the interval between l (the lowest end of the response scale) and h (the highest end of the response scale). Estimation of a fractional response model requires that the dependent variable is in the form of a proportion in the range of [0 ,1], with a positive probability mass on the extremes. Therefore we transform each response scale variable into a proportion. 3.5 Compensation practices at NHO Our interest is in examining the value of information even in the absence of incentive compensation tied to performance improvement. The vast majority of rank and file workers receive minimal to zero incentive compensation and it is important to examine if improving their information set can improve decisions. Prior research by (Banker et al. 2000) finds improvement in nonfinancial performance following the implementation of an incentive plan that includes nonfinancial performance measures. In their setting, although customer satisfaction measures were tracked before the implementation of an incentive plan that includes such measures, these measures did not have a performance effect until it was used explicitly for incentive purposes. Similarly, (Kelly 2007)

15

experimentally demonstrates that “though providing feedback on nonfinancial measures alone may improve managerial decisions over time, the key to enhancing decision quality may be providing both feedback and incentives on nonfinancial measures.” In Evans et al. (1997), even though the hospitals previously collected mortality data, performance improvements were only observed when the mortality measures were disclosed to the public. This suggests that disclosure to external stakeholders, rather than the information per se, has a performance effect. Whether or not increased patient satisfaction information improves performance in the absence of an explicit link to incentive compensation, and when the information is disseminated to internal as opposed to external stakeholders is an open empirical question. We aim to study the value of information unfettered by the effect of financial incentives. Consequently, we first examine physician compensation practices at NHO hospitals using field and archival data to determine if there is any link between patient satisfaction performance and compensation. Field evidence of compensation practices at NHO We conducted interviews with hospital administrators at the NHO headquarters to glean information about physician compensation practices. These interviews revealed that there is no explicit link between physician compensation and patient satisfaction performance. Essentially, Japanese NHO physicians are government employees. Each physician, nurse, and paramedic is classified into a particular grade based on a hierarchy, and each grade is provided compensation as per the government salary schedule. The typical compensation package includes: a monthly salary, allowances for cost of living, overtime and travel. Appendix B contains information on employment, compensation, and promotion systems at NHO. Archival evidence of compensation practices at NHO To empirically examine whether there is any link between patient satisfaction and physician compensation, we estimated the following model of physician bonus as a function of patient satisfaction: Physician Bonust = α + β1 Overall Inpatient Satisfactiont-1 + β2 Overall Outpatient Satisfactiont-1 + β3-7 Year Dummy +β8 Competition+ β9 Hospital Dummy+ β10 Size (2)

16

where Bonus is the total annual bonus payout by the hospital to physicians. We include control variables for competition, hospital type, and size. We control for competition because a large body of research in economics, management, and strategy finds that competition intensity increases firms’ performance pressures (Banker and Mashruwala 2007; Chen et al. 2013; Scherer and Ross 1990), especially in industries that compete on non-price aspects such as healthcare (Joskow 1980; Pautler and Vita 1994; Robinson and Luft 1985). Additional information may be more valuable in more competitive industries as suggested by Raju and Roy (2000). We measure competition intensity as the number of private and public hospitals per 100,000 people in the geographical area (prefecture) where each NHO hospital is located. The Japanese National Health Organization is comprised of two types of hospitals, general hospitals and sanatoriums. General hospitals are similar to private hospitals and are allowed greater discretion in the choice of healthcare services they offer. Sanatoriums are expected to provide not only the services that are provided by general hospitals, but, in addition, they are required to provide special services “that cannot be dealt with properly by Non-National Hospital Organizations due to historical and social reasons.”11 These include treatment of expensive, long-term, risky, or communicable ailments such as tuberculosis, AIDS, Alzheimer’s, ALS, complex mental illnesses, and invasive or terminal cancer. Sanatoriums face a different set of pressures. We include a control indicator variable Hospital, which takes the value of 1 for general hospitals and 0 for sanatoriums. If patient satisfaction were taken into consideration in bonus payouts, the coefficient on

and

would be positive. We use lagged satisfaction scores because the satisfaction scores for year t are only released to member hospitals in the following year and therefore likely to only be incorporated into the following year bonus payments. The results of estimating equation 2 (Table 2, Column 1) indicate no significant association between bonus and either inpatient or outpatient satisfaction. These results suggest that patient satisfaction is not taken into consideration in the determination of bonus payments. Thus, we

11

National Hospital Organization (Independent Administrative Institution) page 1; http://www.mof.go.jp/english/filp/filp_report/zaito2004e-exv/24.pdf.

17

conclude that improvements in patient satisfaction are not driven by the motive to increase incentive compensation.12 --- Insert Table 2 here --3.6 Patient satisfaction and hospital revenues Grant revenues It is possible that patient satisfaction is driven by the desire to increase grant revenue for NHO hospitals. If so, although there is no explicit tie of satisfaction performance and incentive compensation, hospitals may have a pecuniary benefit to improving satisfaction. To test this, we estimate the following regression model:

Hospital Grant Revenuet+1 = α +β1 Overall Inpatient Satisfactiont-1 + β2 Overall Outpatient Satisfactiont-1 + β3-7 Year Dummy +β8 Competitiont + β9 Hospital Dummy + β10 (3) In equation 3, Hospital Grant Revenue is the lagged amount of grant revenues. We use two-year lagged grant revenues because conversations with hospital managers indicate that the review of grants happens mid-term over a five-year program (i.e., every 2.5 years). Results are provided in Table 2 (Column 2) and indicate no statistical association between patient satisfaction and hospital grant revenue. Our conversations with NHO administrators revealed that hospital grants are based on the research output of the hospitals rather than patient satisfaction.13 Patient revenues Prior research finds a positive relationship between customer satisfaction and subsequent revenues (Ittner and Larcker 1998; Hallowell 1996; Chen et al. 2009). The mechanisms by which customer satisfaction influences revenues include customer attraction, customer loyalty, and word of mouth (Rust et al. 2002; Szymanski 2001), which also operate in the non-profit health care industry (Stizia and Wood 1997; Gemme 1997). When patients are satisfied with the hospital, they tend to return.

12

We re-estimated equation 2 using two-and three-year lags and did not find significant results. We re-estimated equation 3 using three-, four- and five- year lags and did not find significant results. Similar analysis of changes in grant revenues did not yield significant results. 13

18

Additionally, their opinion influences others’ choices (Gemme 1997).14 A successful patient satisfaction program can therefore result in increased revenues. To test the relationship between patient revenues and satisfaction, we estimate the following regression model:

Hospital Patient Revenuet = α + β1 Overall Inpatient Satisfactiont-1 +β2 Overall Outpatient Satisfactiont-1 + β3-7 Year Dummy +β8 Competitiont + β9 Hospital Dummy + β10 Sizet (4) In equation 4 we use lagged values of satisfaction in order to allow for word of mouth and other mechanisms to take place. We find a statistically significant association between patient satisfaction and lagged revenues (Table 2, Column 3), which indicates that patients respond to hospitals’ efforts to improve patient satisfaction.15 Evidence that patient satisfaction increases hospital revenues introduces potential for physicians to obtain pecuniary benefits from increasing satisfaction if compensation is tied to hospital revenues (Finkler 1983). We test the relationship between physician bonus and patient revenues and find no association.16 Prior studies on the role of nonfinancial measures as lead indicators of financial performance are generally subject to the confounding effects of incentive compensation. Our setting allows us to contribute to this body of literature by providing evidence of a positive relationship between satisfaction and future revenue even in absence of compensation-based incentives. 4. Results of hypotheses tests 4.1 Information effect of patient satisfaction H1 predicts a positive effect of patient satisfaction information. That is, the release of patient satisfaction information increases nonfinancial performance in subsequent years on the disclosed measures due to the incremental value of the information signal regarding the level of performance. If the

14

Gemme (1997) reports survey results indicating that 90% of patients’ choice of health care provider are influenced by other patients’ opinions, and that 40% of surveyed subjects had consulted a patient who had used the service of the organization they had chosen to use. 15 We re-estimated equation 4 using contemporaneous patient satisfaction data and found similarly significant associations with patient revenue. 16 We tested the association between bonus and patient revenues, and between bonus and lagged patient revenues and did not find significant results (p-value of coefficient on bonus term > 0.35 in all estimations).

19

patient satisfaction signal has information value, then we would expect the following: first, patient satisfaction performance will increase for all hospitals following the first release of satisfaction information. Second, the increase in satisfaction will be greatest during the first year following the initial release and the rate of increase in satisfaction will be lower in the subsequent years relative to the first year because the value of new information is most salient in the year subsequent to the release.17 Univariate analysis Table 3, Panel A provides information on the mean inpatient and outpatient satisfaction for each year subsequent to the information release partitioned by the level of hospital satisfaction performance during the first year of the release (2004). It can be seen that in the year following the release (2005), the average satisfaction for the full sample of hospitals increased significantly for both outpatients (the t value of difference in mean outpatient satisfaction of 4.043 in 2005 and 3.962 in 2004 is 3.13, p