The development of evaluation as a concession

The development of evaluation as a concession

Evaluanon andProgram Planmng, Vol 2, pp 231-234, 1979 0147-7189/79/030231-04502 00/0 Copyright © 1979 Pergamon Press Ltd Pnnted in the U S A All rig...

332KB Sizes 1 Downloads 1 Views

Evaluanon andProgram Planmng, Vol 2, pp 231-234, 1979

0147-7189/79/030231-04502 00/0 Copyright © 1979 Pergamon Press Ltd

Pnnted in the U S A All rights reserved

THE DEVELOPMENT OF EVALUATION AS A CONCESSION

ARTHUR WIENER Child and Youth Services Regina, Saskatchewan

ABSTRACT The organzzatton o f evaluation activities takes place wzthzn the already exzstTng programs and power arrangements o f complex organizations In this context, evaluation ts hkely to develop along the hnes o f a licensed concesszon operatmg within a particular program area and as an expression o f a concession made by complex orgamzatzons to the Meal o f ratzonal decision-making In addition, the formatTon o f professzonal career paths tn evaluation may lead to career tmmobzhty The "evaluation as a profession "issue ztself could be resolved not by evaluation b e c o m m g a profession but by evaluatTon b e c o m m g a specialty o f the professional manager

The pre&ctmns Morell and Flaherty (1978) make a b o u t the development o f evaluation as a profession are prechctlons made from the standpoint of a proacnve or "evaluataon on the m a k e " perspectwe An alternative set o f pre&ctlons made from a reacnve or "evaluauon being m a d e " perspectwe is presented m ttus a m c l e F r o m a reactwe perspecnve, the actlvmes o f evaluators and evaluation actwmes are wewed not so much as ln&catmns o f the development o f evaluation as a professmn but as mdlcatmns of the development o f evaluatmn as a concession The reactwe standpoint takes into account the power of already existing arrangements to affect the development of evaluatmn In terms o f Parsons' (1951) dehneation o f the social value-orlentatmns o f unlversahsm and p a m c u l a n s m , these existing arrangements are regarded as powerful enough to shape the development o f evaluation according to the value-onentatmn o f partlcular1sin Within the framework o f pamcularlsm, achievements are valued to the extent that they are "relatwe to and/or on behalf o f the p a m c u l a r relatmnal context in which the actor [evaluator] is involved " As a value system, parnculansm sharply contrasts with umversahsm which requires that achaevements be valued to the extent that they are. "in accord with umversallzed standards and generalized rules relatwe to other actors [evaluators] " It ~s clear that it is the value-orientation of umversahsm x~hlch is central to the professions and which dlstmgmshes them from other forms o f orgamzed actwlty The desire to serve the public good to act altruistically, to be graded b.,, principles, and to estabhsh and maintain an eth-

lcal code are all expressaons o f umversahsm This being the case, a characterization o f the development o f evaluatmn in terms o f the values o f partlculansm would undermine the validity o f the "evaluation as a professmn" claim While "professmn" suggests the development o f a set o f actwmes according to umversahst~c principles, the term "concession" is introduced to suggest the development o f a set of actwmes according to parnculanst~c characteristics (e.g., self interest, expediency, and m-group ethos) and to hlghhght the reactwe nature of certain developments There is sufficient evadence in the hterature cited by Morell and Flaherty and m other work concerned with the functmnlng o f complex orgamzatlons to warrant the e x a m m a t m n o f the development o f evaluation as a concesstun m at least two senses o f the word First, there ~s a &stlnct posslbdlty that evaluation work will come to be organized along the hnes o f a licensed concessmn located within a p a m c u l a r program and/or sub-program area (e g . programs areas health, crime, e d u c a n o n , with subprogram areas being acute care, mental health, dehnquency, remdw~sm, teaching effectweness, remedmtlon) The licensed concessmn model o f evaluatmn suggests that social legmmlzation, techmcal development and support bases, ethical norms and sheer access to data will be available to evaluators largely as a result o f their p a m c u l a r relationships to, and membership in, some pre-existing pro° gram area The fact that such program areas will serve both as a power base and as a source o f data suggests that the hcensed concessmn arrangement may subvert and pre-

Requests for repnnts should be sent to Arthur V~]ener Child and k outh Services 1601 College Avenue Regana Saskatchewan Canada 231

232

ARTHUR WIENER

clude the more general development of evaluation as a profession In a second and more familiar sense of the term "concession " evaluation represents a concession made by complex organizations to the rational model of decisionmaking In an effort to promote the view that complex organizations are neutral, rational Instruments for carrying out important social functions and to meet newer criteria of accountability, complex organizations concede the virtues of evaluation. At the same time, however, they attempt to control the destablhzmg potentialities inherent in a critical scrutiny of organizational goals and effectiveness In the best of all possible worlds, "evaluation functions provide information needed by rational decision-makers for discrete decisions " In lesser circumstances, "evaluation may be seen as a ritual whose function is to calm the anxieties of the citizenry and to perpetuate an image of governmental ratlonahty, efficacy and accountability," (Floden & Wemer, 1978). Within the context of complex organizations the misuse, disuse, and abuse of evaluation work suggests that the concession to evaluation has been made, for the most part, grudgingly This view of the development of evaluation as a concession made by complex organizations to the rational model (Argyrls, 1973) is mostly ignored by Morell and Flaherty's proactlve "evaluation on the make" perspective even though some of the evidence cited in support of their view provides evidence for the evaluation as a concession view Before turning to this evidence, it is Important to discuss how it is that an "evaluation on the make" perspective misses such evidence This can be partly explained by the fact that the proacttve view stresses the motivations and needs of evaluators and not the character of the demand press in which evaluation activities take place This stance (combined with Morell and Flaherty's preference for, and rehance on, classical sociological descriptions of how professions develop) leads to a nostalgic distortion of the current scene and to the ignoring of alternative possibilities. One case in point is the nostalgic distortion of Wllensky's (1964) contribution, Nostalgia finds expression in Morell and Flaherty's preference for things past, l e., for the known problems of developing professions descnbed by Wllensky and other sociologists. Distortion occurs when Morell and Flaherty cite Wllensky in support of their mew that evaluation is on the path to becoming a profession while paying little attention to the lmphcatlons of Wdensky's major conclusion that "the idea that all occupations move toward professional a u t h o r i t y - t h i s notion of the professionalization o f everyone-is a bit of SOClologmcal romance " Paying little attention to Wllensky's warning about the ascription of "profession" to a set of activities as an exercise in SOClOlogmal romance, the authors pay even less attention to Wllensky's formulation o f an alternative to the classical notion of professmn In "The Professionalization o f Everyone 9'' Wllensky discusses many of the barriers to formlng new professions,

especlall.v the barriers existing professions and other alread3, estabhshed power arrangements exert He suggests that an important alternative to the classical professional is the hybrid "'program professional ' With the growing importance of specific p r o y a m knowledge (both technical and tacit I, with the 1re.leasing ability of complex organization to modulate and to control information, the center of gravlt2¢ for evaluators will we suggest, increasingly be particular program areas within already existing organizations and not a profession of evaluation To put it another way, the ability of a unlversahstlc form of organization (1 e , the professional form) to counter-balance the power of complex organizations needs to be both deep and extensive in order to be effective If the partlcularlstlc, plurahstlc and fragmenting power of complex organizations cannot be overcome or balanced, then the shape of organized evaluation activity is likely to be mapped onto the program features of already existing organizations This raises the possibility of professional evaluation activities being carried out wzthout there ever being a formally constituted profession of evaluation. Following Wdensky (1964), the program professional concerned with evaluation would "combine professional standards with a programmatic sense " Their form of professional orgamzatlon would not follow the classical lines anticipated by Morell and Flaherty because the generic view of evaluation simply would n o t be an orgamzmg principle, rather, a categorical view of evaluation pegged to some particular program area would serve as an organizing principle Should this categorical view of evaluation come to serve as a principle of orgamzatlon, then concerns about the merits and demerits of the profession of evaluation would be misdirected Here, evaluation activities shorn of their "'value free social science" aura and forced to take into account (1 e , react to) already existing programs and territorial claims will begin to resemble a licensed concession operating within, and depending upon, a particular program area for such necessary supplies as social legitimacy (including ethical codes), lntra-organlzatlonal credlbihty, and access to data and new techniques In return for license and supplies, evaluation program professionals would be expected to contribute to the welfare o f the program and orgamzatlon in which and for which they operate. With respect to those public and private organizations set up to "do evaluation," it is reasonable to expect that such organizations will have to adapt to many of the same social and economic circumstances to which other members of the "research society" (Glatt & Shelly, 1968) have already adapted. Boguslaw describes the societal ethos which arises as a result of such an adaptation as being characterized by 1. Partlculansm The acceptance or rejection of research clmms based on who makes them, 2 Mlserlsm The hoarding of one's findings to prevent their use by rivals, and 3. Organized dogmatism The taboo against raising

The Development o f Evaluation as a Concession

serious questmns about prevmus research done by your own group (Glatt & Shelly, 1968) What is anticipated then is that as wlth other research orgamzatlons, organizatmns dedicated to the business o f evaluatmn research will find themselves being at least shghtly more dedicated to the evaluation busmess as they are forced to deal with internal and external problems which have httle to do with evaluation research itself To those seeking to estabhsh a professmn o f evaluatmn, this d e s c n p t m n o f some o f the forces whach inhibit, dwert and pervert thls posslblhty provides an impetus for a closer examination of this quest for a professmn Admittedly, there is somethmg qmte appeahng to the idea o f building a field of knowledge and striving towards the creation of a new professmn One o f the most important reasons for this appeal is the sense of a u t o n o m y such a n o t m n must gwe to those who feel constrained b o t h by the dlsclphne imposed by their sklU acqmsltion and by the various constraints that affect the use o f these skills m complex orgamzatmns In an important sense, this quest for a professmn constitutes an a t t e m p t to construct a ratmnal realm of discourse and freedom within an increasingly complex, historical, event-structured, fragmented, and resource hmated SltUatmn Seen this way, the quest is to a large extent a response to the v~c~ssltudes o f modern hfe and an a t t e m p t to retain a sense o f a u t o n o m y and craftsmanship in the face o f the requirements o f complex orgamzatmns for group cohesion, compliance, etc To those who seek to develop evaluatmn as a professtun as a route to greater a u t o m o m y , the picture o f evaluation as a concession is at best cold comfort and at worst just another way of increasing an uncomfortable awareness o f current personal and organizational constraints. F o r those who do persist m pursuing the path to professionahzatmn as a route to greater a u t o m o m y in their work hfe, some a d d m o n a l discussion o f the ironic possibilities of professlonallzatmn might prove enlightening and possibly humorous Again the aim of this paper as to discuss evaluatmn from a reactwe point o f view. In "Professmnallzatmn as Career I m m o b i h t y , " Goldner and Rltti (1967) show how industrial organizations create professional career paths in reaction to internal pressures for career advancement The end result ~s career l m m o b l h t y and consequent loss o f influence at the pohcy level for those finding themselves standing on professional career ladders These ladders are reserved for those in the organizanon whose technical talents are too valuable to lose but who are pohtlcally unacceptable for p r o m o t m n to management and pohcy positions It is then entirely possible that the development

233

of evaluation as a professmn may be one way of managing the critical and destablhzing thrust of evaluatmn research b y l m m o b d m n g evaluators In addition, a more general effect o f the creation o f professional careers in evaluatmn may be the prowsion of what Larson (1977) calls, "a sense o f order to come." The end result would not be as Morell and Flaherty anticipate in their approving discusstun o f the development o f a civil service track for mental health program evaluatmn m New York State. "Career," w r i t e s garson, closely brads the projected self to organlzatmns or to professmnal lnstltutmns which insure " c o n t l n m t y m status in the labor market " The expectation o f career is therefore a powerful factor o f conformity with the existing social order and a source o f basic conservatism Thus instead of being a route to a u t o n o m y , the development of evaluation as a profession for some could be the road to career l m m o b l h t y and function so as to control the critical and destabdizing potentialities o f evaluation research Another Ironic possibility of what the development o f evaluation as a profession might mean is anticipated by Cox (1977) in his discussion of evaluation and management He comes to the conclusion that to be effectwe evaluators must "Inconvemence themselves " For Cox this means that an evaluator increases the chances of his or her work being used to the extent that he or she adapts to the managerial style This means adapting to such inconvemerit and counter-professional characterlmcs o f the managerlal style as unrelenting pace, brevity, variety and fragm e n t a t m n o f work. short range demands of external authority, casual attitudes towards vahdlty, and pohtlcal expediency The more a professional evaluator can adapt to the managerial style, the more hkely will evaluation research be utilized However, the mote the professional evaluator adapts to the managerial style, the less he or she will be able to adhere, for example, to the professional norm of competency (which at a minimum requires much more than a casual regard for the issue o f validity) and to the norms o f altruism and service To be a professional is to be ineffective, to be effectwe is to violate professional norms Here irony blends into paradox and absurdity Perhaps we have taken the "evaluation as a professmn" issue too seriously and have not recognized the transitional nature o f this conceptuahzatmn F o r if Cox is correct about the o v e m d m g importance o f the manageraal style. then evaluation will develop not into a profession but into a specialty of the professional manager

REFERENCES &RGYRIS C Some bruits of rational m a n organizational theory Pubhc 4dmmtstrattonRevle~ 1973 3

F L O D E N R E , & WEINER. S S R a t i o n a h t y to ritual The multiple roles o f evaluation m government processes Pohc~ Scwnces, 1978 9

COX G B Managerial style I m p h c a u o n s for the utilization of p r o g r a m e ~ a l u a t l o n m f o r m a t l o n Elaluatton Quarterh, 1977 1(3~

G L A T T E , & SHELLY. M W ( E d s ) York Gordon and Breach 1968

The research socwty New

234

ARTHUR

WIENER

G O L D N E R . F H , & RITTI, R R Professlonahzataon as career lmmoblht?~ American Journal o¢ Soczology 1967. 72(5)

atlon as a profession Current status and borne pred~ctaons Ltah¢atzon and Program Planmng, 1978 1 11-18

LARSON, M S The rise o f professzonahsm A soclologzcal analvsis Berkeley Umverslty of C a h f o r m a Press, 1977

PARSONS T The social system Glencoe N "f

MORELL, J A , & F L A H E R T Y , E W The d e v e l o p m e n t of evalu-

Free Press 1951

WILENSKY, H L The professlonatlzataon ot ever)one ~ 4mertcan Journal o t Sociology, 1964, 70(2)