CLOUDS AND SUN
LINE
  HORIZON SITE  

Public Education Policy Through Policy Impact Analysis

The Role of the Educational Planner*

by Charles Teddlie, E. Raymond Hackett, and James L. Morrison

[Note: This is a re-formatted manuscript that was originally published in World Future Society Bulletin, 1982, 16(6), 25-30.]

Using a current planning problem faced by five Louisiana parishes, the authors illustrate step by step how policy impact analysis techniques and the advisory guidance of a state educational planner can be used to develop policies for attaining desirable futures for education on a region-wide basis.

The decade of the 1980s promises to be characterized by shifting demographics, changing enrollment patterns, changing program demands, and fiscal restraints.  Given these conditions, identifying, analyzing and evaluating educational policy options at the local and state levels take on increased importance.  It is our contention that the policy impact analysis model as described by William Renfro (1980) in an earlier issue of WFS Bulletin provides a framework within which a variety of futures research techniques may be used to develop and assess specific educational policies.

Our purpose in this article is to briefly review the policy impact model, describe a current situation in the state of Louisiana amenable to the application of this model, and illustrate the role that a planner at the state level could play by using the model to provide assistance to local school districts.

The Policy Impact Analysis Model

There are four stages in the policy impact analysis model: monitoring, forecasting, goal setting, and policy analysis and implementation.  Monitoring refers to the identification and selection of issues of concern to policy makers.  The second stage of the model, forecasting, involves using a variety of futures research techniques to forecast probable futures and their relationship to selected issues.  In response to the projected trends and probable futures, policy makers then establish goals: the third stage of the model.  

In the fourth stage, a variety of possible policies are analyzed in order to determine their probable impact, and are ranked on those characteristics deemed important; e.g., relative costs versus benefits.  Those policies ranked at the top are then implemented.  Evaluation occurs when the stages of the model are repeated using additional analyses and further refinements.  This model, illustrated in Figure 1, will be described in more detail below.

 

Figure 1

Policy Impact Analysis Model

Stage 1
    Monitoring
Stage 2
    Forecasting
Stage 3
    Goal Setting
Stage 4
    Policy Analysis/Implementation

            

Louisiana: An Example

The Louisiana State Department of Education (LSDE) was recently contacted by local school boards and state legislators from five parishes in the northwest region of the state, a section in which the mining of extensive lignite coal resources will begin in the near future.  Local school officials were concerned about what demographic or socioeconomic trends in their area they might expect since these would, of course, affect the validity of their enrollment projections and facilities planning for the next ten to fifteen years.  

It must be noted that the role of the state educational planner in Louisiana is a sensitive one, since the current constitution (written in 1974) delimits the state's role in local school business matters.  However, the planner can do monitoring and forecasting for the local school board and can aid the board in goal-setting and policy implementation by coordinating local efforts with regional and state activities.  This coordinating role with regard to goal-setting and policy implementation is particularly important in an area such as the northwest region of Louisiana, where the needs of five parishes must be taken into consideration.

In the following sections, the role of the state education planner as the doer of the monitoring and forecasting stages and as the facilitator for the goal-setting and policy implementation stages will be delineated using the northwest Louisiana example.  It should be noted at the outset that this project is in the planning stages, and that none of the proposed activities by the LSDE has yet been accomplished.  It is hoped that working through the application of the policy impact analysis model to this problem area in advance will aid the LSDE if the project is undertaken.  

Stage 1: Monitoring

The three components of monitoring are: 1) identifying the area of study; 2) selecting appropriate indicators (or variables) for study; and 3) developing and plotting the historical data.  For the state education planner in Louisiana, the identification of an appropriate area of study could come from several sources: 1) requests from the state Superintendent of Education; 2) requests from the Board of Elementary and Secondary Education; 3) requests from local school boards; 4) requests from state legislators; and 5) personal identification of an area of study by the planner.  

In the case of the northwest Louisiana project, requests for technical assistance were received from local school boards and from state legislators.  These requests reflect growing concern about orderly growth and development in the lignite impacted area.  During 1979, ten corporations requested permits from the State of Louisiana for the exploration, mining, and use of lignite in the area.  It is estimated that over one billion tons of recoverable lignite exists in Louisiana, comprising 17.4 percent of the state's known energy resources (Louisiana State Department of Natural Resources, 1980).

In 1979, Governor Edwin Edwards designated the previously mentioned parishes as an Energy Impacted Area pursuant to Section 601 of the Powerplant and Industrial Fuel Use Act of 1978 (PL 95620).  A proposal by Senator Don Kelly of Natchitoches Parish during the 1981 Louisiana Legislative Session to earmark 20% of the state's Enhanced Mineral Income Trust Fund for development of government facilities in the area was defeated, but debate over the proposal focused more attention on the needs of the area.  One result of this debate was the realization that more detailed information was needed on the real impact that lignite coal development would have on governmental services in the area.  

Once an area of study has been identified, the next step is to select appropriate variables for study.  The sequence of events that might lead to an increased need for school facilities in the region is as follows: industrial growth produces an increase in population, which leads to an increase in school enrollment, which may necessitate increased school facilities.

Variables of interest may be broken down into the following broad categories: 1) indices of industrial growth, particularly employment data; 2) population data on a ten year annual basis for the municipalities and parishes in the region, broken down by age groups; 3) school enrollment data by parish for the past ten years; 4) current number of students and classroom teachers for all schools in the five parishes; 5) current number of schools and size of sites plus number of classrooms and conditions of school' buildings; 6) current information on school transportation systems in the parishes, including the number of buses and the number of students transported annually; and 7) current plans for expansion by local school boards plus information about the indebtedness and fiscal condition of each school district.  

Once the variables have been selected, the next stage in monitoring is to develop a database.  This involves locating the appropriate sources for the data and determining the number of years of data that one needs.  Renfro (1980) proposed that the planner have as many years of historical data as the time horizon of the study.  Since the impact of the lignite coal resource development will be felt most severely in the next ten years, ten years of historical data will be required for most of the variables.  Data more than ten years old will have marginal utility for the study.

Sources of data for the industrial indices include: 1) the Federal Economic Development Agency (EDA) for the region; 2) the Louisiana Association of Business and Industry; and 3) annual business data for the region compiled by James R. Michael of the Louisiana Tech University College of Administration and Business.  Additional information on the industries that have applied for permits for lignite exploration may be obtained through the Department of Natural Resources Geological Survey team.  This information was gathered as part of Federal reporting requirements for Section 601 of the Powerplant and Industrial Fuel Use Act of 1978 mentioned above.  These data include estimates of the number of construction, operations and secondary employees currently in the area plus projections of the number of employees that will enter the area over the next fifteen years.  

Population data from the 1980 United States Census is available through the Louisiana State Planning Office; and annual estimates of the population for each year from 1970 to 1982 are available on the parish and municipality level from Louisiana Tech University.  These estimates will provide the most accurate picture of annual population growth in the region for the past ten to fifteen years.

School enrollment figures, by parish, are available through the Bureau of Research at the LSDE for the past fifteen years.  The data for the past three years are available on computer tape; and the data for previous years could easily be computerized.  The current number of students and classroom teachers for all schools in the five parishes will be available through Principal Summary Session Reports, sent annually to the Bureau of Research.  Information on the number of Classrooms and condition of buildings is available through Part II-A of the Annual Financial and Statistical Report.  This part of the annual report has been expanded for the 1981-82 school year, such that data for that year will be the most thorough ever collected by the LSDE.  

Information on school transportation systems is also collected as part of the Bureau of Research's Annual Report process.  Plans for expansion of existing facilities can be obtained by a special survey of the school board in each parish.  Extensive information about the indebtedness and fiscal condition of each school district is available through Part II-A of the Annual Report.  Four years of this data is on computer tape; the variables of interest could be easily computerized for previous years.  

Thus, the information required for the monitoring stage of the project can be easily obtained and entered into a computerized database.  The state education planner can collect and organize this data on a regional basis for the local school boards and state legislators more thoroughly and accurately than they could themselves.  This database will then be extensively used in the second stage of the policy impact analysis-, model-forecasting.  

Stage 2: Forecasting  

It is possible to categorize forecasting techniques as exploratory or normative.  Exploratory techniques are used in Stage 2 to analyze trends and cycles to determine the most likely future developments.  Normative techniques are used in Stage 3 (goal setting) to define the most desirable future developments and help identify a course of action that should lead to them.  In the northwest Louisiana example, the purpose of the forecasting stage is to make the most accurate projections of school enrollment for each of the five parishes over a ten year period (say, 1982-92).  These enrollment projections would then be used in the goal setting stage to help define the most desirable future for each parish with regard to additional school facilities development.  

Both quantitative and qualitative techniques will be used in Stage 2 to define the expected future most accurately.  Quantitative techniques employ mathematical models to predict the future, while qualitative techniques use judgments by experts to predict the future.  Various quantitative mathematical models, each with a different set of assumptions, may be used to project enrollment figures.  It will be necessary to check these projections, however, against expert opinion before proceeding to the goal-setting stage.  It may be that the assumptions of the underlying mathematical models do not conform to the actual situation in each parish.  For example, a mathematical model using a linear projection technique may be appropriate for Nathcitoches Parish, but inappropriate for Red River Parish, where extensive construction operations will have an immediate impact on school enrollment that radically alters the linear pattern.  Expert opinion regarding the likelihood and impact of extraordinary events can be used to alter the mechanical projections made by quantitative techniques.  

The quantitative techniques that may be used to project school enrollments in northwest Louisiana may be either pattern-based models (time-series models) or casual models.  Pattern-based models vary in complexity from simple averaging and ratio methods to cohort survival, Markov-chain, and various curve fitting formulas.  Causal models develop projections on the basis of the numerical values of indicator variables which have been demonstrated through multiple regression to have a strong relationship to enrollment levels.  In the northwest Louisiana example, these variables would most probably be measures of industrial growth attributable to lignite coal resource development in the area.  Unfortunately, indicators of industrial growth associated with this lignite development are tentative at this time, primarily because no coal will be mined until 1984.  Some indicators of industrial growth have appeared in the area, but preliminary examination of the relationship between these indicators and enrollment levels indicate that the relationship is not strong enough at this time to warrant the development of detailed causal models.  

Instead, it seems more appropriate to examine pattern-based models as they are presently being employed in Louisiana to project school enrollments and to determine how these models may be improved.  The Board of Regents and the Louisiana State University Division of Institutional Research currently use cohort survival techniques to project elementary and secondary enrollments.  These projections are then used to project numbers of freshmen entering colleges and universities in the state.  This method is adequate for projecting the size of incoming freshmen classes, but does not accurately project elementary and secondary enrollments, particularly in areas where extensive in- or out-migration occurs.  For example, previous projections of school enrollment by the Board of Regents in some of the parishes in the lignite-impacted area have indicated enrollment decreases, whereas enrollments have actually increased.  

An alternative model has been developed by the Department of Natural Resources (DNR).  This model projects the number of new students entering the area directly as a result of construction, operation, and support services for the coal mines.  Under Section 601 reporting requirements, DNR gathered aggregate estimates of the number of employees that will be needed by the mines in either construction, operation, or support capacities.  DNR then calculated how many of these employees would migrate into the area during a three year period (1981-83).  Estimates of the age distribution of children for the in-migrating workers with families present were then made.  These calculations allowed DNR to estimate the in-migrating school-aged population for this three year period.  Unfortunately, these predictions have not proven to be very accurate, primarily because of delays in the schedules for constructing and operating the mines.  

To summarize, then, the cohort survival technique method of forecasting school enrollments is inadequate because it does not allow for the effect of in-migration, while the DNR technique has failed thus far because of faulty industrial-estimates of the number of in-migrating workers.  However, several alternative pattern-based techniques are available to the state education planner.  The planner must choose from among these techniques the one most appropriate for each parish.  At present, the most attractive alternative may be double exponential smoothing (Gardner, 1980).  This time-series technique allows one to selectively weight historical data points on the basis of the assumed relative importance of more recent information.  For example, the enrollment figures in Red River Parish could be collected for the last twelve years (since 1970).  A formula could then be derived to predict enrollment for the next three years based on the twelve-year trend with heavier weights given to the enrollment figures for the last years of the historical data.  As more in-migrating students enter schools and elevate recent enrollment figures, projections for the future will increase at a greater rate using this technique.  

Once a quantitative model has been used to project enrollments for a given period of time, the projections should be checked against qualitative forecasts generated by experts from each parish.  The delphi technique could be used to develop a combined forecast of enrollment for each parish based on the opinions of a group of experts within those parishes.  Thus, the double exponential smoothing technique could be utilized and then separately checked in each parish using the delphi technique.  If there are discrepancies between the quantitative and qualitative estimates, then a set of alternative enrollment projections (high, medium, low) based on alternative assumptions could be developed for the policy makers to utilize when setting their goals.  

Stage 3: Goal Setting

In the goal-setting stage of the policy-impact analysis model, likely futures derived from exploratory forecasting are converted into desirable futures through normative forecasting.  For example, let us suppose that the state education planner forecast an increase of 500 students in De Soto Parish for the 1982-84 period and also knew that the current school facilities in De Soto were at maximum capacity.  The likely future is that the schools would be severely overcrowded by 1984; the desirable future might be that a new K-12 school be constructed by 1984.  If this were the desirable future, the specific goals could be set up for the following two years to accomplish the desired future by 1984.  

Normative forecasting techniques are, by necessity, decision-making procedures.  The decision to construct a new school must be made by the local school board and superintendent with input from others concerned-such as state representatives and local citizens.  It is at this point that the sensitive nature of the state education planner's role in policy-impact analysis becomes apparent.  

In the monitoring and forecasting stages, the state education planner organized data and developed likely forecasts to meet information requests from local school boards and legislators.  As noted above, the planner is the doer during these stages.  When goal setting and policy analysis/implementation begin, however, the planner must assume the role of facilitator.  The state education planner provides likely futures based on quantitative/qualitative methods, but it is the local policy makers who must define their desirable future and set goals for making that future happen.  

In the northwest Louisiana example, the state education planner would first report back to the school boards and legislators who had requested information.  The planner would present to each parish the likely future for that parish and for other parishes in the region.  During these separate meetings, the local school boards might see the need for a coordinated approach to education planning and budgeting.  They might then ask the planner to coordinate a meeting of all affected authorities, including members of the school board from each parish, the school superintendents of each parish and local state legislators.  It is incumbent upon the local school boards to request the state education planner to assume this facilitating role; the planner should not assume the role without the request of the local authorities.  

In the first joint meeting of these local school representatives, the state education planner should provide matrices of likely futures for the region.  One such matrix might have parish on one dimension and high/low enrollment projections on the other dimension.  This matrix would thus have ten cells that could be collapsed into five high enrollment futures or five low enrollment futures, which in turn could be further collapsed into one high enrollment or one low enrollment future for the region as a whole.  Of course this matrix would be accompanied by other relevant quantitative/qualitative information such as current condition of buildings, current maximum capacity of buildings, school transportation patterns, plans for expansion, and fiscal conditions in all the school districts. 

The state education planner would, therefore, provide the policy makers with exploratory forecasts and aid them in generating future scenarios.  The representatives from each parish would, of course, come into the meeting with some general idea of a desirable future for their parish.  These desirable futures could be combined and restated using delphi procedures.  For example, the parishes might generate a common desirable future that included earmarking 10% of the state's Enhanced Mineral Income Trust Fund for school construction in their region.  Planning for the most efficient and timely use of the funds could be accomplished by considering each parish's desirable future and compromising on a future most desirable for the region as a whole.  

This process of normative forecasting will necessarily involve a great deal of discussion, debate, and several rounds of forecasting in a delphi-like process.  During this process, the state planner will serve at the pleasure of the local representatives.  The planner may be asked to further facilitate the discussion or may be asked to leave as the policy decisions are being made.  

Stage 4: Policy Analysis and Implementation  

According to Renfro (1980), the fourth stage seeks to determine which policy options and responses are available to decision makers to transform the most probable future into the most desirable future.  The initial task in this stage is to identify those events that could have positive or negative effects on trends related to the goals set.  Once these positive or negative events have been identified, policies designed to bring about the positive events or decrease the probability of the negative events can be implemented. 

Again, the role of the state education planner is a sensitive one.  The planner is not a policy maker, yet the local school boards or legislators may continue to seek technical assistance from the planner.  The major contribution that the planner can make at this point is to aid the policy makers in assessing the impact that potential policies may have on events that will affect trends.  

The planner should take each potential policy and set up a policy-to-event impact matrix.  This procedure may become very complex, but it is absolutely necessary for the policy-impact analysis model to work.  A simple example may make this part of the model easier to understand.  

Assume that only two parishes (Red River and De Soto) were involved in lignite coal resource development.  Further assume that the two local school boards and superintendents met and established two policy options: 1) Red River Parish will pursue the immediate construction of a new elementary school in the western part of the parish across the Red River from De Soto Parish; 2) De Soto Parish will pursue the construction of a new elementary school in three years in the eastern part of the parish across the Red River from Red River Parish.  What impact could these policies have on events that might affect enrollment trends?  

As noted in Figure 2, the immediate construction of a new elementary school in Red River might influence newly arriving construction and operation workers with families to settle in Red River Parish rather than De Soto Parish, which only has plans for a new elementary school.  This would rapidly increase the number of school children in Red River Parish and decrease the number of school children migrating into De Soto Parish.  These events in turn could stimulate a still greater demand for school construction in Red River Parish and severely depress the demand for more school construction in De Soto Parish.  Thus a cooperative effort by the two parishes involving slightly different time-lines for school construction might be needed to produce a balanced situation in the two school systems to accommodate new in-migrating students.

Figure 2

Policy-Event-Trend Matrix

Policies

Events

Trends

(P1) Red River Parish pursues the immediate construction of a new elementary school (E1) Number of students in Red River Parish (T1) Building of future schools in Red River Parish
(P2)    De Soto Parish pursues the construction of a new elementary school in three years (E2) Number of students in De Soto Parish (T2) Building of future schools in De Soto Parish

Note that Figure 2 plots only the effect of P1 on E1, T1, E2, and T2.  If the state education planner is involved in the policy analysis/implementation stage, the planner must create realistic policy-event-trend matrices and then aid the local school boards in evaluating both new and old policies.  Again, the planner must be careful not to overstep the advisor role, which must be confined to helping local authorities create the most realistic policy-event-trend matrices using delphi and cross-impact analysis.  This final stage leads back to the monitoring stage, for which the planner has a primary responsibility.  Moreover, beginning the process of monitoring anew enables the planner to evaluate the effectiveness of policies by comparing actual impacts with those forecasted.  This, of course, requires that the data base be updated and maintained in order to evaluate the forecasts and policies and to add important new trends as they are identified.  Implementation of the model also requires that current and past events be reevaluated, and that probabilistic forecasts be updated in order to enable goals to be redefined and reevaluated.

This process allows the education community a maximum hand in defining its place within the evolving social environment.  To those familiar with long-range planning techniques, the policy-impact analysis model may offer nothing new or startling.  However, this model ties together current forecasting and planning techniques and provides a guide for their systematic use in the policy-making process.  We hope that the example offered here lends substance to our assertion that the policy-impact analysis model is a useful tool for state-level educational planning.

*This article is based on material originally presented at the 1982 annual meeting of the American Educational Research Association.

References

Gardner, D. (1980). Enrollment forecasting with double exponential smoothing: two methods for objective weight factor selection. Portland, OR: Portland State University.

Louisiana State Department of Natural Resources. (1980). Growth management plan for lignite impacted area. Baton Rouge, LA: DNR Geological Survey.

Renfro, W. L. (1980). Policy impact analysis: a step beyond forecasting. World future society bulletin, XIV, 4.