Prepared by
James D. Riggle
Roger R. Stough
Denise Hanchulak
Michal J. McElwain
The Center for Regional Analysis
The Institute of Public Policy
George Mason University
September 1998
EXECUTIVE SUMMARY
Virginia’s Center for Innovative Technology (CIT) is a state-sponsored private, nonprofit organization whose goal is to bolster the economy by helping companies take better advantage of technology. It offers a range of programs, including technology development centers, awards for development and commercialization of technology, centers for the solution of manufacturing problems, small business incubators, and sources of information and strategic advice on trends in technology and markets. CIT works with customer companies either directly or through partner organizations such as universities, regional manufacturing centers, and federal laboratories.
CIT’s current three-year objectives (1998 -2000) are to assist in adding or retaining 225 companies and 7,500 jobs, and to add $250 million to the state’s competitiveness. For the fourth successive year, these economic indicators were measured through an independent study by asking customers of CIT and its partners to report actual results that could be attributed to CIT’s assistance. The purpose of this study is to assess the economic impacts of CIT’s programs and services, and the related impacts of the programs and services of CIT’s partners, in fiscal year (FY) 1998. Levels of customer satisfaction were also measured.
Several principal findings stand out:
CRA has now conducted the first four CIT impact studies, measuring impacts achieved during each year by customers who received services in the current fiscal year and previous years. Essentially the same survey questionnaire has been used for all studies to allow comparisons over time for measures of Jobs, Companies and Competitiveness, and Customer Satisfaction. However, the research team made significant changes in the survey instrument for FY 1998 based on the previous years’ experiences and on discussions with CIT staff, partners, and executives of customer companies to improve the accuracy of the survey document and the findings of the study.
In addition to measuring impacts in the three key areas of Jobs, Companies, and Competitiveness, the CRA team assessed Customer Satisfaction, another important concern for outcome-based management strategies.
Study Results
10,609 Net Jobs Added or Retained
(Target for 1998: 2,500 Jobs)
CIT customer companies attributed the addition of 3,213 jobs in Virginia to their involvement with CIT programs in FY 1998, at average wages of $38,900 per job. The companies also reported 7,824 jobs retained with average pay of $33,100 per job as a result of CIT assistance. Firms reported eliminating 428 jobs. Thus, a net total of 10,609 jobs are attributable to CIT activities during FY 1998.
132 Companies Started, Attracted, Converted, or Retained
(Target for 1998, 75 Companies)
Sixty-seven customer companies reported that their companies were started during FY 1998 as a result of CIT programs or assistance. Another 65 companies reported that CIT helped attract them to Virginia, begin converting them from defense to commercial products or markets, or keep them in Virginia when they otherwise would have moved.
More than $1.9 Billion in Competitiveness
(Target for 1998, $85 Million)
Several questions in the survey are used to obtain measures of competitiveness. Accounts of increased capital and sales of products and services during the just-completed year, however, are the most significant and current indicators of increased competitiveness. Companies reported net annual sales increases attributable to work with CIT of slightly more than $200 million, compared with $48.4 million in FY 1997. Even more noteworthy was the amount of capital raised during FY 1998, which can be expected to lead to future increases in productivity and competitiveness. Companies reported raising more than $1.6 billion in private capital and $53.9 million in federal government funding.
Customer Satisfaction
Customer satisfaction measures reflect a very high degree of satisfaction with CIT and its partners. The overall satisfaction rating was 4.3 on a scale of 1 to 5, with 5 being the highest. Even more significant for CIT’s future, 97.6 percent of company contacts said that they would consult a CIT program in the future—a higher percentage than the three previous years (87 percent in FY 1995, 93.8 percent in FY1996, and 94.2 percent in FY 1997).
CIT’s Partners
As noted above, CIT works with partner organizations throughout
Virginia to accomplish its goals. A list of CIT’s partners can be
found in Appendix C of the full Summary Report.
CONTENTS
Section Page
EXECUTIVE SUMMARY i
INTRODUCTION 1
METHODOLOGY 2
FINDINGS 5
CONCLUSIONS, IMPLICATIONS, AND RECOMMENDATIONS 12
Appendix A: DATA FROM FY 1998 ANNUAL SURVEY
OF CIT CUSTOMERS
18
Appendix B: TRANSMITTAL LETTER FROM PRESIDENT
AND SURVEY INSTRUMENT
27
Appendix C: LIST OF CIT’S PARTNERS
28
INTRODUCTION
The purpose of the study is to measure the FY 1998 economic impacts
of CIT’s programs and services, and the related impacts of the programs
and services of CIT’s partners. Customer satisfaction was also measured.
The FY 1998 impact study also marks the beginning of CIT’s work under its
second strategic plan in a series of successive three-year strategic plans
instituted under the direction of Robert G. Templin Jr., and its efforts
as a lead agency for implementation of Virginia’s Blueprint for Technology-Based
Economic Growth.
CIT is a state-sponsored private, nonprofit organization whose goal is to strengthen Virginia’s economy by helping companies take better advantage of technology. It offers a range of programs and services, including technology development centers, awards for development and commercialization of technology, centers for the solution of manufacturing problems, small business incubators, and sources of information and strategic advice on trends in technology and markets. CIT works with customer companies either directly or through partner organizations such as universities, regional manufacturing centers, and federal laboratories. Partnerships are used by CIT to leverage its resources, and thus the effects of CIT’s partners’ programs and services are also included in the study.
CIT’s current three-year goals (FY 1998-2000) are to start or retain 225 companies and add or retain 7,500 jobs, and to add $250 million to the state’s competitiveness. For the fourth consecutive year, these measures were assessed through an independent study by asking customers of CIT and its partners to report actual results that could be attributed to assistance obtained through CIT programs and services. In addition to measuring impacts in the three key areas of Jobs, Companies, and Competitiveness, the CRA team assessed Customer Satisfaction, another important concern for outcome-based management strategies.
Virginia’s Blueprint for Technology-Based Economic Growth
The recognition of advanced technology as a driving force in Virginia’s
economy, and the need to support the business and industry sectors that
are becoming increasingly reliant on technology application and development
prompted leaders of the business community to hold two statewide “technology
summits” in 1997. The result of the summits is a consensus on the
actions that are needed to sustain Virginia’s recent successes and maintain
its leadership position in the highly competitive technology industry sectors.
Virginia’s Blueprint for Technology-Based Economic Growth is the product
of that consensus.
The Blueprint explicitly acknowledges Virginia’s opportunities and needs in several areas: developing and expanding a highly skilled workforce; creating the hard and soft infrastructure needed to compete in a global knowledge-based economy; nuturing a supportive entrepreneurial evironment in Virginia; and accelerating the deployment of manufacturing and information technology.
CIT is a key agent in the implementation strategy for Virginia’s Technology Blueprint. This report documents CIT’s first-year efforts under its most recent strategic plan—one crafted with the goals of the Blueprint clearly in sight.
Independent Review
CIT is believed to be the first state technology program to undergo
a comprehensive independent review of its economic impacts. These
assessments are carried out by independent organizations through customer
surveys. Battelle Memorial Institute in 1995 estimated the fiscal
year 1995 impacts of projects carried out during FY 1988-1994.
The Center for Regional Analysis (CRA) in The Institute of Public Policy
(TIPP) at George Mason University surveyed customer companies that participated
in projects in FY 1995, FY 1996, and FY 1997 to estimate economic impacts
that occurred during those and previous fiscal years.
CRA is also responsible for the 1998 CIT impact study of customers who received services in FY 1998 and earlier to determine the FY 1998 economic impacts of CIT programs and assistance. Essentially the same survey questionnaire was used in 1998 as in the FY 1995, 1996 and 1997 studies to allow comparisons over time, for measures of Jobs, Companies and Competitiveness and Customer Satisfaction. However, the research team made significant changes to improve the accuracy and validity of the data based on the previous years’ experience and on discussions with executives of CIT customer companies. The survey was also shortened from previous versions.
This report reflects results from 515 survey responses received from CIT’s current and previous years customer companies between early June 1998 and August 19, 1998. CRA researchers contacted all of the more than 720 FY 1998 customer companies, and obtained a 44 percent response rate. In addition 199 of the previous years’ customer companies responded.
FINDINGS
Company Profiles
The responses received from CIT customers over the course of four years
demonstrate increasing diversity across most descriptive measures of the
customer companies. During the research for the earliest surveys
the research team received complaints and criticism that CIT focused principally
on large, established companies. The survey findings in 1995 gave
some credibility to those criticisms along two dimensions. First,
very few small, young companies responded to the survey. Second,
the economic impacts that were reported tended to be concentrated among
a few large companies. As numbers of company impacts and the numbers
of companies reporting job impacts indicate, those early trends have been
almost completely mitigated. The following figures provide a more
current view of the Virginia companies served by CIT and its partners.
The reported results are based on responses of 515 Customer Companies, including both current year customers (FY 1998) and customers from previous years (FY 1991-1997). However, the findings reported here indicate results that occurred during FY 1998 only. These numbers reflect actual reported figures, with the exception of the discussion of the indirect and induced effects of jobs and wages. The actual numbers of Jobs, Companies, and Competitiveness that could be attributed to CIT may be higher than the findings of this study, since only about 44 percent of the FY 1998 customer base responded. However, once the entries on CIT’s customer lists are revised and repaired it is likely that the FY 1998 responses will represent a large majority of all actual CIT and Partner customers, and that most annual impacts have in fact been identified.
Jobs
A principal indicator of CIT’s efforts under its 1998-2000 strategic
plan—as was the case in the 1995-1997 plan—is the number of jobs added
or retained as a result of CIT programs and services. The FY 1998 survey
asked clients directly what changes in numbers of employees during the
fiscal year could be attributed to their work with CIT. It
also asked for information on average annual salaries per job, which was
provided for more than three-fourths of the jobs reported. As a result
of having an average salary estimate based on a high rate of response from
FY 1998 clients, CRA researchers are confident that the average wage reported
is likely to be representative of all the jobs created or retained by CIT
clients in FY 1998.
CIT client companies attributed the addition of 3,213 jobs in Virginia, at an average salary of about $38,900 per year per job, to involvement with CIT programs in FY 1998. The companies also reported 7,824 jobs, with average annual salaries and wages of about $33,100, retained as a result of CIT assistance. Companies reported eliminating 428 jobs. Thus, a net total of 10,609 jobs are attributable to CIT activities during FY 1998. The jobs target established in the current CIT strategic plan was 2,500 direct jobs. These numbers (10,609) reflect direct jobs as actually reported by companies. Figure 8 compares CIT’s impacts in creating and retaining jobs with the corresponding targets for fiscal years 1995, 1996 and 1997.
Companies
CIT was credited with helping start, attract, retain, or convert 132
companies—exceeding the FY 1998 Company target of 75 companies. Of
those companies, 67 reported that their companies were started as a result
of CIT programs or assistance, a number equal to about 13 percent of all
respondents. Another 65 companies answered yes to the other Company questions
about being attracted to or retained in Virginia, or converted from defense
contracting. These numbers are significantly higher than the results
from 1995, 1996, and 1997. Figure 9 compares the reported results
with CIT’s target for companies in fiscal years 1995, 1996, 1997, and 1998.
These findings can be explained in part by looking at the company profile data. Slightly more than 48 percent of companies reported that they had between one and ten employees in FY 1998, compared to more than half (52.5 percent) of the respondents in 1997. This small size range should be expected for start-up firms and young, established companies. Only about one-third of FY 1995 respondents, and slightly less than half of FY 1996 respondents had 10 or fewer employees. Figure 10 illustrates the differences in company size by number of employees.
In FY 1998, only 22.6 percent of the firms responding said they had been in operation for less than three years, compared to 28 percent in FY 1997; 57 percent of the firms in FY 1998 reported having been in business for more than six years, compared to 55 percent in 1997; in FY 1998, 39 percent said they had been in business for more than 10 years, compared to roughly 36 percent of the companies in 1997. The comparable figures for the youngest firms in 1995 and 1996 were 19 percent and 17 respectively; for firms in the middle age range the percentages were 67 percent in FY 1995 and 1996; and for the oldest firms the previous year figures were 48 percent in FY 1995 and 41 percent in FY 1996. Figures11 and 12 illustrate the differences in age among companies in all four studies.
The relatively high percentages of younger, smaller firms in the client population suggest that CIT staff continued to emphasize company start-ups in FY 1998, or that the demand for start-up assistance is still on the rise. At the same time, the mix of middle-aged and older firms would be expected to yield a number of companies attracted, retained, or converted from defense contracting, because these companies have been in business long enough to have faced decisions about moving, merging, or dealing with external changes in the economy and government contracting. It should also be noted that the number of responses from previous-year companies increased to 199 in FY 1998 from about 120 in FY 1997. The larger number of past customers who responded to the survey could explain the lower percentages of young and small companies between FY 1997 and FY 1998.
In 1998, 16 completed questionnaires contained multiple yes answers on measures of start-up, attraction, retention, and conversion (compared to 34 multiple responses in 1997). These responses seemed counterintuitive initially as they had in the previous year, and likely owing to errors or a misreading of the question; it seemed highly unlikely for a company to have been started and retained in the same year. Upon checking and validating all the multiple responses, CRA researchers determined that while most responses did involve errors, several other respondents had indeed started their companies and then considered leaving Virginia within as little as three months. One company reported spinning a new commercial firm out of a defense contracting firm, and thereby argued that his multiple response was accurate. They had stayed only because of additional assistance or attention from CIT.
Competitiveness
Many questions in the survey—all acknowledged as appropriate by prominent
Virginia scholars and corporate leaders—focused on measures of Competitiveness.
The survey asked respondents about changes in sales, capital investment,
and capital expenditures, patents, and cost savings. Among the reported
results of respondents’ involvement with CIT are the following:
Changes in Sales:
These various measures for Competitiveness total more than $2.3
billion. However, some of the most widely recognized and useful
measures of Competitiveness are private and public capital raised and increases
in sales. Reported results for these two measures alone totaled more
than $1.8 billion, far higher than the FY 1998 Competitiveness target of
$85 million. Figure 13 shows the Competitiveness impacts achieved
as a result of help from CIT and its partners in fiscal years 1995, 1996,
1997, and 1998 with the corresponding targets for comparison.
The Competitiveness measures reported for FY 1998 are particularly noteworthy for two reasons. First, as in the case of Job and Company impacts, the number of firms reporting positive impacts continues to increase. For example, 67 firms reported raising private capital, 45 received federal grants or contracts, and 39 received patents. Moreover, while the number of patents issued in FY 1997 was significantly higher, they were issued to only 24 firms. The second point to consider with respect to FY 1998 Competitiveness impacts is that the contact person at the firm which raised more than $1.5 billion in new private capital was completely comfortable in attributing part of the credit for the company’s success to CIT and its respective Partner. In that particular case the firm has worked with CIT and Partner’s programs and services for several years, and the FY 1998 impact was an example of the “delayed” impacts cited as a justification for surveying past customers.
Customer Satisfaction
CIT continues to enjoy very high levels of customer satisfaction, higher
across all measures in FY 1998 than in any previous year. This fact
is borne out by two measures: a direct question about the degree of satisfaction
and a query about the respondents’ willingness to consult CIT again in
the future. For FY 1998 the average overall satisfaction rating was
4.3, on a scale of 1 to 5 where 5 is the highest. Moreover, 97.6 percent
of CIT clients would consult CIT in the future, and only 2.4 percent would
not. The opportunity for repeat business is especially important
because many technology-rich projects take several years from conception
to commercialization, as demonstrated by the results of the FY 1998 study.
The vast majority of CIT customers are apparently interested in maintaining
their relationships with CIT until the payoffs occur. Even some respondents
who reported being dissatisfied with their FY 1997 experience with CIT
said they would, nonetheless, work with CIT again.
Satisfaction ratings are most telling when combined with customers’ other assessments of CIT’s service. Respondents indicated that CIT’s help in meeting the company’s expectations rated a 4.25 average score on a scale of 1 to 5. CIT staff’s adherence to schedules for delivery of services rated a 4.4 on a scale of 1 to 5. Notwithstanding the fact that bringing products and services to market is the expectation most frequently cited by CIT customers, CIT chose to drop the question assessing the degree to which their programs and services satisfactorily yielded commercial results in FY 1998. The near-total (over 97 percent) willingness on the part of its clients to continue to do business with CIT should assure a stable customer base in the future. It is worth emphasizing even more strongly than in previous years that this supply of companies offers the opportunity to reduce the emphasis on increasing the number of CIT customers, and instead allow CIT’s staff to focus more on the outcomes of their caseloads, thereby deepening and strengthening the services they provide.
Indirect and Induced Effects on Virginia’s Economy
Direct jobs are full-time or full-time equivalent positions companies
added or retained as a result of their involvement with CIT’s programs.
The 3,213 jobs added and 7,824 jobs retained during FY 1998 are direct
jobs attributable to companies’ work with CIT and its partners. Indirect
jobs and income effects are estimates of the effects of CIT customers’
interaction with other companies and sectors of the economy, purchasing
inputs to increase production as a result of their new or retained employees.
Induced effects are the effects of households’ spending of the wages from
both direct and indirect jobs.
In 1998 the overwhelming majority of respondents who reported adding or retaining jobs also reported wage and salary data for those jobs. Therefore indirect and induced effects can be calculated for both jobs and income. These measures are estimates of the impacts of CIT programs and services on the entire Virginia economy.
The 10,609 jobs attributable to CIT in FY 1998 should generate 24,825 indirect and induced jobs. The total job effect—including direct, indirect, and induced employment—for FY 1998 is estimated to be 34,434 jobs.
The average salary or wages of the 3,213 jobs added by CIT customers in FY 1998 was about $38,900, resulting in direct income of $124,985,700, and additional income of $238,722,687. The total income effect of the jobs added in FY 1998 is $363,708,387.
Likewise, the average wages for the 7,824 retained jobs was about $33,100, resulting in direct income of $258,974,400, and additional of $494,641,104. The total income effect of jobs retained in FY 1998 is $753,615,504. The combined income effect of jobs added and retained in FY 1998 is $1,117,323,891.
To determine the net income effect of CIT’s jobs impact for FY 1998 the income from the 428 jobs eliminated must be subtracted from the total above. The average wage for the jobs eliminated was $36,000, resulting in a loss of $15,408,000 in direct income, and $36,054,720 in indirect and induced income. The total reduction in income due to jobs eliminated in FY 1998 was $51,462,720.
Thus, the net income effect of jobs added and retained as a result of CIT’s programs and services in FY 1998 is $1,065,861,171.
Summary
In summary the findings indicate that overall, CIT exceeded all of
the economic impact targets for Jobs, Companies, and Competitiveness set
for FY 1998 and surpassed the rankings for Customer Satisfaction, for all
previous years. The results of the FY 1998 impact study CIT surpassed
its three-year goals for Jobs, and Competitiveness.
CIT’s annual state appropriation for FY 1998 was slightly more than $11 million. Total economic impacts attributable to CIT programs and services during the fiscal year were more than $2.3 billion. Added to this figure is more than $1 billion in personal income received by the citizens of the Commonwealth from direct, indirect and induced employment attributable to CIT’s and its partners’ efforts during the same period.
The long-term effects of CIT’s programs and services is made clear by answers to survey questions and comments from survey respondents. Several respondents indicated that although they were unable to cite measurable economic impacts or indications of improved competitiveness or efficiency during the time period of the study, they expected that benefits would occur in the future. These specific questions were added to the impact survey for the first time in FY 1997, and the responses seem to be borne out by the results in 1998. In the FY 1995 and 1996 studies respondents made similar points (leading to the modifications in ther 1997 survey), suggesting that many CIT clients are engaging in projects that ultimately may provide significant economic benefits but would not be undertaken in a purely market-driven environment because private investors and capital markets demand a quicker payoff.
From a procedural standpoint, the FY 1998 impact survey was more efficient and productive than the FY 1995, 1996, and 1997 efforts. Only about half of the initial mailing of questionnaires were deliverable by the U.S. Postal Service in 1995, whereas more than 90 percent were delivered in 1996, 1997, and 1998. The number of responses in FY 1998 was almost 20 percent higher than the previous year, and the number of FY 1996 responses was more than twice as large as the number received for FY 1995. Most importantly, however, the FY 1998 study was better planned and was completed several weeks faster than earlier studies, thereby providing quicker feedback to CIT and more time for CIT to act on the study’s findings.
The CIT impact survey process remains labor-intensive, requiring individual telephone contacts with most customers to obtain responses. The FY 1998 study included sending e-mail surveys to all CIT customers with recorded e-mail addresses. Slightly more than 30 e-mail surveys were returned, about 7 percent of the total sent—and of the total number of survey responses. The relatively low level of responses is not altogether surprising. Recent experiences and research on technology-rich sectors of Virginia’s economy indicate that the use of e-mail and the Internet for business purposes is limited to a minority—albeit--- a growing one—of all companies. While no systematic study has been conducted to track the number of CIT customers who ask to reply by e-mail each year, the number is growing. There is no reason to assume that CIT customers are any different than other companies, with respect to their use of information technology.
As discussed elsewhere in the report, lack of e-mail responses from CIT customers to the survey is not surprising when compared to the use of e-mail and the Internet by other firms. “Pushing” the use of this and similar technologies is relevant to CIT’s mission. Since pushing the use of technology is seen as a desirable and important outcome of CIT’s efforts, then efforts should continue to identify options and to devise a more effective approach aimed at moving toward increasing the number of annual impact surveys that are answered electronically. CIT managers should acknowledge that the low e-mail response rate is principally an implementation issue, notwithstanding minor technical problems with the 1998 effort.
One of the most noteworthy changes for FY 1998 was the modification of the survey instrument itself, along several dimensions. The accuracy of the responses improved, and not one complaint about the length of the survey was received in 1998. Moreover, as noted previously, the number of responses exceeded 500 for the first time.
In closing the FY 1998 Summary Report it is worth noting that all CIT annual impact studies through FY 1998 have been conducted during a sustained period of almost unfettered economic growth. However, 1998 was a year of serious economic downturns in much of Asia, and economic instability across the globe. It remains to be seen whether, and how, Virginia’s economy is affected by global instability and slowed economic growth. Subsequent studies may reflect international economic change.
Appendix A
DATA FROM ANNUAL SURVEY
OF CIT CUSTOMERS
PART I. CLIENT COMPANY PROFILES
Company Size
Current Number of Employees:
1-10
48.2% (240)
11-25
15.3% (76)
26-100
20.0% (100)
101-500
10.4% (52)
501-1000
2.6% (13)
1001-2000 1.2%
(6)
2001 +
2.2% (11)
R&D Percentage of Annual Budget
< 1%
28.3% (134)
1%-5%
25.2% (119)
6%-10%
14.0% (66)
11%-50%
17.2% (81)
> 50%
15.3% (72)
Current Annual Sales
< $1 million
47.7% (227)
$1-10 million 32.1%
(153)
$11-100 million 15.3% (73)
> $100 million 4.8%
(23)
Years in Operation
< 3 years
22.6% (112)
6-10 years 18.0% (89)
3-5 years 20.2%
(100)
> 10 years 39.2% (194)
Industry Sector
PART II. INVOLVEMENT WITH CIT PROGRAMS AND SERVICES
This section of the survey was significantly restructured from the
FY 1995 questionnaire, based on responses in the previous year and on discussions
between CRA and CIT staff. Its content is essentially the same, however.
Expectation
Services
PART III. COMPANY BENEFITS (ECONOMIC IMPACT)
Company Start-up, Attraction, Retention, or Conversion
Did CIT assist with the start-up of your company?
Yes: 67 companies
Did CIT assist with the attraction of your company
to Virginia? Yes: 8 companies
Did CIT assist with the retention of your company
in Virginia? Yes: 36 companies
Did CIT assist with the conversion (from defense
to commerce) of your company? Yes: 21 companies
Jobs Added and Jobs Retained
Number of jobs added: 3,213 Avg.
salary/job $ 38,900
Number of jobs retained: 7,824
Avg. salary/job $ 33,100
Number of jobs eliminated:
428
Total direct employment: 10,609
Change in Sales
Sales retained:
43 firms $
195,888,000
Avg. annual increase:
126 firms $
217,318,859
Avg. annual decrease:
9 firms
$ 17,135,000
Net annual increase:
$ 200,183,859
Capital/Funding Secured
Private capital raised:
67 firms $ 1,682,551,302
Value of federal contracts:
45 firms $
53,926,073
Net value of capital secured:
$ 1,736,477,375
Capital Expenditures
New Capital Expenditures: 176 firms
$ 1,890,466,584
Sale of Technology or Market Rights
Number of Patents Issued:
95
Value of License/Rights Sold:
$ 102,500
Cost Savings
$ 14,707,400
PART IV. COMPANY ASSESSMENT
Customer Satisfaction
The first question asked, “Overall, how satisfied were you with the
services you received from CIT or its Partners?”
Very Dissatisfied Very Satisfied
1
2
3
4
5
0.7%
2.7%
13.6%
31.9%
51.2%
(2)
(8)
(41)
(96)
(154)
Average = 4.3
Other Measures of Customer Satisfaction
The second satisfaction measure focused on the opportunity for repeat
business. The question asked, “Based on your past experience, would you
consult with CIT in the future?”
YES
NO
97.6%
2.4%
(291)
(7)
The following two questions were substantially revised from the FY 1995 survey to assess customers’ opinions about CIT’s performance in meeting the expectations for individual projects.
Not at all Fully
Meet your expectations?
1
2
3
4
5
Average = 4.25
1.7% 3.0%
12.6% 33.9%
48.8%
(5) (9)
(38) (102)
(147)
Did CIT deliver service on
schedule?
0.7% 3.9%
8.5% 26.5%
60.4%
Average = 4.14
(2) (11)
(24) (75)
(171)
Appendix B
Discussion of Survey Methodology
METHODOLOGY
The 1998 study built on the three earlier impact assessments
for FY 1995, 1996, and 1997 by the Center for Regional Analysis.
The questionnaire used by CRA in FY 1998 and the earlier surveys allowed
for the comparison of results for Jobs, Companies and Competitiveness and
Customer Satisfaction from year to year. However, the survey form
was refined in several ways in 1998—as had been done after each earlier
survey—in response to problems identified in previous years by CIT staff,
Partners, and customer companies. Their comments were obtained anecdotally
and through focus groups conducted by CRA and CIT staff.
The FY 1998 study contains data from two groups of CIT clients:
The Survey Instrument
The survey instrument has four parts: Company Profile; Company Involvement
with Programs and Services; Company Results from Programs and Services;
and Company Assessment of Programs and Services. Responses to the
first two parts are shared with CIT for evaluating their customer base
and the focus of their efforts, and for updating customer records.
Responses to Parts III and IV are regarded as confidential information
and are reported only in terms of aggregate statistics.
The questionnaire concentrates on indicators of the economic impacts
of CIT programs and services. The survey findings report data that
include measures and qualitative assessments of the numbers of companies
and jobs created and retained in Virginia, as well as economic competitiveness
(including net increases in sales, patents, operating efficiency, business
expansion, and capital formation). Data indicate which CIT programs
and services clients used most frequently, and the report assesses clients’
opinions of CIT performance through measures of customer satisfaction and
their willingness to do business with CIT in the future. The questionnaire
also elicits information about firms in a variety of sectors of Virginia’s
economy, including company profiles and business activities.
Validity and Reliability Issues
More than 300 customer companies (314, or about 44 percent) of the
722 FY 1998 CIT Customers to whom surveys were mailed responded, as did
199 customers from previous years (about 15 percent of the total 1,343
previous-year firms). Inferences from the data obtained in
the survey will be more accurate than those from the earlier studies because
the response rate was higher and larger percentages of all respondents
reported achieving impacts. Moreover, several of the specific questions
in the fiscal 1997 survey instrument were revised for the third consecutive
year, or eliminated, on the basis of accumulating experience and discussions
with senior executives at CIT, client companies, and CIT field staff.
These modifications to the survey resulted in fewer mistakes and misinterpretations
on the part of the respondents.
Concerns about validity (essentially the accuracy with which survey
questions elicit measures of desired outcomes) in the FY 1998 survey were
most apparent in the questions assessing the numbers of companies started,
attracted, converted, and retained with assistance from CIT. Even
though the wording of this question was changed from the versions used
in all three earlier survey instruments in an effort to correct vagueness
and eliminate multiple responses by individual firms, it is evident that
this question still needs some additional clarification and emphasis.
About 16 responses contained more than one yes response to the Company
impact questions—compared to 34 multiple responses in 1997—although respondents
were instructed to mark no more than one response, and the beginning and
end dates of the fiscal year were included in each question. It would
be misleading to count any individual’s response for the Company impact
measure more than once, even if the respondents answered the questions
accurately. For instance, it seemed unlikely that a company would
have been both started and retained in the same year. However, in
a few cases this turned out to be the case. CRA researchers made
follow-up telephone calls in every case of this kind to verify multiple
entries on returned questionnaires, thereby ensuring accurate recording
and eliminating double counting.
Issues of concern about reliability (the generalizability of the survey
findings to the entire population of CIT customer companies, and whether
the findings would be the same or similar if the survey were repeated)
are not major, but are worth noting for three reasons, all of which deal
with the definition of “customer.” Reliability issues are minor concerns
in part because the survey is focused on a specific population, and is
administered to the entire population thereby eliminating the need to generalize
the results to a larger group using inferential statistics. The researchers
routinely make follow-up telephone calls to verify any questionable or
unreadable survey responses. The concern about the “customer” issue
is that the word does not mean the same thing to people within the specific
population.
First, since the company was chosen as the unit of analysis beginning
with the FY 1996 study, rather than the project, as in the FY 1995 study,
this definition is even more important because it is the basis for determining
whom to ask the questions. As a result of this decision, single companies
did not receive multiple questionnaires as they had in 1995 if more than
one project or service had been provided to the company, sometimes with
multiple points of contact.
Second, CIT’s definition of what constitutes the CIT\customer relationship
should be more precise, since respondents who had only attended conferences
or seminars, or simply exchanged business cards said that they did not
consider their contacts with CIT to be value-adding. It is also possible
that CIT’s partners need to reconcile their definition of a customer with
CIT. In any event, it is important to distinguish mere activities
such as establishing contact with a new company from the much different
relationship with a customer or client.
Third, about 500 potential respondents were either unreachable, or
they did not recognize themselves as CIT customers, insisted they had never
heard of CIT, or denied that they were involved in CIT projects, and declined
to participate in the survey on those bases. The same problems were encountered
in the FY 1995, 1996, and 1997 studies. Further investigation in 1998 revealed
that many of the companies are out of business, or designated points of
contact are no longer at the company. Many of the problematic companies
had originally been placed on the customer mailing list by former CIT employees
and had, in effect, been inherited by their successors. CIT’s customer
mailing list is currently being revised and updated, and the changes should
be expected to result in higher response rate for the FY 1999 study.
These problems notwithstanding, the CRA research again produced more
and better data than in earlier years, and identified some trends that
strengthen the findings of the FY 1995, 1996 and 1997 studies. For
example, the numbers of companies that report positive impacts continues
to increase, and the relative importance of large impacts reported by a
small number of companies continues to decline. Other findings mark
new directions and efforts by CIT field staff, including the introduction
of assistance with electronic commerce, and offering electronic products
and services. The findings continue to point the way to useful modifications
in CIT programs and services, and for similar evaluation studies carried
out in the future.
Appendix C
TRANSMITTAL LETTER FROM PRESIDENT
AND SURVEY INSTRUMENT
Appendix D
List of CIT’s Partners
Entrepreneurship Centers
Central Virginia Entrepreneurship Center-VCU
Technology Resource Alliance-GMU
ODU Entrepreneurship Center
Business/Technology Center-VA Tech
Manufacturing/Regional Partnerships
Manufacturing Technology Center
Technology Application Center-ODU
The Peninsula Advanced Technology Center
Virginia’s A.L. Philpott Manufacturing Extension
Partnership
Special Focus Partnerships
InterCom-UVA
SEVAnet-CNU
The Virginia Scotland Partnership
Virginia Spaceflight Center-ODU
Technology Development Centers
Center for Drug Design, Delivery, and Clinical
Applications-VCU
Center for Semicustom Integrated Systems-UVA
Center for Wireless Telecommunictions-VA Tech
Fiber & Electro-Optics Research Center-VA Tech
The Center for Electrochemical Science &
Engineering-UVA
Virginia Power Electronics Center-VA Tech
University Intellectual Property Offices
George Mason University-GMU
Old Dominion University-ODU
The College of William & Mary-W&M
Virginia Commonwealth University-VCU
Virginia Tech-VA Tech
Partners from Previous Years
Center for Advanced Ceramic Materials-UVA
Center for Bioprocess/Product Development-UVA
Center for Excellence in Command, Control,
Communications, and Intelligence-GMU
Center for Magnetic Bearings-UVA
Coal and Minerals Technology Center-VA Tech
The Institute of Public Policy
The Institute of Public Policy (TIPP), which embraces George Mason University’s graduate teaching, research, and outreach activities in this field, was inaugurated in 1990 to focus the University's commitment on public policy studies and research. The free-standing structure of the Institute and its commitment to interdisciplinary training and research at the doctoral level allows it to reach across the University’s schools and colleges to bring together knowledge and skills needed to address a wide variety of policy concerns. The three areas of major substantive concern in the Institute are public management policy, science and technology policy, and regional development policy. These are complemented by cross-cutting themes in American values and culture as expressed through institutional patterns of governance and a global perspective on international competition and cooperation. Advanced analytical techniques of policy analysis are also emphasized. The Institute works closely with the University scholars in environmental science policy, health policy, and other policy related fields.
With the challenges facing governments all over the world to adapt to and utilize continuing innovations in science and technology, to cooperate internationally and participate in a globally competitive world, and to foster such fundamentally American institutions and values as representative values and individual rights, this Institute places George Mason University in the forefront of public management research and policy analysis.
THE CENTER FOR REGIONAL ANALYSIS
The Center for Regional Analysis is located at The Institute of Public Policy. The Center’s purposes are to conduct research on the Northern Virginia region and the National Capital area, and to provide support for regional leaders through conferences, workshops, strategic planning, leadership development training, and customized research and presentations. The Center is also recognized at the national and international levels for conferences and workshops it sponsors and for research and consultation it provides on regional economic development and related policy. The Center specializes in regional economic modeling, modeling, strategic planning, leadership development training, program evaluation and economic impact assessment.
The Center is supported by The Institute of Public Policy, the Friends
of Northern Virginia Chair in Local Government, and George Mason University,
as well as through grants and contracts from federal, state, local and
foreign sources. Overall, the Center receives about 90 percent of
its $1.5 million annual budget through grants and contracts.