Transcription
PM30 Snippet Series: Earned Value Management System Academic Research Study and Introduction
US Department of Energy, Office of Project Management, Project Controls and Policy Division. An Office of Project Management Snippet is intended to be an abbreviated way to communicate important information in news. This snippet discusses the exciting results of the PM-sponsored Arizona State University EVMS academic research study and the deployment of IP2M meter. We present this light bulb idea, confident that everyone can benefit from it.
Academic research is intended to create knowledge, drive innovation, and address challenging problems. It is designed to be from a voice of authority and can change how people view whatever is being researched.
Sponsored by the Office of Project Management in April 2019 and led by Arizona State University, this multi-year study was designed to identify and explore areas of consensus and conflict on the use of an EVMS to manage a project or program. A steering committee was established to provide overall guidance and review of the research study process. It consisted of 41 subject matter experts from academia, government, and industry. Twelve workshops collected 1,534 responses from 159 project and program management, project controls, business and financial management professionals from over 90 organizations having a diverse range of backgrounds and experiences.
Notable participating organizations committed to furthering the research study include the Arizona State University, the Naval Postgraduate School, the Department of Energy, the Office of the Secretary of Defense, the Defense Contract Management Agency, the National Aeronautics and Space Administration, the National Reconnaissance Office, the Washington River Protection Solutions Company, the Los Alamos National Laboratory, the Lockheed Martin Corporation, the Ron Technologies Corporation, and many more.
All three of the research study hypotheses were proven to be correct:
First, EVMS maturity attributes and environment factors can be defined.
Second, EVMS maturity attributes and environment factors can be measured.
Third, EVMS maturity attributes and environment factors are relational and correlate with project and program performance.
Arguably the most noteworthy finding coming from the research study is that there are statistically significant differences between the performance of projects and programs implementing mature EVMS and those that are less committed. Projects and programs implementing a mature EVMS were found to have more reliable data and deeper insights into performance issues, leading to rational decisions and better outcomes. Researchers found that a range of environment factors, including project and program cultures that are averse to the appropriate use of an EVMS, can increase the risk of cost and schedule growth. The research study's introduction of environment factors and their influence on an EVMS provides a material change from past thinking.
A positive correlation is a relationship between variables that move in tandem; that is, in the same direction. The research identifies the independent variable belonging to the environment on the x-axis and the dependent variable belonging to maturity on the y-axis. The independent variable, environment, is the variable the research manipulates and is assumed to have a direct effect on the dependent variable, maturity. The Pearson correlation coefficient (R) was used to determine the linear correlation between the two variables. For 35 completed projects and programs, the resultant correlation coefficient of .83 shows a strong positive correlation.
The research data are given in the scatterplot diagram shown. Each dot represents one project or program at its 20% completion point and is placed at the point corresponding to the measurement of the environment and maturity. The closer the data points come to forming a straight line when plotted, the higher the correlation between the two variables or the stronger the relationship. For example, the Navy's FA-18F Engineering Manufacturing and Development program is shown as a reference point. Program P9 received an environment score of 7.30 and a maturity score of 6.84.
The recent collection of additional data points and insights is encouraging. These results reaffirm a strong positive correlation between the environment and maturity of an EVMS.
The research defines a project or program environment by four categories and 27 factors. The culture, people, practices, and resources make up these categories. Each category is further defined by factors. The environment refers to the internal and external circumstances and conditions that can influence the performance of a project or program. Most of these circumstances and conditions are influenced by human beliefs, behaviors, and attitudes, including understanding the importance of teamwork, demonstrating an ability to problem-solve and make timely decisions, and being transparent with others, no matter how challenging it might be.
Data collected as part of the research study were statistically analyzed to develop weighted scores as a mechanism for assessing the environment. Of the four categories, culture accounts for the largest percentage of weighted scores with 313 points, or 31% of the total possible 1,000 points. Culture is where an organization's values and beliefs can either enable or hinder implementing an EVMS.
The research defines an EVMS by 10 interrelated management subprocesses and 56 attributes. The organizing, planning and scheduling, budgeting and work authorization, accounting considerations and direct budget and cost management, analysis and management reporting, change control, material management, subcontract management, and risk management subprocesses define an EVMS where the whole functions better than the sum of its parts. Each subprocess is defined by multiple attributes. Mature refers to the degree of integration and effectiveness of each management subprocess to produce reliable data with timely insights into performance issues that lead to rational decisions and better outcomes.
Data collected as part of the research were statistically analyzed to develop weighted scores as a mechanism for assessing maturity. Those participating in the research study overwhelmingly found that planning and scheduling had the biggest impact on an EVMS. It is this subprocess that aims to develop the integrated master schedule, resource requirements, and time phasing for the performance measurement baseline.
The solution directly explains what opportunities there are to solve the area of concern, condition to be improved, or difficulty to be eliminated that is introduced in the research study problem statement. It's the light bulb idea: If a tool could analyze your project or program and predict your chances for success, would you want to know?
Integrated Project/Program Management Maturity and Environment Total Risk Rating, pronounced IP2M meter, quantitatively measures a project's or program's environment and the maturity and effectiveness of an EVMS. IP2M meter environment and maturity assessment scores are plotted against color ranges, providing actionable insights to maximize performance and predictability. Think of it as calculating a project's or program's personalized credit risk score that is used to detect environment and maturity issues and predict how likely performance baseline objectives will be met.
Thinking back to the scatterplot diagram, in the FA-18F program, the IP2M meter environment score of 7.30 and maturity score of 6.84 places it in the upper orange range. The research found that projects and programs in this range are likely to experience upwards of a 48% cost growth and 27% schedule growth without intervention. These are additional costs that are typically not covered by designated contingencies. The actual cost and schedule performance outcomes of the FA-18F program appear to validate the predictive capabilities of IP2M meter and the color range designations of green, yellow, orange, and red.
At the Department of Energy, we believe IP2M meter is a light bulb idea that diverges from traditional implementation and assessment methods by considering a project's or program's environment for determining the maturity and effectiveness of an earned value management system. IP2M meter looks to inform, insight, and empower project and program teams to maximize performance by leveraging the capability of an EVMS. It emphasizes an ethos of teaming between customers, contractors, and stakeholders as a key ingredient for establishing high-performing environments.
The IP2M meter is designed to simplify and democratize an earned value management system to maximize its acceptance and use as a helpful management and decision-making tool. At its core, democratization is all about giving an EVMS back to those in most need of its capabilities: project and program leaders, managers, and functional teams. Simplifying its definition is about making it less complicated and therefore easier to understand and use.
The IP2M meter provides actionable insights by benchmarking a project's or program's environment and the maturity of an EVMS. The good news is that the use of IP2M meter doesn't have to stop after the initial assessment. Projects and programs are encouraged to periodically reassess the environment and maturity of an EVMS to drive continuous improvement that research shows can lead to better performance outcomes.
Right-sizing an EVMS using IP2M meter can match its implementation to workload requirements at the lowest possible cost. IP2M meter allows projects and programs to tailor an EVMS to best support its unique management needs. A lack of EVMS tailoring has been cited as a barrier to its useful implementation and a reason for rising project and program costs.
Reciprocity expands the use of IP2M meter as a best practice across government and industry. IP2M meter is a repeatable and automated management solution that promotes the practice of reciprocity with others for mutual benefit. This last point is in support of the Office of Management and Budget's efforts to reduce the burden of certifying EVMS across the federal government.
The good news is that IP2M meter is online and ready to use. Improving project and program performance is possible, but not through business as usual. Transformative change is needed. Transformative change means doing things differently. The research found that changing the way we think about and implement an EVMS can lead to better performance outcomes. Remember, an earned value management system can position your project or program for success, but only if you let it. An IP2M meter was developed for just this purpose.