World's most popular travel blog for travel bloggers.

Project Management Metrics #1: Schedule and Effort/Cost Variance

The goal of this metric is to measure the performance as well as progress of the project against signed baselines.  This metric is very important and is the base for profitability of project. The EVM (Earned Value Management) concept, as defined by PMI standard PMBOK, is the commonly used method to track this metric. It integrates project scope, cost and schedule measures to help the PM to assess and measure project performance and progress. The principles of EVM can be applied to all projects, in any industry. Under this method, at any given point in time, project performance to date is used to extrapolate the expected costs and duration at project completion. This technique uses past performance (i.e. actuals) to more accurately forecast future performance. EVM develops and monitors three key dimensions of each work package:
Planned Value (PV): How much you planned to spend for the work you planned to do i.e. it is the authorized budget assigned to the work to be accomplished for an activity or work breakdown structure component. Total PV is also known as Budget at Completion (BAC). PV at any stage = (Planned % Complete) X (BAC)
Earned Value (EV): Earned value is the value of work performed expressed in terms of the approved budget assigned to that work for an activity or work breakdown structure component. It is the authorized work that has been completed, against the authorized budget for such completed work i.e. EV is ‘how much you planned to spend for the work you actually did’. Earned Value is also known as the Budgeted Cost of Work Performed (BCWP).
Actual cost (AC): Actual cost is the total cost actually incurred and recorded in accomplishing work performed for an activity or work breakdown structure component. It is the total cost incurred in accomplishing the work that the EV measured. I.e. how much you spent for the work you actually did. Actual Cost is also known as the Actual Cost of Work Performed (ACWP).
Using these three variables project Schedule variance and Cost variance metrics can be derived which shows if the project is running over or under budget; project is running behind or ahead of schedule, as follows:
Schedule Variance (SV) is the measure of schedule performance of the project. It is the difference of Earned value and the planned value i.e.  SV = EV – PV
  • Positive result means that you are ahead of schedule.
  • Negative result means that you are behind schedule.
Cost Variance (CV) is the measure of cost performance on the project. It is equal to earned value (EV) minus actual costs (AC). Any negative CV is often non-recoverable to the project.
CV = EV – AC
  • Positive result means that you are under budget.
  • Negative result means that you are over budget.
EV, PV and Actual Costs
Since EVM method allows PM to extrapolate the expected costs and duration at project completion based on project performance to date, PM can develop a forecast for the estimate at completion (EAC) which may differ from the budget at completion (BAC) based on project performance. Forecasting of EAC involves making estimates or prediction of conditions and events in the project’s future based on information and knowledge available at the time of forecasting. EAC is typically based on actual cost (AC) incurred for work completed, plus an estimate to complete (ETC) the remaining work. I.e. EAC = AC + ETC.
Based on this PM can also derive another metric, Variance at completion (VAC) = BAC – EAC

 

Project Management Metrics #2 – Productivity: Resource Utilization

The objective of this metric is to measure productivity of resources involved in project and let PM assess over or under-utilization cases.
Utilization% = Total Effort spent by resource/Total Budgeted Effort for the resource2
Budgeted effort is the planned billable work of resource. Any over-utilization and under-utilization indicated by this metric has an impact on the project’s profitability. It is important for the PM to track this metric very closely and find out the reason for deviations and the action items to bring back resource utilization to optimal level. Delayed projects, increased ramp up activities, less work provided by customer, unplanned vacations, less competent resources can impact this metric. To get better control over this metric, robust time reporting systems should be available in the organization. Using this, PM can analyze effort distribution across different project phases/activities. For e.g. Effort distribution can tell PM that how much effort is being spent on defect resolution, customer support or design activities. PM can take corrective actions based on this, if required. For instance, if the resource is complaining that customer support is taking considerable time but the effort distribution shows it otherwise, PM can see where the corrections are needed on what resource is doing. Effort distribution from time reporting systems can also tell the areas of improvement for  better estimations/planning for the next project.
 Effort Distribution

Project Management Metrics #3: Change requests to Scope of work

Signed Scope baseline with customer forms the baseline for the entire project planning and development. Any change to signed scope should happen in controlled manner. So here comes another important metric for PM to track i.e. the number of change requests coming from customer for the already signed scope of work. Each and every change request, once approved by internal change control board (CCB), requires update to Scope baseline which in turn has a cascade impact on cost baselines and schedule baselines and resource plans. Uncontrolled change requests often result in project scope creep and further impact negatively on the project cost/schedule, which is the worst thing to happen for any project. PM should never allow such scope creep. Based on the magnitude of the variance from original scope baseline, CCB should decide whether to accept or reject the change request and this decision should be communicated back to customer. In case of acceptance of change request, the impact on project cost and schedule should be clearly communicated in written form to customer and a written agreement from customer secured on those from customer before proceeding.

Project Management Metrics #4: Quality and Customer Satisfaction

Throughout the execution of project, Quality Assurance should always be on the radar of project manager. Quality here is defined as the number of severe, medium or low defects delivered through the lifetime of the project. It indicates the health of the deliverable to the end user and drives the Customer Satisfaction. PM needs to define, based on project type, what severe, low and medium means. Quality should be reported throughout the life of the project; the later defects are caught, the more impact they will have on the project. Under quality metrics, following are the key ones to track:
Defect density = Total number of defects found/ Measure of size.
For e.g. in case of software projects this can be: how many defects are found in 1KLOC (Kilo line of code). In general, size measure can be considered as planned effort like ‘person day total planned effort’.
Defect age
Number of days since the defect is open and not fixed. It can also be inferred as the time customer has been waiting for their issues to get resolved,
Defect resolution rate = Total number of defects resolved/ Total effort spent1
Rate of closing the open defects over a period of time. If the rate of resolution is not in line with the defects being opened over a particular time, this indicates to the PM a situation of concern.
Defects Tracking
Number of defects reported by customer
PM should keep this as a separate metric to differentiate from the defects reported out of internal testing and the defects reported by end user i.e. Customer. Customer satisfaction depends a lot on the quality of deliverable provided and on how fast defects raised by customer are resolved.
As said above, the later defects are caught, the more impact they will have on the project, it is worth to mention here about Pareto’s principle i.e. 80/20 principle, which PM can use to categorize causes of defects and late time entry relationship. As per this law 80% of the problems are due to 20% of the causes. PM can concentrate on these 20% causes impacting the project most.
Pareto Diagram

Project Management Metrics #5: Gross Margin

Gross Margin (as I wrote in my earlier post on key performance metrics) is the mother of all metrics and the quickest way to determine if your business in on track or not and acts as an early warning system to put in place margin improvement initiatives. Ultimate goal of project execution is to bring revenue to organization with the approved gross margin. Gross margin (GM) is basically the difference of total revenue and the total cost spent on project i.e. profit.
When a project is started, certain GM levels for the project are approved by project sponsor. This approved GM value is generally based on project scope definition, duration, a forecast of resources: onsite, offshore and organization’s investment analysis. Project PNL (Profit and Loss) statement gives a way to PM for tracking his/her projects GM metric at any point of time. For this, PNL statements and forecasts should be current documents i.e. changes in project parameters need to be reflected quickly in this statement to keep the PM informed about any potential risks to project profitability. All the above four project management performance metrics impact this metric, if not handled in controlled manner. A good organizational level PNL tool rather than manual excel sheets reduces the overhead on PM here.

In summary, metrics improve decision making ability by providing the foundation and rationale for the decision by making explicit what is usually implicit in the decision-making process.
So what are your experiences when it comes to project management performance metrics and tracking? What are some of the other project management metrics you have been tracking?  We would love to hear and learn from you.
Pic Courtesy : http://www.flickr.com/photos/ebonysweden/10120374843/
Reference : http://obolinx.com/resources/2013/10/five-project-management-performance-metrics-key-to-successful-project-execution-operational-excellence/

Project Management Metrics #1: Schedule and Effort/Cost Variance

The goal of this metric is to measure the performance as well as progress of the project against signed baselines.  This metric is very important and is the base for profitability of project. The EVM (Earned Value Management) concept, as defined by PMI standard PMBOK, is the commonly used method to track this metric. It integrates project scope, cost and schedule measures to help the PM to assess and measure project performance and progress. The principles of EVM can be applied to all projects, in any industry. Under this method, at any given point in time, project performance to date is used to extrapolate the expected costs and duration at project completion. This technique uses past performance (i.e. actuals) to more accurately forecast future performance. EVM develops and monitors three key dimensions of each work package:
Planned Value (PV): How much you planned to spend for the work you planned to do i.e. it is the authorized budget assigned to the work to be accomplished for an activity or work breakdown structure component. Total PV is also known as Budget at Completion (BAC). PV at any stage = (Planned % Complete) X (BAC)
Earned Value (EV): Earned value is the value of work performed expressed in terms of the approved budget assigned to that work for an activity or work breakdown structure component. It is the authorized work that has been completed, against the authorized budget for such completed work i.e. EV is ‘how much you planned to spend for the work you actually did’. Earned Value is also known as the Budgeted Cost of Work Performed (BCWP).
Actual cost (AC): Actual cost is the total cost actually incurred and recorded in accomplishing work performed for an activity or work breakdown structure component. It is the total cost incurred in accomplishing the work that the EV measured. I.e. how much you spent for the work you actually did. Actual Cost is also known as the Actual Cost of Work Performed (ACWP).
Using these three variables project Schedule variance and Cost variance metrics can be derived which shows if the project is running over or under budget; project is running behind or ahead of schedule, as follows:
Schedule Variance (SV) is the measure of schedule performance of the project. It is the difference of Earned value and the planned value i.e.  SV = EV – PV
  • Positive result means that you are ahead of schedule.
  • Negative result means that you are behind schedule.
Cost Variance (CV) is the measure of cost performance on the project. It is equal to earned value (EV) minus actual costs (AC). Any negative CV is often non-recoverable to the project.
CV = EV – AC
  • Positive result means that you are under budget.
  • Negative result means that you are over budget.
EV, PV and Actual Costs
Since EVM method allows PM to extrapolate the expected costs and duration at project completion based on project performance to date, PM can develop a forecast for the estimate at completion (EAC) which may differ from the budget at completion (BAC) based on project performance. Forecasting of EAC involves making estimates or prediction of conditions and events in the project’s future based on information and knowledge available at the time of forecasting. EAC is typically based on actual cost (AC) incurred for work completed, plus an estimate to complete (ETC) the remaining work. I.e. EAC = AC + ETC.
Based on this PM can also derive another metric, Variance at completion (VAC) = BAC – EAC

 

Project Management Metrics #2 – Productivity: Resource Utilization

The objective of this metric is to measure productivity of resources involved in project and let PM assess over or under-utilization cases.
Utilization% = Total Effort spent by resource/Total Budgeted Effort for the resource2
Budgeted effort is the planned billable work of resource. Any over-utilization and under-utilization indicated by this metric has an impact on the project’s profitability. It is important for the PM to track this metric very closely and find out the reason for deviations and the action items to bring back resource utilization to optimal level. Delayed projects, increased ramp up activities, less work provided by customer, unplanned vacations, less competent resources can impact this metric. To get better control over this metric, robust time reporting systems should be available in the organization. Using this, PM can analyze effort distribution across different project phases/activities. For e.g. Effort distribution can tell PM that how much effort is being spent on defect resolution, customer support or design activities. PM can take corrective actions based on this, if required. For instance, if the resource is complaining that customer support is taking considerable time but the effort distribution shows it otherwise, PM can see where the corrections are needed on what resource is doing. Effort distribution from time reporting systems can also tell the areas of improvement for  better estimations/planning for the next project.
 Effort Distribution

Project Management Metrics #3: Change requests to Scope of work

Signed Scope baseline with customer forms the baseline for the entire project planning and development. Any change to signed scope should happen in controlled manner. So here comes another important metric for PM to track i.e. the number of change requests coming from customer for the already signed scope of work. Each and every change request, once approved by internal change control board (CCB), requires update to Scope baseline which in turn has a cascade impact on cost baselines and schedule baselines and resource plans. Uncontrolled change requests often result in project scope creep and further impact negatively on the project cost/schedule, which is the worst thing to happen for any project. PM should never allow such scope creep. Based on the magnitude of the variance from original scope baseline, CCB should decide whether to accept or reject the change request and this decision should be communicated back to customer. In case of acceptance of change request, the impact on project cost and schedule should be clearly communicated in written form to customer and a written agreement from customer secured on those from customer before proceeding.

Project Management Metrics #4: Quality and Customer Satisfaction

Throughout the execution of project, Quality Assurance should always be on the radar of project manager. Quality here is defined as the number of severe, medium or low defects delivered through the lifetime of the project. It indicates the health of the deliverable to the end user and drives the Customer Satisfaction. PM needs to define, based on project type, what severe, low and medium means. Quality should be reported throughout the life of the project; the later defects are caught, the more impact they will have on the project. Under quality metrics, following are the key ones to track:
Defect density = Total number of defects found/ Measure of size.
For e.g. in case of software projects this can be: how many defects are found in 1KLOC (Kilo line of code). In general, size measure can be considered as planned effort like ‘person day total planned effort’.
Defect age
Number of days since the defect is open and not fixed. It can also be inferred as the time customer has been waiting for their issues to get resolved,
Defect resolution rate = Total number of defects resolved/ Total effort spent1
Rate of closing the open defects over a period of time. If the rate of resolution is not in line with the defects being opened over a particular time, this indicates to the PM a situation of concern.
Defects Tracking
Number of defects reported by customer
PM should keep this as a separate metric to differentiate from the defects reported out of internal testing and the defects reported by end user i.e. Customer. Customer satisfaction depends a lot on the quality of deliverable provided and on how fast defects raised by customer are resolved.
As said above, the later defects are caught, the more impact they will have on the project, it is worth to mention here about Pareto’s principle i.e. 80/20 principle, which PM can use to categorize causes of defects and late time entry relationship. As per this law 80% of the problems are due to 20% of the causes. PM can concentrate on these 20% causes impacting the project most.
Pareto Diagram

Project Management Metrics #5: Gross Margin

Gross Margin (as I wrote in my earlier post on key performance metrics) is the mother of all metrics and the quickest way to determine if your business in on track or not and acts as an early warning system to put in place margin improvement initiatives. Ultimate goal of project execution is to bring revenue to organization with the approved gross margin. Gross margin (GM) is basically the difference of total revenue and the total cost spent on project i.e. profit.
When a project is started, certain GM levels for the project are approved by project sponsor. This approved GM value is generally based on project scope definition, duration, a forecast of resources: onsite, offshore and organization’s investment analysis. Project PNL (Profit and Loss) statement gives a way to PM for tracking his/her projects GM metric at any point of time. For this, PNL statements and forecasts should be current documents i.e. changes in project parameters need to be reflected quickly in this statement to keep the PM informed about any potential risks to project profitability. All the above four project management performance metrics impact this metric, if not handled in controlled manner. A good organizational level PNL tool rather than manual excel sheets reduces the overhead on PM here.

In summary, metrics improve decision making ability by providing the foundation and rationale for the decision by making explicit what is usually implicit in the decision-making process.
So what are your experiences when it comes to project management performance metrics and tracking? What are some of the other project management metrics you have been tracking?  We would love to hear and learn from you.
Pic Courtesy : http://www.flickr.com/photos/ebonysweden/10120374843/
Reference : http://obolinx.com/resources/2013/10/five-project-management-performance-metrics-key-to-successful-project-execution-operational-excellence/
  • Level One :Initial - The software process is characterized as inconsistent, and occasionally even chaotic. Defined processes and standard practices that exist are abandoned during a crisis. Success of the organization majorly depends on an individual effort, talent, and heroics. The heroes eventually move on to other organizations taking their wealth of knowledge or lessons learnt with them.
  • Level Two: Repeatable - This level of Software Development Organization has a basic and consistent project management processes to track cost, schedule, and functionality. The process is in place to repeat the earlier successes on projects with similar applications. Program management is a key characteristic of a level two organization.
  • Level Three: Defined - The software process for both management and engineering activities are documented, standardized, and integrated into a standard software process for the entire organization and all projects across the organization use an approved, tailored version of the organization's standard software process for developing,testing and maintaining the application.
  • Level Four: Managed - Management can effectively control the software development effort using precise measurements. At this level, organization set a quantitative quality goal for both software process and software maintenance. At this maturity level, the performance of processes is controlled using statistical and other quantitative techniques, and is quantitatively predictable.
  • Level Five: Optimizing - The Key characteristic of this level is focusing on continually improving process performance through both incremental and innovative technological improvements. At this level, changes to the process are to improve the process performance and at the same time maintaining statistical probability to achieve the established quantitative process-improvement objectives.
  • Level One :Initial - The software process is characterized as inconsistent, and occasionally even chaotic. Defined processes and standard practices that exist are abandoned during a crisis. Success of the organization majorly depends on an individual effort, talent, and heroics. The heroes eventually move on to other organizations taking their wealth of knowledge or lessons learnt with them.
  • Level Two: Repeatable - This level of Software Development Organization has a basic and consistent project management processes to track cost, schedule, and functionality. The process is in place to repeat the earlier successes on projects with similar applications. Program management is a key characteristic of a level two organization.
  • Level Three: Defined - The software process for both management and engineering activities are documented, standardized, and integrated into a standard software process for the entire organization and all projects across the organization use an approved, tailored version of the organization's standard software process for developing,testing and maintaining the application.
  • Level Four: Managed - Management can effectively control the software development effort using precise measurements. At this level, organization set a quantitative quality goal for both software process and software maintenance. At this maturity level, the performance of processes is controlled using statistical and other quantitative techniques, and is quantitatively predictable.
  • Level Five: Optimizing - The Key characteristic of this level is focusing on continually improving process performance through both incremental and innovative technological improvements. At this level, changes to the process are to improve the process performance and at the same time maintaining statistical probability to achieve the established quantitative process-improvement objectives.
We have to generate the test cases for the above web page. Normally there are two types of testing. They are
i)                   White Box Testing
ii)                Black Box Testing

Black Box Testing
            In black box testing can help to get the design and coding correct with respect to the specification. Black box testing is mainly used to test the functionality and features of the system. In black box testing, there are two strategies followed as given below.

1) Boundary Value Analysis:
            In the web page there are two text boxes, which has to enter no. of rows and no. of columns. We must enter both no. of columns and no. of rows for matrix. If we don’t given any of the two values then we don’t get the matrix.

Test Case
r1
c1
Expected Output
1
-
-
We get an error message
2
10
-
We get an error message
3
-
11
We get an error message
4
10
11
We get text boxes to enter matrix values

In order to complete our aim of calculating the transpose of a matrix, we have to enter both no. of rows and no. of columns for the matrix.

Test Case
Input
Expected Output
1
Clicking Enter Button
Web page having text boxes
2
Clicking Cancel Button
We get message

On successfully entering the no. of rows and  no. of columns, when we click the Enter button we get the web page having the text boxes to enter the value of the matrix. When we click the Cancel button then we get a message.
            If we enter both no. of rows and no. of columns then we get a web page of have text boxes of number equal to the no . of  rows * no. of columns. In order to get the text boxes to enter values of the matrix we have to enter the both the no. of rows and no. of columns.

            We have to enter the values in all the text boxes, then only the values are entered into matrix. Suppose we gave 2 * 2 matrix then we get 4 text boxes.

Test Case
T1
T2
T3
T4
Expected Output
1
-
-
-
-
Error Message
2
3
-
-
-
Error Message
3
-
4
-
-
Error Message
4
-
-
7
-
Error Message
5
-
-
-
10
Error Message
6
11
10
-
-
Error Message
7
14
13
55
-
Error Message
8
-
12
11
10
Error Message
9
12
11
-
14
Error Message
10
14
-
15
16
Error Message
11
-
-
13
12
Error Message
12
10
11
12
13
Transpose Matrix


When we don’t enter the value into any text box, then we get a error message. On successfully entering into all text boxes we get the transpose of the give matrix, in the next web page