Why don't you start off by telling us a little bit about yourself and your career and what you're currently working on today?
ROBERTO MELI: I was enrolled in the IT field quite early in my career. As a student in civil engineering, I encountered computers and computer science in London during a visit to the science museum. I was fascinated, so I started to work and to study in the material world of software. After earning my degree, I worked on methodologies as a junior consultant at the Elea Group. I was involved there in the creation of methodologies for ICT environments of software development and business processes. I also served as the project manager of the first Italian software for a case tool, SHIP, which was an acronym for Software Help for Information Planning. “Help” was not really correct, but we only realized that later on – it should start with “aid” for information planning. It was the only Italian product in that field for the most used methodology, which was called MAPS.
After five years, I left the Elea Group and had the opportunity to become the technical director of a small IT services company named DPO, and I have worked there ever since. The name is an acronym for Data Processing Organization. DPO was started back in 1957 as a software services company. When I took over, I transformed it into a consulting and training company, because we were too small to offer IT services in a market that increasingly demanded services which could only be offered by much larger companies. I preferred to remain small and modified the mission of the company so that we could focus on the measurement field and the process management competence area. We did a lot of training in those areas for big Italian companies and we became the leading company in the software measurement area. I am currently CIO of the company. Although we are still a small company, our brand is highly regarded in the Italian and international market for this area.
Apart from the business, I also contribute to non-profit associations like GUFPI and ISMA, which is the Italian Software Metrics Association. I also participate in the Cosmic Group as a member of the core team, and I served as the chairperson of the practices committee that coordinated the effort to publish standards. Additionally, I work with the International Institute of Research to organize the Sofware Measurement European Forum, which is one of the leading European events on software measurement.
In the year 2000 The Standish Group reported that 70% of all software projects undertaken by large, small, and mid-sized organizations came in over budget, over time, or not at all. Do you have any thoughts on why software development practices have gotten into such a state? How can improved software estimation have a positive impact on turning these numbers around?
ROBERTO MELI: I don't believe that IT product managers are less capable than product managers in other areas of software engineering, but our failure rates are higher than those experienced in other areas, so I think there's a problem with the product. Because software is immaterial, it can be difficult to evaluate the end results of a project, especially for the end user. They have different expectations about what the product is building up to and those expectations are not so easy to change. You are always in the middle of the effort, so it is difficult to maintain a stable process. And it is especially difficult to identify the real requirements for the application to be built correctly.
Very often software is a supporting tool for processes and these processes are ever changing, so it is frequently a major challenge just to understand the business objectives. It is therefore not surprising to me that requirements are one of the major courses of failure in software products. In my experience, there is evidence that analysts are not always capable of understanding customers' objectives. And when you don't have a clear idea of what you have to develop, it is very difficult to have a clear idea of what resources you need, so estimation becomes difficult too – not only because you lack methods and tools, but because you don't know the scope of your estimation exercise. That's really key. I think that good estimation practices can be very useful for improving production processes. If you start with a good statement of requirements and work with experienced people in your company or outside your company you can provide good support throughout the production process using good estimates.
There are other problems that arise for good estimation. One is that all of our technologies are inconstant in our field, so once you have learned how to use them effectively and efficiently you have to go and change them again. As a result, you find that your experience is not so useful for new predictions or new needs. There are two positive aspects of estimation when it is done properly. One is the technical aspect, because a good estimate allows the manager to make predictions based on holistic experience, not just on his or her personal experience. The other positive dimension is the social aspect. Good estimates make it easier for you to negotiatie for resource allocation, because your estimates are based on rational, explicit models and not only on impressions and people's intuition.
Could you elaborate on some of the different estimating techniques that are available? Maybe compare and contrast some of these techniques while offering some insight into how an organization can determine which ones are right for them?
ROBERTO MELI: I First of all, I have to say that the word estimation was used at the beginning only for management folks. When we originally used the word estimation we meant effort and cost or duration estimation. Today there is an additional component – size estimation. That's because lines of code or function points are viewed as the main cost driver. So to estimate properly, we also need to have a measurement validization for the size of the software application. One of the problems, though, is that we generally need the data for effort and cost at a time in the lifecycle when the data for deducing the size of an application does not exist.
Generally speaking, an estimation model can be of two types. It can be a direct estimation or a derived estimation. In a direct estimate, the practitioner tries to go directly to the final results that he or she wants. He or she looks at the documentation, the constraints, at all of the information that is available on the product and at the process and then estimates the directly. There are many product managers who do that. In some ways it is like an iceberg ““ most of the reasoning is obscured under the surface at the intuitive level. Very experienced people can be very good at making direct estimations, but they are not able to explain how they arrived at the results. Consequently, it is not easy to share the results with other people or to make a business case based on the estimate.
In contrast, a derived method is a step-by-step written or modeled process that starts with input points and, by means of an algorithmic model, produces a final estimate. There are many models like this. And these models are based on historical data. Such models can also be based on either local historical data or general world market databases such as the ISBSG.
I suggest using publically available models instead of the proprietary models, because with public models it is possible to open the box and to understand what data was used for which projects, the quality of the information, and whether a model was prepared for the exact domain that you are working in. It is also possible to add your own data to the public data so that you have a mix of data that is more reliable for making predictions. Another approach is to use a mixture of different models, as opposed to just one. You will get a good result if you make estimates using two or three different methods and then calculate a result based on the weighted average. Each method has advantages and disadvantages, so it is a good idea to work like this.
The mission of estimating software is not one of discovering the right value as one would in a scientific operation. It is about giving the product a reasonable context in terms of resource assignment. With this information we can better acheive the objectives that have been set for the product. For example, consider a case where managers cut a product budget while maintaining the same objectives for the product. This frequently occurs many times in real organizations and it is very strange. We end up asking, “If the same project could be achieved with less resources, why did we waste the resources at the beginning?” Actually, the question is not about whether we can maintain the objective with less resources, but how risk is impacted by doing sso. When we started the project and we assigned the initial budget to the product we believed that this was an realistic assessment that carried a certain level of risk. When we cut the budget and maintained all of the objectives, the risk of failure is now higher. So risk management and resource estimation are two processes which fit naturally into a project manager's practice.
If you had an organization that was at level 1 in terms of their process and measurement capabilities, what would be some simple first steps that they could take to begin improving their estimates and start making a difference as soon as possible?
ROBERTO MELI: It is quite important to draw on an extensive pool of outside experience. When people take their collective experience, formalize it and condense it, they create a resource that allows them to capitalize on others' experiences in order to improve their own capabilities. You can see a very impressive change in your capabilities within a few days of training like this. If you only rely on your experience, you will be very limited because each project manager may not lead many projects in his or her life. In order to benefit from sufficient experience, it is necessary to have access to other people's experience. So training is the first step. You can make use of webinars or online training or conferences, but training is the crucial first step.
It is also important to learn from your own experiences. Very often people who are involved in production processes are not inclined to learn from experience. As soon as they have finished the product, they just pass to the next one and do not reflect. They do not think about what happened.
So it is important both to have training and also to register and learn from your own experiences. It is also important to have access to well established models and tools. I would suggest the use of tools, but without losing control of them. One of the disadvantages of tools is that very often they are seen as a black box – you just feed them data and receive the results and you never know what's happening inside the black box. But that is not a way to learn; it is just a way to drive the machine. So using a tool is important, but it is also important to understand how it functions. My final suggestion is to share your experiences and knowledge with other experienced colleagues in your organization. Communicate with them frequently and to try to involve them as much as possible in each estimation exercise.
Biography of Roberto Meli
Roberto Meli is an expert in project management, software development methodologies, and software metrics. He attends to and teaches training courses in Italy and foreign countries and is a coordinator of GUFPI – ISMA CPC (Counting Practices Committee, Technical Committee for the Rules of function point counting). Since 1990 he has been the Managing Director and CIO for DPO.