Why do projects never go as originally planned?

Listen in Italian on Apple Podcasts, Spotify or Google Podcasts

Sergio Panagia: Welcome to Moze’s podcast, a space for discussion and technical analysis on the development of new digital products. I am Sergio, partner of the firm and today with me is Luca, founder of Refactoring, formerly manager at Translated and CTO at Wanderio.

Luca Rossi: Hi Sergio, thanks for having me, I am very happy to be here.

Sergio: Refactoring is a weekly newsletter on software development, about collaboration between humans, between people, of processes and personal growth. Luca has extensive experience in the organization of working groups for the development of digital projects, and this is what we’ll be talking about today: how to manage personnel, be they both internal or external to the company, how to foster satisfied, happy and effective collaborators, effective time management: time that usually we are never able to get back, as well as project costs.
Luca, I want to start with a burning question: why do projects never go as originally planned?

Luca: This is the be all and end all of questions, Sergio. So, in my opinion, there are so many reasons. I think, one of the main ones is, compared to other disciplines, including engineering; in information technology and software development we approach planning differently. If a bridge needs building, okay? and a master plan is drawn up, it is projected in advance because materials must be bought, there is a great risk if a mistake is made in the course of work. With software it is much easier to do and undo, right? Learning during the course of a project, one can adapt to feedback, whether it comes from customers or managers, and therefore the planning stage, at least in my experience, is more streamlined. The requirements we write are requirements in which we accept that there are areas of uncertainty, which will then be unraveled during the project. This, on the one hand, in my opinion, is functional to how software development happens, which in fact has oriented itself towards agility, right? Of short iterations, and so on. On the other hand, it makes it so that we cannot expect to be able to give with those same requirements which are, in fact, let’s say, more streamlined, accurate estimates and then binding about when the project will be delivered and when everything is ready and in production.

Sergio: You said requirements change and that is why, often, it is difficult to make estimates. But since everyone needs to make estimates to allocate budget, we have to evaluate how long it will take to complete activities; I ask you, in your experience, what is the best way to approach the estimate of a project?

Luca: There are several ways to make project estimates. In my opinion, there is a very important role which comes first, if you want, which is to organize the work, well and divide it into small iterations and batches, ok? Small functions, small activities, which are independent one from another, so that these activities are at lower risk, taken individually, and faster to accomplish. Why is this? Because if you have to estimate fifty small features versus ten large features there is much less risk and therefore more reliability, is what I see in estimating small things. Because in large initiatives there are more pitfalls, more difficulty and one tends to diverge more easily. Hence this activity of dividing the work in software that can then go up independently every one or two days or even several times a day, if possible, greatly reduces uncertainty and reduces the risk, regardless of the estimation technique that is adopted. This being said, there are different techniques, right? The most widespread, if we want, is to estimate the time that the developer employs to write the software. In my opinion it is the most widespread but also the least effective, because writing software, they say: “knowledge work”, right? “Creative work”. Which means the time estimate is very dependent on the person, it is very dependent on who, then actually makes the estimate.

Sergio: My estimate may be different from that of my colleague doing the same job.

Luca: Completely. Then, that estimate, meanwhile, is only valid if I am the one doing the activity, in the best case, and then the same reason that we said before is also valid, that is, that the requirement is perhaps incomplete, makes the time estimate; which immediately smacks of commitment; completing it then in that time, makes it unreliable.
What I think works better is to pass from an intermediate level, if you want, in which an estimate is given, even an arbitrary one, of the complexity of the activity, which is to say, if we have a stretch of road that we have to travel, don’t say right away “it takes me half an hour to do it” without knowing how long it is, but try saying, it is “x” long. Then if you are better at running you take less time, if I’m not as good it takes me longer.
In this industry, this estimate of the complexity is often gives in story points, right? Which are numbers even arbitrary, if you want, that are given by the software teams, but which can then be used to calculate velocity, the speed, let’s say, of the team, measured as an number of points that are consumed, performed over a period of time. This measure, on the one hand, is more reliable over time, especially if you have a history of team performance, on the other it is more, let’s say, a little more subtly, perhaps, but it is “safer”, psychologically more controllable by developers, because it is a number that remains under the control of those who develop and it doesn’t become immediately, “you told me it would take two days, so the day after tomorrow it’s done, right?” As often happens and becomes the source of tensions and problems. But of course let’s say, this something that goes to the clear advantage of the developer, and what I do, anyway as a technician, I am absolutely on the developer’s side, but what advantage can the manager have in such an approach?
The manager, in my opinion, can understand the advantage, hence this estimate is still more reliable, when you consider the team’s velocity over time compared to the mere statistics of the number of hours that are needed to complete the task. He or she is not dependent on a single person, because the estimate is typically done as a team. You decide that a feature is worth four story points, and then it is undisputed that in reality the team consumes “n” story points for example, per week, because there are those who are faster and those who are slower, and therefore it is more controllable as a forecast.

Sergio: So you say, rather than delude yourself that it is four days, knowing that then it won’t be four days, let’s talk about story points knowing that the concept of velocity exists, and that the velocity is then recalculated on the average of actual consumption during work, in such a way as to define clear agreements and long friendships. This, more or less, is the principle.

Luca: Yes, let’s say that then, in my opinion, if you manage to set a working relationship where there’s a workflow for which functionalities are released quickly, almost every day, and also the manager sees the added value that is brought incrementally every week, this then reduces a lot of the focus and attention on estimates, deadlines, and then you can set up a relationship that, in my opinion is healthier too, simply based on defining at the beginning of the week, or every two weeks, what we want to do and what value we can bring, and let’s focus on that. Clearly this you can only do if you organize your work in small batches, because if you have a three-month mega-project, where the expectation is, we’ll see you at the beginning and then at the end of the job, then it’s obvious that there is much more pressure on the deadline itself, right? On when everything has to be done.

Sergio: And you speak of the sacrosanct principle of Agility, which is that of small batches of work, of small and frequent iterations, with frequent releases too. I’ll tell you that even in Italy I’ve seen this happen, so it’s not, let’s say, an illusion, but in Moze, and also in many other similar realities, there are often cases where making important plans in advance is needed, because maybe the manager says, I still have to do these things, they are important blocks.

Luca: In that case how, in your opinion, is it possible to approach big, important project in an intelligent way, in small batches and especially, as requirements change, how to be ready to react promptly to change?
Look, of course it depends a lot from project to project, right? There are actually projects where it isn’t possible to show the value to the end user before you have accumulated a whole host of features which must be released all together, and there is no escape. Projects that instead are is no longer possible to divide into iterations that, let’s say, diminish the risk. In general, planning is right, it is never wrong in itself, it helps understanding and helps create expectations. Let’s say I believe that we must plan to the right degree, the bare minimum for our need to make decisions, so the plans are for us to be able to make decisions one way or the other. Then what I have seen a few times is that people become attached to the plan more than the true value that you are bringing, right? Therefore the definition of the project’s success becomes “I will deliver the plan”, missing an opportunity during the project, perhaps, to deviate slightly, because you notice that there is an easier way or of more value to do the same thing for the user, because in any case you will be learning during the activity, about what you are doing. So, I would still plan, but I would try to organize the activity in some iterations even when it’s not possible to get feedback from the end user. However, you have demos, views with all stakeholders, thus simulating what the result what it would be with the end users or involving future end users so that, if necessary, deviating from the plan is of value, it can be done during development rather than later, when I don’t want to say it’s too late but certainly much more expensive.

Sergio: Very true. You spoke earlier of people, of the Agility built around people, and now, again, you’re mentioning the importance of involving users, collecting feedback and then making decisions. And someone has to make them, there must be someone in charge who takes responsibility for making choices, and this brings me to the question of teams, therefore of projects’ work teams. In Refactoring, you have spoken of generalists and specialists and the differences between these roles, would you like to tell us in a little more detail what those differences are?

Luca: Sure, let’s say, what I also mentioned in Refactoring and comes a little from my experience, as well, as founder and CTO of a startup, which is the generalist experience by definition, if you like, where you have to do everything, wear a thousand hats, and when such an experience ends it also makes you wonder, ok, but what is it that I really know how to do, what do I have to do? Hence the distinction between generalists and specialists it is not a clear distinction but it’s evidently more of a continuum. Specialists are people that compared to, if you want, a professional path, which is clear, they follow a specific path and direct their growth on a certain skill. This does not mean that soft skills are left out and other parts of management between people and so on, but specialization on a certain vertical is important.
The generalists instead are people that maybe accumulate more experiences and who do not have clear links, which are, let’s say, immediately, exploitable between them, right?
As a founder, who is also doing some design work, but also a little development, back-end and customer support. Let’s say that beyond the skills, generalists and specialists, in my opinion, grow differently and they are found to have different personal characteristics, because specialists tend to invest greatly on their vertical, and think that their career, their own path depends on that. So, if we were to exaggerate by saying, if you only have a hammer, everything looks like a nail, when they want to apply what they can do, they take a risk.
The generalist, on the other hand, is maybe a person which is more open to learning different things because they do not perceive their value as related to a skill but more precisely to this type of opening.
Obviously there is no better or worse, it depends, in my opinion, on the company’s phase in its lifecycle the type of business conducted in a startup team, which is in the process of being formed or in its early stages. Generalists typically bring a lot of value, because they can be pragmatic and find the best way to do something without being too tied down to how they think it can be done, right? And when the company grows what’s the right mix?
Because I, sometimes, happen to find teams where there were too many generalists and in other cases where there were almost only specialists, then it was difficult also to be sufficiently abstract.

Sergio: What is, in your opinion, a good mix, talking about team and the division of roles and responsibility?

Luca: Look, first of all, it goes without saying that the more the team grows, responsibilities are structured and outlined, the more space is created for specialists, right? And specialists can make you take that leap in quality, being vertically adept at that particular are of competence. In larger teams, in larger companies, generalists typically become good managers or good at maintaining an overview, which is what distinguishes them. So figures of managerial responsibility, removing, of course, personal sensibilities, often benefit from varied experiences, for example, a product manager who also comes from a technical background, a front-end engineer who may also have a sensibility for product design.
So, let’s say, they are people who manage to carve out a space which is typically of management or connection, that fills a gap in communication between department in which there may be difficulties if you only have people who speak strictly one language.

Sergio: Great, this seems like an excellent orientation to me, also to understand how to create the right work team. You talk about communication, you speak of dialogue between people, and this brings me to the next theme: we are living in times where the tools for communication and collaboration through digital means are pervasive, we are used to it by now, working between one notification and another by Slack, SMS, Messenger, WhatsApp even, yet we know, the work necessary to develop a digital product, to solve, in general, complex problems requires significant commitment like the right focus, without interruptions, as much as possible. How, in your opinion, in your experience and from the things you’ve seen, is it possible to reverse course, organizing a work team effectively, maximizing productivity, but at the same time ensuring the right collaboration and the right level of exchanges between its members?

Luca: Sure, this is a very important issue, andI am going to say something banal by saying that communication within a team is fundamental. Meanwhile, I’ll tell you that in my opinion we are learning a lot about this, that is, in these years, it is as if we were conducting a lot of experiments, right?
Before, we didn’t even have chats and now there’s Slack, there are audio-only channels that never existed before, there are more conference calls, now that we are all working remotely, and now there is the possibility of working remotely, how does this change? How does working change in different time zones?
According to me, there is also confusion because we find ourselves in front of situations, which then ultimately make us better understand what it means to communicate as well as the various methods of communication and what pros and cons they have from the people’s point of view. If you need to trim a little with a hatchet and divide between two great worlds, synchronous, impromptu communication is definitely required, as you said, and that could be Slack, or WhatsApp, but also a conference call, which is fleeting, in real time, and so on. And the whole world, instead, of asynchronous communication written, reasoned, that maybe passes from documentation to more organized conversations.
Both are useful, but you have to know when to use them and why. In my opinion. communication, let’s call it volatile, impromptu, if you want us to hear from you in a conference call or some other way, on Slack or by messenger, is powerful, because it makes people sit down at the same table at the same time, right? And so maybe it will allow you to converge quickly, it is perhaps most the powerful weapon we have, but precisely because it is powerful, it should be used sparingly, because it is also very costly in terms of energy, in terms of people’s focus, you have to stop what you’re doing, you have to make everyone agree to remain focused at the same time.
So it’s something that, in my opinion, must be kept for the most urgent matters or those that are more complex or more personal, You have a “one-to-one” because obviously nothing can replace face to face interaction, but the benefits of asynchronous communication written exchanges, if you want, they are such and many that lead me to say they should make up the majority of interactions.
So the fact that writing a document, which is obviously more structured and reasoned than what can be written on a chat, is permanent, and therefore more usable, and searchable by people, it contributes to the knowledge, if you want, of the team. It makes it so that most of the important decisions but also most interactions, in an absolute sense, are more suitable for the future, without considering that also, from the point of view of organizing one’s time it is very advantageous, that interactions happen asynchronously over time. Let’s say, I write a document, I send it to you, you comment on it and I’ll see you later, even if we are working remotely, in different time zones, we therefore optimize our work. So the rule on this is about 80/20, in my opinion.

Sergio: And I seem to understand that you are saying, that these pervasive tools though very useful, it is okay to use them asynchronously, be it Slack, email, whatever, let’s use them asynchronously. So if I ask you a question and then I wait for you to finish working on your things, I cannot expect an instant response, because that instead, becomes synchronous. If Slack becomes a tool for synchronous communication, for which I have the expectation, even with e-mails, of an immediate response, at this point we all end up communicating a lot and working less.

Luca: Absolutely, yes. And this is all the more important as now we are no longer, in most companies all sitting next to each other with the ability to understand what one is doing at that moment, and maybe we can write to that person more easily because we know it won’t interrupt. We are all in different places, in some cases in different time zones, and therefore it takes more flexibility and in my opinion it also takes a choice of tools that facilitate the qualities that we seek, because so many times we think a chat means going against the current, right? Because chats, and I don’t mean Slack in particular, but in general, are meant to notify you as soon as possible, to make you interact as soon as possible, and therefore the instrument’s karma if you want, encourages you towards a certain type of behavior.

Sergio: Got it. Today with Luca we have talked of projects, processes, how to work on estimates, and how to best organize a work team and related collaborations. If you want to share your experience write to us on mozestudio.com. Luca, thank you.

Luca: Thanks, thanks to you Sergio for inviting me, I am very happy about our chat.