This whitepaper provides insight into our experiences with Agile Processes, and describes how best to apply Agile in companies and projects. The paper goes through the threats that a project faces during the implementation and use of Scrum, with suggestions on how to minimize them. And finally it presents a model to help you understand the maturity of your current Agile processes – whether you are applying robust or weak Agile processes.
In a few short years Agile has become the current reality in IT project management. It has led to new business models, driven by increasing demand for flexibility, closeness with clients, and better overall results with the methodology.
With time to market continuing to be a critical metric for businesses, there’s a need for faster and better communication, in order to increase development speed. As a result, there needs to be real understanding between those who wish to have a problem solved (users), and those who can solve the problem (the team). As the client is able to see pieces of the whole product, it makes it possible to continually adapt the original specifications , so you are constantly getting the most value out of the process, as well as better handling the project’s budget.
Agile is also a particularly attractive methodology because both parties (client and provider) experience increased motivation. This is because there is complete transparency and visibility on the product being developed. Agile Processes helps stakeholders view and provide feedback on small functional parts of the final product during the development process.
By methodically applying Agile best practices – involving frequent deliveries of incremental products, team capacity planning, constant product and process improvement, and focusing on people and results – the company is able to quickly and easily change requirements, provide much needed space for innovation, and simultaneously achieve improved quality and results.
In my many years of experience managing Agile projects, I’ve noticed some trends in the way companies define, develop and manage Agile.
The effectiveness of the Agile process depends on several factors: how the process originated, how it was implemented and maintained, its basis on previous experience, and the commitment of people to the project.
All these variables can determine whether the process is one of two extremes:
too robust or too weak.
If you sit on any of the extremes, you need to make changes to find a balanced Agile process. A balanced Agile process is one that satisfies the project and team needs, as well as the company goals. Every project needs processes, and depending on specific factors you’ll choose a strict, balanced, or weak process. The most important aspect is that everyone agrees and understands the process in order to be able to implement it. Of course, there’s always room for improvement and that usually happens when the team reaches a certain maturity level that allows changes to be implemented — and often it is the team itself that requests these changes, as part of continuous improvement.
Agile tends to adapt more easily to flexible organizations, and projects with a rapid rate of changes. But what would happen if the company is flexible but the project is rigid? Or if the company is rigid but the project changes constantly? Many will start by applying some principles of the Agile Manifesto at the project level. Although it may not be easy to do it at an organizational level, it’s a good starting point. Some examples of the initial actions you can take: include the client in regular meetings to achieve higher commitment; provide demos to show accomplished requisites; formalize retrospective meetings to learn from past situations, and improve and even apply some methods like the “Agile Unified Process” that follow some of these best practices. Some call this a “hybrid implementation” while others will say it’s simply Hybrid Agile (when you implement portions of the full Agile process to a project). In other words, companies will adopt basic aspects of the Agile model and mix them with internal processes and aspects to make implementation possible.
Usually, you expect processes to become more mature, as you implement practices as the project moves forward. Ideally the company will try to move closer to a full Agile implementation, where roles, artifacts, components, meetings and responsibilities are clearly defined, understood and applied. Together this should result in a stronger and more balanced process, which is able to overcome threats and problems more easily. In Agile the client is the one who – usually – creates and manages the requisite backlog for the product/project. This list clearly represents their expectations in terms of value, cost and deliveries. And the team must be ready to satisfy client expectations, meaning they must work together as a team.
Some Agile processes, implemented in the short term, tend to become a “personalized or proprietary process” – in other words it only works for the team executing it. This is not viable for the long term because you can’t reproduce the processes in other teams in the company.
Some of the situations that can become threats include:
The team starts to work with high levels of uncertainty, without knowing if they will be able to close the sprint successfully. The team can be idle (the team’s capacity is higher than the assigned tasks for the sprint) waiting for more information, which has a negative impact on both performance and motivation. Also, poorly defined requirements increase the rate of rejection of developed software, because that’s not what the client originally wanted. Identifying what the customer wants is hard when the information is poor, or when they only have defined requirements at an “epic” level and not at a user story level. Often the user story evolves during the sprint which changes the original estimation and – typically – ends up with the team not finishing the task in the agreed period of time.
How do we avoid threats in these situations? Team members must work hard and help to collect information to load the requirements as precisely as possible, strengthening both formal and informal communication. The team will work to increase productivity, generating more value for the client. The Scrum Master pushes the client to be aware of what’s happening day-to-day and involves them to act as a real Product Owner, providing information, priorities, and criteria for the development team.
This can occur with regard to the general status or the overall progress of the project. It can lead to project failure. This happens when no one is in charge of constantly updating the project’s critical information on the issue tracker (both at a user level or user story level), or when the wrong tool is used, or even when there is a poor understanding of the use of tracking tools.
How do we avoid threats in these situations? Teams need to utilize and process reports and controls, and analyze any deviations. To accomplish this, the team must add data methodically to the tool, day after day. Many teams set “team motivation” penalties for those team members who fail to do this (for example, they must bring food for the rest of the team). Some people think that the Agile processes does not include formal reporting, but actually it’s quite the opposite. If all the data is loaded into the tracking tools, the company will have up-to-date reports available at any time. If you do not use a tracking tool, then every time you want to create a report, you need to collect the relevant information.
This occurs when the team does not have enough autonomy/freedom to plan their own sprints and the task sizes are imposed, or when the planned time box is not respected or it changes. In some other cases the team does not participate in the planning session because they don’t understand the objective of the current iteration. In those cases the team needs a “grooming” session, to understand the goals and avoid wasting time trying to figure it out.
How do we avoid threats in these situations? Based on what’s on the backlog, teams have the responsibility to decide what they will be working on (empowerment) and the tasks needed for each user story. This makes the team commit to a fulfillable objective that also satisfies the product owner. The Sprint must be negotiated at the beginning to reach a common understanding/agreement. The team as a whole must identify the tasks, the interrelation between them (critical path), the amount of effort taken for these tasks, and assign them between team members, while always keeping in mind the team’s capabilities.
In this situation each team member acts individually and performs their tasks without taking into consideration the sprint goals. This results in lost team synergy.
How do we avoid threats in these situations? Experience says that team conformation is not the result of one person’s effort, but the joint effort and activities of all the team members working together, to achieve better results than they would as individuals. One of the main aspects that make Agile teams highly productive is “team empowerment”. Agile team members have more freedom to make decisions, but also higher and mutual responsibility towards project results. Therefore it is expected that the team is constantly motivated to achieve synergy. A good balance between planning meetings and the team’s flexibility will result in higher commitment and autonomy to make decisions when facing difficulties. Self-organizing teams are the ideal.
This post-mortem analysis helps the team grow and improve. One of the best artifacts is the retrospective meeting, in which client, team and Scrum Master participate. It starts with the team doing a demo and then the retrospective to analyze what worked well and what didn’t.
How do we avoid threats in these situations? Although it may sound hard, it’s completely necessary to apply “brutal honesty” to bring to the meeting the best feedback and achieve active listening. Each member will share their thoughts and the rest must actively listen to evaluate, empathize and grow together. It’s a “moment of truth” that’s ideal for sharing both positive and negatives aspects, as well as improvement ideas.
Here team members may not know how to handle the distance between team members. Communication is bad, members do not interact frequently, there is a lack of definitions, and there are poor interpretations of requirements. In short, there is misunderstanding. But actually Agile itself can help improve remote work, and in turn become a competitive advantage.
How do we avoid threats in these situations? The team, including the client must learn to use communication tools effectively to help improve the effectiveness and efficiency of meetings and decisions. The team needs to consider both geographic differences as well as cultural differences. The company must implement spaces to boost communications and integration such as visits, informal meetings or virtual meetings.
When this occurs everyone is not on the same page in terms of the main deliveries in the production environment, and what is being planned for delivery. The roadmap of a product is a high-level overview plan that describes how it is expected that the product will evolve in time. If the team has access to this information it has more visibility about the future of the project. This will help align the budget, as well as analyze possible threats together with the client.
How do we avoid threats in these situations? The client needs to understand that the team needs to know the long term vision of the project, in order to be able to plan better for the future. The PO must share the vision of the expected evolution of the product, identifying the exact commitment the team must make in order to achieve their goals.
Companies sometimes create a new version of the process, but one which is distant from real Agile processes. For example, some teams use “Scrumban” (Scrum + Kanban) or hybrid methodologies (Agile processes + waterfall), mixing predictive with empirical elements.
How do we avoid threats in these situations? Each project must be analyzed individually to understand what’s best for it.
Agile processes, when applied correctly, add great value and benefits to projects. However the implementation of Agile is challenging, and there is a wide spectrum of choices regarding how exactly to implement Agile. In particular remember the following key points: