What is Agile
This article will explore what is Agile, which is one of the most popular and exciting product development approaches in use today. We will also uncover what is Agile methodology, so you can apply the thinking to benefit your own organization. Agile has transformed how products of many kinds are developed and delivered. The time between releasing new products is now a couple of weeks or even less, whereas previously it was measured in months. Agile development will bring early delivery of products, encourage a culture of continuous improvement, and provide the ability to support making rapid and flexible changes to meet the needs of your organization.
Although the Agile definition was initially aimed at developing software, sectors outside of IT have asked “what is Agile” and now embrace Agile thinking, for example, by applying the approaches to planning projects and creating new products such as automobiles. It is now the most commonly used approach globally for developing anything, even extending to the development of new processes, new organizational structures, and new ways of working. This way of doing things encourages collaboration between product development teams and the customer, delivering products in a flexible and nimble way. However, it does require an open mind that is receptive to being guided by ideas and concepts instead of rigid frameworks and processes.
If you look up the term in a dictionary, you will find that it is defined as the ability to move quickly and easily, with grace, suppleness, and dexterity. The opposite is something that is clumsy, inflexible, and slow to operate. That is what characterized the historic practices for software development, which were heavy with documentation and management control. In contrast, today’s software development methodologies provide early delivery and continuous improvement of Agile software, with the ability to support rapid and flexible changes to the requirements.
Navigating through any Agile software development life cycle requires an understanding of the concepts. These were first set out in the Agile Manifesto. This was first published in 2001 along with the Twelve Principles. Taken together, these defined what differentiates an Agile methodology. This new approach for Agile programming provides a way of thinking and a set of values that promotes a fast and nimble way to work, always focusing on meeting user requirements efficiently and effectively. There is an emphasis on efficient production, collaboration, communication, and rapid development of smaller sets of features under the guidance of an overall plan.
The Agile scrum methodology was one of the earliest Agile software development methodologies. In this approach, a small team, known as a scrum, develop software in short time-boxed iterations. An Agile methodology scrum is a self-sufficient, self-determining team. In the scrum Agile methodology, there is no manager that directs the team; instead, the team is empowered to develop solutions, ways of working, and resolution to issues themselves. This common theme of self-determination is shared across all the different Agile software development methodologies.
Agile Meaning
Because this concept is based on a set of principles and values, there are many different approaches to development that can claim to have an Agile meaning. Scrum Agile has been around the longest and is still the most widely adopted example of an Agile process flow, but several other more recent methodologies have been created using the same Agile basics. For example, Kanban and DevOps are both Agile methodologies that embrace the principles and values set out in the Manifesto and the Twelve Principles, and hence share the same Agile methodology meaning. To align with the authentic meaning, any methodology must visibly embrace the Agile principles. This is how the Manifesto defines the meaning of Agile:Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Principles of Agile
The definition of Agile is based on the Twelve Principles of Agile Software, published in 2001 to support the Manifesto. Taken together, these two artifacts provide the guide for all of the individual methodologies that have come to be known as ‘The Agile Movement.’ Together they describe a culture in which change is welcome, and the customer is the focus of the work.
The Twelve Principles are:
We follow these principles:
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity–the art of maximizing the amount of work not done–is essential.
- The best architectures, requirements, and designs emerge from self-organizing teams.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
These principles can drive the following beneficial behaviors:
- Customer satisfaction through early and continuous software delivery – Customers of Agile methods are happier when they receive working software at regular intervals, rather than waiting extended periods of time between releases.
- Accommodate changing requirements throughout the development process – The ability to avoid delays when a requirement or feature request changes is a key feature.
- Frequent delivery of working software – the team operates in software sprints or iterations that ensure regular delivery of working software.
Collaboration between the business stakeholders and developers throughout the project – Better decisions are made when the business and technical team are aligned.- Support, trust, and motivate the people involved – self-determining teams are more likely to be motivated and hence deliver their best work.
- Enable face-to-face interactions – Communication is more successful when development teams are co-located.
- Working software is the primary measure of progress – Delivering functional software to the customer is the ultimate factor that measures progress.
- Agile processes to support a consistent development pace – Teams establish a repeatable and maintainable speed at which they can deliver working software, and they repeat it with each release.
- Attention to technical detail and design enhances agility – The right skills and good design ensures the development can maintain the pace, constantly improve the product, and sustain change.
- Simplicity – Develop just enough to get the job done right now helps maintain interest and understanding.
- Self-organizing teams encourage great architectures, requirements, and designs – Skilled and motivated team members who have decision-making power, take ownership, communicate regularly with other team members, and share ideas that deliver quality products.
- Regular reflections on how to become more effective – Self-improvement, process improvement, advancing skills, and techniques help team members to work more efficiently.
Agile Overview
Agile development is one where requirements are discovered, and solutions are developed by self-organizing and cross-functional teams, using collaboration with end-users, adaptive planning, evolutionary development, early delivery, and continual improvement, and where the development cycle encourages flexible responses to change. If a methodology has any of these aspects missing, then it is not a truly Agile model.
The detail of what is Agile software development is process specific and what is specific to an Agile development methodology can and will vary between organizations. That is one of the most powerful aspects of Agile development. The Agile development process detail for any Agile product development can be tailored to suit the characteristics of the organization, the capability of the development teams, and the software development cycle while still following the Manifesto principles. These provide a great Agile overview that is still as valid today as when it was first published nearly 20 years ago.
There are a number of different Agile development practices, each depending on the specific application of the 12 principles. These can have slightly different software development cycles. For example, a scrum software development cycle will have fixed iterations, whereas these do not exist in a Kanban development process. However, common to what is an Agile development methodology is the regular delivery of products in short timescales, produced without excessive documentation.
Scrum
The features of a scrum software development cycle are:
- The developers are organized into one or more small ‘scrum’ teams
- There is a product owner for each scrum team, who represents the voice of the customer and the products stakeholder to the team. They define the requirements as a set of ‘user stories’
- A scrum master acts as a buffer between the teams and any distracting influences, and facilitates the development process
- Work to be done is held as user stories in a product backlog
- The product owner decides on the priorities of the user stories
- The development team take prioritised user stories and carry out all the necessary tasks to deliver working software
- The working software is delivered in a series of incremental sprints
- Sprints have a fixed duration of between and 4 weeks
- The sprint starts with a sprint planning event that agrees the goal for the sprint and which backlog items will be tackled
- A daily scrum is held with all team members to review progress and agree how to address any impediments to delivery
- At the end of each sprint, a sprint review and sprint retrospective shows progress to stakeholders and identifies any lessons and improvements for subsequent sprints
- Only working software is output from the sprints
Kanban
Kanban is a system for workflow and process management that provides visual signals to communicate information to improve efficiency and effectiveness. Kanban is used in many sectors, including manufacturing, process improvement, software development, project management, program management, and IT Service Management (ITSM). Kanban is particularly used in Lean systems, as it provides a simple and understandable way to identify waste, especially bottlenecks between the different steps in an end-to-end process.
The principles of Kanban are very much aligned to the principles included in the Manifesto and Twelve Principles:
- Kanban visualizes the workflow so that it is easy to understand
- Kanban encourages acts of leadership at all levels
- Kanban helps to measure and improve collaboration
- Kanban encourages respect for the process, roles & responsibilities
- Kanban helps the team to make the process explicit and easy to understand
The term word Kanban is Japanese for ‘signboard.’ In many implementations of Kanban, the term is also used to describe a board that is used as a visual system to display tasks, their status, and their progress towards completion. This Kanban Board uses different columns and colors to differentiate between different types of task and their status. This is the most commonly used form of Kanban.
This Kanban Board plays a vital role in displaying the workflow of tasks. It helps to achieve the 12 principles by optimizing the flow of task between different teams and between different stages. Kanban Boards are a useful method for defining, managing, and improving services. By displaying work items visually on the Kanban Board, team members can see the state of every piece of work at every development stage and get a common understanding. Moreover, a team member gets an overview of who is doing what so that they can identify and eliminate problem areas in the product development process.
The Kanban methodology allows work to be prioritized according to the needs of the customers. As work moves from one state to another, extra work can also be added from the backlog to maintain a steady flow of work through the system. The development team members collaborate with each other to improve the flow of work throughout the project. Kanban boards are commonly used in conjunction with the Scrum approach to display progress and support discussions at the daily sprint meetings. However, Kanban as an approach differs from Scrum, as the process is never restricted to a set process or a defined sprint backlog. Kanban is more able than Scrum to maintain flexibility.
History of Agile
The history of Agile development goes back to the 1990s when new approaches such as Extreme Programming were developed to address frustrations with the long timescales and poor quality of the software development cycle. Before the advent of Agile development practices and the start of the history of Agile, the final product of many inflexible software development cycles often did not meet the user’s needs. Thought leaders in the practices wanted to address these issues by defining what is Agile.
In early 2000 a group got together to develop a new approach, which laid the foundation for the Agile development process. They published a number of articles that referenced new “lightweight” processes for the software development lifecycle. In February 2001, the history of Agile really started with the publication of the Agile Manifesto for Software Development and the Twelve Principles of Agile Software. These were the foundation for all Agile development models.
This codification of what was a methodology for Agile development provided an alternative to the documentation driven and cumbersome software development cycles of the time. This definition was easy to understand. This helped its adoption by many who created their own Agile development process. Using an Agile development cycle first took hold in small start-up companies but soon spread wider.
Agile development process thinking has continued to be developed and innovated throughout its history, resulting in a number of different Agile development models. Each of these Agile development practices focus on the customer. Throughout the history of Agile, these practices have promoted new organizational models for the software development cycle.
The concepts in the definition of what is Agile demonstrated the value of delivering good products to customers by recognizing that people are the most important asset. History demonstrates that following Agile development model values will always fuel interest, adoption, and development of Agile development practices.
Future of Agile
The Agile Manifesto principles are just as valid today for the future of Agile as when they were first published nearly 20 years ago. The pace of today’s technology change, the ever-increasing expectations of business and consumers, and the globalization of supply chains require Agile methodologies for almost every aspect of life, not just Agile software development. The future of Agile will be just as vibrant and valuable as the history of Agile.
Over recent years new Agile methodology steps have emerged to cope with the required pace of delivery and specific applications such as Agile project management. These new steps still align with the Agile manifesto principles and hence are still recognizable as being Agile. Agile Scrum has become less attractive to many organizations, particularly those whose existence depends on making rapid changes to consumer-facing products. The fixed delivery cycles of Scrum create issues; hence this approach will feature less and less in the future of Agile. Agile methodologies such as Kanban and Continuous Delivery overcome the limitations of Scrum, and the future of Agile will continue to see Agile methodologies enhanced with new ideas. We are also likely to see new Agile methodologies in the future of Agile, as organizations discover new ways to interpret the manifesto principles and develop new Agile frameworks.
One expected area of growth in the future is Agile project management. Before the advent of Agile frameworks for Agile project management, project management practices shared the same issues that led to the development of the Agile manifesto principles, with projects overrunning both timescales and budgets. In the recent history of Agile, Agile project management methodologies emerged and have started to be widely adopted as the project management community recognized the value of Agile management. As the use of Agile project management increases, it is highly likely in the future of Agile that these Agile frameworks will evolve as organizations learn and innovate.
In the future of Agile, Agile methodology steps for developing software may also change as new technologies emerge, particularly with the continued growth of remote collaboration tools and the use of enterprise software. There will also be new steps created for when the Agile framework is used outside software. For example, the human resources discipline has started to apply the Agile Manifesto values for activities such as recruitment. As the future of Agile sees Agile management techniques extended across every part of an organization, the methodology steps will be amended to suit the particular characteristics. Twenty years ago, at the start of the history, the use of the Agile framework was restricted to very few early adopters. Today and for the foreseeable future of Agile, Agile methodologies and Agile frameworks are a fundamental tool for many different disciplines and organizations.
Agile pros and cons
Whilst the values and principles of Agile can be applied to a very wide variety of circumstances and situations, Agile pros and cons are many, and there will be some organizations where the Agile cons outweigh the pros. Despite the proven benefits of this flexible methodology, it cannot be applicable for everyone in every circumstance. One of the challenges is that it requires everyone to buy into the methodology, not just the software development team.
A commonly seen problem is that management have a deep-seated desire to be in control of everything, which is completely at odds with the principle of self-determining teams. For Agile to be truly effective, every part of the organization and every individual in it must understand and embrace the principles and values. For some organizations that is too much to ask. The whole organization needs to change its culture from being authoritative and hierarchical to the open, trusted, empowered culture inherent in Agile thinking.
In theory anyone can make the necessary transition, provided that they have the desire and determination to understand the principles and values and then make the necessary changes to their attitude, behavior and culture. However, not everybody is built the same. There can be barriers at both individual and organizational levels to embracing what Agile has to offer. Here are some of the potential barriers to adoption.
It requires a culture that is open and built on trust, and the giving of feedback, both within and outside the team. This can be a major challenge for some organizations. This is particularly the case for those with an entrenched hierarchical management structure, where management always direct and routinely check the work of their subordinates, and are not used to getting feedback. - It needs a change in mindset so that you only work on one thing at a time, and deliver that one thing. Only then can you move on to what the next work item is. Teams are given just enough work that they can complete a single iteration. This powerful concept helps to avoid teams becoming overloaded. This can be a challenge to organizations that expect every project to show some progress every period.
- It promotes different organizational structures. If you want to increase the rate of delivery from your team, you can’t just add more individuals into it. One of the developments in the organization of software development teams has been the introduction of the ‘two pizza’ concept. This is where the team should be of a size where the team members can eat two pizzas at one sitting. This concept results in a team size of 5-7 people, which co-incidentally is a very similar size to the smallest organizational unit in armies for over a thousand years! A downside of this approach is that if you want to get more throughput you have to stand up a new team. Hence it may not be the best option for organizations where the required rate of production changes.
- It expects teams to be self-sufficient and self-determining. Hence adoption of the methodology typically results in the loss of traditional management roles. The organization needs to be prepared for this, and either re-deploy managers that are no longer necessary or dispose of them.
Before you start adoption, it is important that you consider the Agile pros and cons. These can vary depending on which particular approach you look at, for example, Agile vs Waterfall vs Scrum. To understand the difference between Agile and Waterfall it can be helpful to build an Agile vs waterfall comparison table.
Some key differences concerning Waterfall vs Agile are:
Waterfall is a strictly formalized and linear approach to software development, whereas Agile pros include its flexibility.- Waterfall starts with the development of a hierarchy of fixed requirements, unlike Agile, where the requirements are encouraged to change over time. This is an important difference of Agile vs waterfall methodology and a key element of Agile pros and cons.
- Waterfall defines user requirements in considerable detail, whereas Agile captures them as simple ‘user stories,’ in the form of ‘I would like to be able to ….’. This is an important differentiator between Agile vs Waterfall, as it is more likely to give users what they want.
- In Waterfall, the detailed design and documentation is done before any coding is attempted, in Agile coding starts with just the user stories, and the design evolves over time. This is a good example of where Agile pros and cons derive from the same feature; as for safety-critical products, the design might have to be detailed upfront.
It isn’t feasible to have an Agile and Waterfall model, with both in the same methodology, as they are in such great contrast to each other. Considering the Waterfall vs Agile methodology, Waterfall is today only appropriate for large scale projects with outsourced coding, where the requirements are not going to change between design and delivery. In the next section, we will consider this with reference to Agile vs waterfall project management.
Implementation considerations
The most important thing to consider before you consider implementation is your ability and desire to transform the culture across your whole organization. Adoption requires open minds that are prepared for change. This aspect of Agile can frighten the types of individuals who expect certainty in everything. It also requires collaborative team working, where every team member supports their colleagues; there are no ‘bosses,’ and lessons are learned after issues rather than blame being apportioned. This can be a major change for an organization that has been used to autocratic and hierarchical management, using task-oriented instructions.
Any implementation must, therefore, be supported by an aligned organizational change management (OCM) initiative. The OCM activities can and should themselves be done in a flexible and incremental way, recognizing that every organization and every individual are different and that there is no single prescribed approach to get to a culture that is suitable for Agile working.
Agile requires customers to work as part of the delivery teams, with frequent and early opportunities to see the work being delivered, so that they are part of the process to make decisions and changes throughout the development project. These customers need to be prepared for this involvement, as some customers might not have the time or interest necessary for successful delivery.
It also requires all team members’ full dedication to the collaborative culture, working together towards a common goal. Before starting the transition to these new ways of working, you should consider the capability of your individual team members to work in this way. There is a risk that some might not want to work in these new ways, as they are more comfortable with fixed approaches. It would be best if you worked out what you can do with these individuals. Education and training in the new concepts is one approach, but if that fails, you may need to re-deploy staff.
You will need to review the different approaches and methodologies available and decide which is best for you. This selection should be made carefully; otherwise, your investment may be wasted if you make an inappropriate choice. Hence you need to budget for and plan for up-front education of key individuals on the wide variety of lean approaches. Whilst you may get some help from external consultants, selecting the best approach requires knowledge of your organization, products, values, and existing culture, as well as knowledge of what is Agile.
When making the selection of the most appropriate approach, you must look beyond the jargon and product marketing. New transformational approaches like this do not come in a shrink-wrapped box, with regular updates. Agile is a concept that can be of immense value to your organization, but the transformation will require a lot of dedication and hard work. You should view your implementation as a new adventure. You are likely to have challenges in the initial stages, which you may need help with to fix, but ultimately moving to Agile can only be done by your own staff. This will require open minds, flexibility, and above all, a focus on the customer. Trying to stick to a fixed timetable for implementation is likely to lead to failure.
Have someone in your organization that is an evangelist for what is Agile. They should be prepared to lead the transformation but be a good listener too. Following the Agile principle of asking for feedback, and acting on it, is fundamental to a successful implementation.
What is Agile management
Agile management in action varies in detail between which Agile methodology is being used. For example, in Agile scrum project management, the project manager has management responsibilities, just the same as in waterfall project management. In scrum project management, the scrum master has no role to play in managing the project. The scrum master role in the Scrum framework is to protect the team from outside influence and to act as a facilitator but not a manager for the scrum process. Agile project management tools sometimes have challenges encapsulating what is Agile management for projects, especially as for scrum project management; all team members have a role to play in managing the project. This creates challenges in any Agile project management tools that expect roles to be restricted to single individuals, which is contrary to what is Agile management theory.
What is Agile project management
The Agile project management process applies Agile principles to the discipline of project management. In an Agile project management methodology, projects are delivered using short sprints. This Agile approach to project management encourages projects to proceed at pace, as it provides a short time focus on the next deliverable. This avoids the situation with non Agile management practices when there are often long timescales between the start of a project stage and the delivery of its products.
Also, in a non Agile project methodology, the deliverables have to be determined in detail at the start of the project, which make them inflexible to change. By using the Agile methodology project management can avoid this issue, as the deliverables are defined at the start of each short sprint. Hence the Agile methodology in project management is more likely to deliver what the user wants. What are the Agile management processes will be different for each project Agile methodology, depending on factors such as the required level of project governance, the type of products being delivered, and the fit of the 12 principles to the specific situation.
A good way to understand what is Agile project management is to contrast it with how project management used to be done.PRINCE2 is one of the world’s most widely-recognized project management methodologies. Before the advent of the new approaches, it was one of the world’s most widely used approaches for managing projects, using a waterfall approach that delivered all the products at the end of the final stage.
PRINCE2 and other similar waterfall project management methodologies use a predictive and plan-based approach, requiring detailed planning upfront to determine the required products and delivery timescales. In contrast, an Agile process for project management brings many short-term, incremental achievements without the need for an up-front detailed definition. This adaptive approach is a key feature of any Agile management style for managing projects.
This means that whilst a non Agile project management methodology such as PRINCE2 essentially steers the customer to remain focused on the project’s original business goals, Agile project management processes are very responsive to changes in the project environment and customer requirements.
Today, Agile project management with Scrum is the most commonly used application of the Agile project management principles and is supported by a number of Agile project management software tools. As what is Agile project management becomes more widely used, it is probable that it will have the same experiences as other applications of the 12 principles, with modifications and enhancements of the project management process being made to get away from the fixed time sprint limitations of using Scrum.
What is a waterfall approach?
The Waterfall approach is much older than Agile and has been traditionally used for large scale software developments for many years. They have contrasting cultures, so cannot be used together. Waterfall is a strictly formalized and linear approach to software development. Waterfall starts with the development of a hierarchy of fixed requirements; change after development is discouraged. The highest level of requirements in Waterfall is user requirements, which are described in a great deal of detail in a formalized manner, not as simple ‘user stories,’ in the form of ‘I would like to be able to ….’.
Each successive level of requirements in Waterfall then goes into the design of more and more detail; the purpose is to define a design for every specialist group involved in each stage of the development and testing. These include usability designs, technical designs, user acceptance test definitions, test plans, and program specifications. The design does not evolve over time in Waterfall, it is fixed at the end of the design stage. All succeeding stages in Waterfall use this design as the baseline for their activities, presuming that the design is still valid. This is often not the case,
This is a typical sequence of events in a waterfall approach to product development:
Gather requirements for the complete product.- Document requirements.
- Sign off requirements.
- Complete designs.
- Sign off designs.
- Code and unit test against requirements.
- Perform system testing against requirements.
- Perform integration testing.
- Perform usability testing.
- Perform user acceptance testing (UAT) against requirements.
- Prioritize any defects discovered.
- Fix any high priority issues..
- Create release documentation, including a description of the outstanding defects and any functionality that hasn’t been delivered
- Deliver the finished product.
In a true waterfall development project, each of these represents a distinct stage of software development, and each stage has to complete and be signed off before the next one can begin. There is also typically a stage gate between each; for example, requirements must be reviewed and approved by the customer before design can begin.
In contrast, the sequence of events in Agile can be summarized like this:
- Gather requirements as succinct, discrete user stories, and store in the product backlog.
- Select enough items from the product backlog that can be completed in one increment.
- Code and test in parallel.
- Deliver the product increment.
- Go back to step 2.
One of the greatest challenges with Waterfall is that it is difficult to capture a complete and comprehensive set of requirements and detailed designs. Frequently, users don’t’ always really know what they want until they can see it. Customers are not always able to visualize an application from a requirements document. This often means that customers are dissatisfied with products delivered using a waterfall approach.
Conclusion
Agile methodologies have transformed how work is done today in a wide range of sectors and industries. Organizations all over the globe have seen the benefits from valuing individuals and interactions over processes and tools, working products over comprehensive documentation, customer collaboration over contract negotiation, and responding to change over following a plan.
The great ideas that were initially confined to software development have now been taken up by areas that were unthinkable back at the inception in 2001. Time has taken the application of the principles set out in the Manifesto from niche specialists to workers at all levels. Which particular methodology is best for you and your organization is dependent on your own circumstances and needs, but you can be assured that there is something that will work for you and your customers, both for now and for the foreseeable future.