Agile Is Not Enough

Agility is critical for companies trying to keep up with customer expectations and emerging business trends. By adopting an agile approach, software development teams can create new products and services rapidly, transform processes, and even help reinvent the organization. But agile teams can stumble as they interact with and depend on others, so it becomes a matter of anticipating and mitigating these choke points in the organization.

Consider a credit card company that wants to update its mobile app so customers can easily check and redeem their reward points. The company creates an agile team of developers, designers, and an initiative owner who understands customer behavior and can make decisions about focus and priorities. This team updates the app in a few weeks, but it takes months for another part of the organization to provide the data feeds from the rewards system, and longer still for another part to integrate these changes into the app, delaying the rollout of the new functionality.

Customers like the new feature, but now they also want to see recent points activity when they log in. The members of the original agile team have moved on, and since everyone is busy, it takes a few months to pull together a new team. This team makes the changes but overlooks a defect that causes the update to fail vulnerability testing. Once fixed, the operations team refuses to release the code to customers without more thorough testing. Disagreements between the development and operations teams about the extent of that testing further delay the new update.

This kind of story is all too common for many companies, even those with a strong technological focus. Several years ago, this was the case at Target. The company suffered from significant technical debt built up from years and years of growth. Critical parts of the business were supported by monolithic architecture that limited how rapidly it could innovate and introduce change. This growth also meant a rapid increase in demand for technology resources — demand that Target met by significantly augmenting its staff with third-party contractors.

The obstacles that Target and other organizations have encountered in their transformation narratives lead us to an important lesson: Agile is powerful, but it is not enough. To have a truly effective digital organization, companies have to fix the speed bumps that slow down the rapid progress of agile software development. Three impediments in particular work against agile in most organizations: rigid architecture, poor talent management, and lack of a product mindset. In this article, we’ll explore each of these and introduce ways organizations can overcome them.

Rigid Technology Architecture

In IT, years of ballooning code bases and software patches have left many companies with inflexible technology architectures. In most companies, the applications that engage with customers and run the business were developed before modern architectural approaches took root, and so they suffer from this rigidity, with too many features and functions often interdependently linked. When you change one aspect of the code, the effect can be cascading.

In most cases, it is unrealistic to re-architect all of the systems to fit modern integration standards using APIs and microservices. Instead, most organizations take one of four common approaches.

• Do nothing. Some applications are too small or too infrequently updated to warrant the investment to modernize.
• Wrap and trap. Expose the application’s core features and functionality through well-defined interfaces (APIs) while creating new functionality in modern systems.
• Re-architect or refactor. Change the application design, including breaking the code base into a series of discrete and independent functional parts and removing hard-coded values.
• Replace. Develop a new application using modern architectural patterns such as microservices.

Different approaches will make sense for different areas of architecture, and executives should think about potential value when deciding which path to take. Factors to consider include the demands for new functionality, frequency of interactions between systems, the cost of current application complexity, fragmentation of valuable data, and risks of interruptions to the business.

Target made these decisions and prioritized modernizing systems by exposing commonly used critical data like item price and availability while leaving alone legacy transaction processing systems that did the job just fine. This allowed teams to focus efforts on optimizing the customer experience, like searching for and purchasing items, instead of wasting time figuring out which of seven pricing systems had the most accurate price for a given purpose.

Rethinking Talent Management

Talent is at the core of the digital technology operating model, and senior executives know they have to recruit the best talent they can afford. Unfortunately, some of the best practices of former years have actually made it more difficult to hire the right people and increase the agility of the IT shop.

For example, as companies began to realize that their IT managers did not always see problems through a business lens, they began to hire new ones with more business savvy. They were good communicators, understood the connection between business priorities and technology, and managed relationships well with the rest of the organization. The problem was that many of them were not deep technologists. As these managers hired more like themselves, the technological skills of many companies suffered.

The unchecked use of external contractors has also contributed to the talent deficit. Some amount of labor sourcing is healthy, since it can help companies fill gaps quickly, acquire new skills, or take advantage of lower labor rates. But some companies shook this piggy bank too often to meet sudden increases in demand or dodge internal head count freezes. As internal talent spent more time managing contractors, their own technical skills atrophied. Contractors gained disproportionate control of intellectual property and innovations that could deliver a competitive advantage. How do you transform a technology organization when half the staff are not your employees?

A third problem has been creating models in organizations for how products are built and maintained and who has accountability when things go wrong. Companies have embraced a model of separating where things are built from where they are supported, because it allows them to farm out maintenance and support to third parties. But who do you call when something doesn’t work or breaks in the middle of the night: the people who wrote the code or those who are responsible for supporting it? Integrating development, maintenance, and support activities at the agile team level helps eliminate unnecessary handoffs while establishing end-to-end accountability. This is a core element of DevOps principles.

To fix these problems, technology organizations will need to make big changes in order to strengthen their technical skills, reduce dependency on contractors, and clarify accountability. Today, most technology organizations have three types of roles: watchers, doers, and thinkers. But as the future of IT takes shape, these roles will need to shift. The watchers in particular — project managers, business analysts, relationship and resource managers — will need to find new roles to play. Wider adoption of agile and new ways of working will mean streamlined overhead, greater accountability and transparency, and less need for translators between business and technical staff.

Some companies recognize the need for a revolutionary change, and they are redesigning their talent model from the ground up, based on a few principles:

• Focus on the engineering manager role. Supplanting the business-oriented IT manager is a true engineering manager, with the technical chops to check others’ code along with enough business savvy to work closely with product managers and business owners. The engineering manager helps rebuild technical credibility and prowess that attract engineering talent, meaning experienced hires as well as campus recruits.
• Retool the watchers. Identify the strongest talent in these ranks and retrain them into roles that add more value. For example, project managers open to change can become scrum masters, and some business analysts could grow into product owners. Unfortunately, there are often more people in watcher roles than there are new roles in the emerging operating model — a fact based on how much more efficient and effective the new model is when you empower teams, give them accountability, and remove unnecessary bureaucracy.
• Democratize accountability. The new talent model involves doers taking on wider roles and accountability. For example, in most cases, product backlogs are being converged to include both development and support activities. When one of our client’s development managers asked, “How many times do you think our top developers are going to receive calls in the middle of the night for production issues before they leave?” the CIO responded, “I don’t know. How many times will we need to call and wake them up before they start writing better code?” As these developers came under pressure to deliver more functionality, reliability had suffered. Developers didn’t quit, as the manager had feared, but instead worked to improve the reliability because they had greater ownership for product outcomes. In parallel, these same developers work alongside the operations team to streamline integration and deployment of code, decreasing time to market while reducing defects.
• Reduce dependency on external contractors. Software innovation within a company needs to come from employees. This means triaging different kinds of work and building agile teams that combine the right mix of internal and external talent. Of course, the goal is not to blindly cut contractors in pursuit of an artificial ratio. Companies should be smart about where they want to go and think critically about which mix will get them there as they marshal resources across technology management and recruiting.

Target found that traditional recruiting efforts were not bringing in the engineering talent the company needed to move quickly in its market. The company’s dependence on contractors for technical talent had limited its visibility within software engineering communities. To put itself back on the map, Target embraced open source technology and publicized its ambitions to transform and scale its digital technology. In fact, it would have been difficult to attract enough talented developers had the company stuck with proprietary code, since many of the strongest developers prefer to work with open source. This helped attract and retain scarce engineering talent and reduce its dependence on third parties.

Product Management Practices to Embrace When Moving to Agile

In an increasingly agile world, the weaknesses of the traditional project-based, waterfall approach to development are exacerbated. Compared with an agile, product-based approach, the waterfall method is less flexible and less responsive to needs of the business and its customers.

Moving to agile addresses some of these problems, but it does not go far enough to solve them for digital products and services. Applying product management practices and disciplines can help close this gap.

• Organize around products, not projects. Organizing work around a product rather than a project aligns development directly to business outcomes. Products are organized around business problems and opportunities, often related to a new capability or a customer experience that can be improved. Technology and business teams should collaborate to develop goals and priorities. Each product gets an owner who is ultimately responsible for developing a road map, defining metrics to measure progress, and making the right decisions to keep it moving.
• Set up persistent teams. In a project approach, teams form and disband. By contrast, product teams persist through the life of the product, ready to continue work on the product as it evolves. This stability makes the team more responsive, productive, and accountable.
• Fund for the long term. Since products have a longer life cycle, they need to be funded annually to support their long-term road map. Some CFOs and controllers may be wary of this change, but a product model allows for a more consistent measure of results than traditional business case approaches because product outcomes are continually measured. Reviewing the funding on a quarterly basis allows companies to adjust for shifts in priorities or changes in investment appetite — or pursue other products with a greater return.

Digital-native companies typically use a product management model given that their main products and services have typically been software-driven, and now more traditional companies are adopting these concepts. For example, Target organized its technology to align with its business capabilities and customer experiences. Its 250+ product managers are like entrepreneurs charged with achieving measurable business results. Those that deliver stronger returns are rewarded with more resources and responsibility.

Agile is a powerful approach to delivering value. But on its own, agile is not enough, and many organizations struggle to reap the full benefits the approach can lend for technology, talent, and product management. While some detractors of agile might say it works only for digital natives, we see much broader potential. Addressing architectural rigidity, closing talent gaps, and adopting a product mindset require commitment and a sustained effort from both business and technology leaders. Eventually, these approaches may seem as obvious as aligning business and technology priorities is today. Until then, the benefits will accrue only to those willing to put in the effort to make their entire organization ready for agile.

Editor’s Note: An adapted version of this article appears in the Summer 2019 print edition under the headline “It’s Time to Rethink the IT Talent Model.”