The Art of Balancing Autonomy and Control

Today, managers recognize that innovation requires a high level of work autonomy for their employees. This encourages curiosity, enables independent thinking, and provides an environment in which employees can experiment and test new problem-solving approaches with minimal fear of failure. At the same time, top-level management and shareholders expect managers to innovate at an increasingly demanding pace, putting top-down pressure on employees to channel this autonomy into productivity. The challenge for managers becomes figuring out how to balance autonomy and control in order to achieve organizational goals without jeopardizing innovation.

The world of hackathons brings the study of balancing high-speed, creative autonomy and administrative control to bear in many interesting ways. Both the hacking and making cultures are centered around creative autonomy, curiosity-led problem-solving, and freedom to independently build solutions. Managing hackathons requires bringing together myriad technologists, designers, and other professionals and supporting their free exploration while simultaneously helping them finish with working prototypes. In these high-pressure environments, how do hackathon organizers chart a path to success, and what can industry managers learn from them?

In the last three years at New York University’s Stern School of Business, we have studied hackathons up close to find out just that. We participated in more than 10 hackathons across different domains, such as health care technology, finance, machine learning, and assistive technology, carefully observing, shadowing, and interviewing both hackathon participants and organizers. There are many strategies hackathon organizers use that enable self-expression and high performance that translate well to a business environment. While hackathons are time-limited bursts of creative and collective energy that ultimately are very different from day-to-day organizational work life, we found that managers can harness this kind of collective energy and find a balance between desired autonomy and control for their employees.

A key insight from the research is that there is a difference in the way hackathon organizers approach the act of managing. Instead of attempting to manage the innovation process when it happens, they focus on diligently setting the stage, and then they step back. The distinction from traditional management is akin to that between directing actors in film versus theater — in the former arena, directors are expected to control and intervene in the process to perfect the finished project, while in the latter, directors focus on preparation in advance as they accept the uncertainty and improvisation which is integral to the live performance. Like a theater director preparing her cast for opening night, hackathon organizers set the stage and conditions for innovative work, giving tools and guidance at times of need, but they minimize interventions to allow for creative exploration and experimentation.

In this article, we focus on three strategies for setting the stage for innovation that can be implemented in organizations across industries.

Strategy No. 1: Set the Stage for Filling Knowledge Gaps, but Don’t Manage Learning

The Bug: Gaps in know-how. Solving problems with unknown or unclear answers is core to innovation work. In order to find solutions to these kinds of problems, individuals need to quickly synthesize and connect new knowledge and combine information from multiple domains. This often means picking up new skills — for instance, learning new programming languages, operating new machinery, or evaluating cutting-edge scientific research discoveries.

For managers, it’s important to balance the benefits of individual innovation with using what has already been discovered before. It’s important to transfer past knowledge from existing experts and tested solutions, as it is expensive and unnecessary to reinvent the wheel. Moreover, learning on the go may prove more difficult than it seems and end up taking precious time, particularly as professionals become specialized over time.

How can managers support employees’ ability to bridge domains, make new connections, and be empowered by learning experiences while still striving for an efficient and effective innovation process? In a hackathon context, this is an even more critical question. A 72-hour sprint is not the right time to take an online course or try to solve a complex problem alone.

The Hack: Create an accessible expert pool. Successful hackathon organizers set the stage before the hackathon to allow easy access to experts in relevant fields. When setting up the hackathon, they carefully consider the relevant experts needed for each challenge and have them offer support during the hackathon. Then, at the outset of the hackathon, the organizers clearly communicate their expectation that participants will identify knowledge gaps as fast as they can rather than try to learn enough to close those gaps by themselves. Setting this clear expectation is critical and helps participants refrain from trying to solve everything by themselves. Instead, participants can admit their knowledge gaps, and organizers praise those who do so for helping the team move forward faster.

In one of the hackathons, a team worked on designing a visual solution to assist individuals with hearing disabilities. They ran into a challenge in programming a translation of sounds to visuals that would be intelligible. They shared their challenge with the organizers, who quickly thought about other industries that deal with such challenges and matched them with a leading autonomous driving programmer with relevant experience.

Without the time constraint of a hackathon setting, professionals in organizations may feel uncomfortable admitting their knowledge gaps and spend significant amounts of time acquiring knowledge from scratch. Managers should aim to cultivate a culture in which identifying knowledge gaps is a sign of efficiency and not weakness. In parallel, they need to build a strong and accessible network of experts from both inside and outside the organization. This network should be accessible to all the relevant professionals via digital, fast, and open tools, instead of being kept behind managers’ closed doors. Open access to experts gives employees autonomy regarding how, when, and from whom they wish to learn, while making the learning process more efficient. Finally, it is important to champion reciprocity in the resultant network, promoting a genuine knowledge-sharing atmosphere in which peer-to-peer learning is possible and accessible.

Strategy No. 2: Set the Stage for Experimentation, but Don’t Manage Experiments

The Bug: Falling into a creative rut. Innovation feeds on experimentation and trial and error, yet the managerial emphasis on control, deadlines, and cost efficiency often steers employees away from trying anything new. Not only is there no time to experiment, but the physical work environment itself stifles experimentation: Going into the same work environment day in and day out, no matter the environment, becomes dull and does not stimulate new thinking. The same stability and order that the organization thrives on inevitably dampens creativity. Even worse is that organizational hierarchies are often manifested through the physical environment, which further demotivates bottom-up innovation. Finally, as digitization increases and monitored work becomes prevalent, employees literately become “boxed in” by their screens. How can managers encourage an inspiring and dynamic environment for experimentation in an organization that needs order, stability, and efficiency? This is a question that hackathon organizers focus on as they aim to create enticing physical spaces for innovation to happen.

The Hack: Provide “sandbox” experiences. Many hackathon organizers create a “sandbox” — a space that attracts and allows for curiosity and experimentation even when located within a regular work environment. Instead of fully transforming the entire environment, which is costly and sometimes unfeasible, they alter the traditional office design to create a stimulating environment. For instance, they might remove individual tables and chairs and reorganize the space to be more communal. Most important, they fill the space with a combination of cutting-edge gizmos, tech equipment, and crafting tools intended to encourage out-of-the-box ideation and experimentation in a playful environment. In an assistive technologies hackathon, for example, organizers transformed the work environment and decided to mount a 3-D printer on every table, regardless of the specific challenge. As a result, many participants found themselves using the 3-D printer as a brainstorming tool — printing their ideas so that experimenting, failing, and adapting occurred at a much faster pace. In another hackathon, the addition of classic woodworking tools inspired electrical engineers to rethink their complex, motor-powered, and programmed prototype in favor of designing a simplified solution that utilizes low-tech physical components in novel ways.

Many organizations can benefit from dedicating more thinking and resources to creating a stimulating work environment in which employees can freely experiment. Size is not the main issue; the emphasis should be on providing stimulating tools and technologies and a vibrant environment. Such spaces should be located in areas that are accessible to employees but that aren’t monitored. This way, employees can easily move there when stuck on a problem, freely explore their own individual or teams’ ideas, fail and adapt, or simply reenergize and refuel their curiosity. This freedom to experiment and fail without managerial control is critical.

Strategy No. 3: Set the Stage for Early Feedback on New Ideas, but Do Not Supply It Yourself

The Bug: Feedback that deters experimentation. Employees often utilize multiple creative approaches to solving a problem, but they are also prone to the availability bias, or getting “premature closure” on the first idea suggested. Receiving feedback on a creative idea, especially from superiors, can be overwhelming. Many managers provide critical feedback with the best intentions, hoping to improve their employees’ ideas efficiently and make them marketable — asking tough questions, finding faults, and poking holes in suggested ideas. This type of feedback often backfires and deters employees from sharing new ideas in early stages and even exploring such ideas to begin with.

How can managers create a supportive environment for new ideas while maintaining a high level of quality and feasibility? Hackathon organizers face an extreme version of this challenge, as receiving feedback when working under acute time pressure is even harder.

The Hack: Initiate a “first ideas” kickoff. Hackathon organizers promote new ideas by removing themselves from providing feedback and creating a safe environment in which new ideas receive very early peer-to-peer feedback. This is done, in many hackathons, in the form of a first ideas kickoff session. A kickoff allows participants to be “on stage” presenting, articulating, and elaborating on the initial idea they have in mind to solve their challenge. The organizers ask all participants to answer the same set of questions for all new ideas and set a norm for fellow participants to focus only on supportive and constructive criticism. Putting all participants and their ideas on the same playing field helps to create openness and collegiality. This happens early in the hackathon and takes only a few minutes for each team but requires them to prepare, rationalize, and improve the initial solution they have in mind and to answer important questions early in the process. The peer-to-peer feedback is not only easier to accept psychologically but also starts everyone off with a learning mindset that can be preserved during the development process.

This habit of starting to work on a project with peer-to-peer feedback, without managerial intervention, can be very helpful even in organizations that do not hold hackathons, since employees and teams tend to work on an idea for a while before opening it to feedback. Encouraging them to present and put their ideas on stage can help accelerate the innovation process. Creating a safe environment enhances the autonomy that employees feel to follow their curiosity and original thinking, while balancing the need to have an efficient learning and innovation process in the organization.

The strategies suggested above, taken originally from hackathons and adapted for contemporary organizations, allow managers to leverage the creativity of their employees and help them perform at a higher level of engagement while maintaining directorial oversight and general control. These strategies do not, however, fit the stage of execution and operation, when oversight and control are critical.

Companies often struggle to find top talent for innovation but then limit their autonomy, creating a vicious cycle, as firms that do not successfully address the autonomy-control challenge are bound to face difficulties in attracting and retaining the most creative workforce. These strategies can help break this cycle by diligently setting the stage before each innovation process instead of micromanaging it.