Experiments and Data for Post-COVID-19 Work Arrangements

Almost all leaders and employees are wondering what work and business will be like after the pandemic ends. Now that approved COVID-19 vaccines are being rolled out across the globe, we can see some light at the end of the long, dark tunnel in which we’ve all been working and living. But what will the attributes of the business environment post-pandemic be, and how will we decide on them? Work from home or go back to the office? Embrace Zoom for sales calls or head back to airports and business travel? In our view, the pandemic and post-pandemic business environments present great opportunities to question long-held assumptions and answer those questions using experiments and data.

Work arrangements and their impacts on productivity, job satisfaction, fairness, collaboration, and social responsibility should be a primary focus area. Companies may need to revisit longstanding systemic issues, such as inclusion: While some organizations have made strides in this area in recent years, the pandemic has had a disproportionate effect on certain groups in the workforce (such as women and parents), which means that organizations now need to assess losses and work to recover and build on pre-pandemic progress.

This is a good time to turn to testing to guide decisions, using all sorts of experiments around factors that have been influenced by the pandemic. The experiments don’t have to be terribly burdensome in these taxing times; some may occur naturally, and many companies now have analytics groups in HR and elsewhere that can gather and analyze needed data. Some companies have gathered enough relevant data over the past year to make fact-based decisions about post-pandemic work arrangements. Others have insufficient data now but can begin conducting experiments to create data that fills in the gaps.

Types of Decisions Where Experiments and Data Can Help

One of the biggest questions for many companies is whether to require employees to return to a five-day workweek in the office. We’ve seen evidence that working from home has had benefits such as less wasted commuting time, less carbon in the atmosphere, and perhaps even greater productivity. According to a Stanford study, there is a huge range in how many days a week employees would like to be in the office post-COVID-19 — the median response is two days, but 25% of respondents said five days, and 20% said never. Some companies have already staked out radical positions, like New York-based Skillshare, which has closed down all of its physical offices. Others, like Netflix, are run by a CEO who calls remote work “a pure negative” with no benefits. Neither of these extreme policies seems to be based on extensive data or analysis, and they involve sweeping positions regarding all workers in all jobs, and across all family and living situations.

There’s no need to make blanket policies with limited evidence. Facebook, for example, has been testing (since before the pandemic) whether being colocated with a manager is crucial to high performance ratings and promotions. Its people analytics group, led by Alexis Fink, examined this in part by looking at outcomes for individual contributors who were geographically separated from their teams and managers. Fink told us that overall, they have not found an impact. At the start of the pandemic, Facebook conducted a variety of internal studies looking at outcomes for teams, and the results suggest that working remotely is neutral to favorable in terms of getting work done and getting it done well. With all this data and analysis, Facebook announced in May 2020 that it would take a measured approach to scaling its remote work strategy.

In 2020, life sciences company Merck found consistent patterns in both employee sentiments and productivity when it asked 80% of its workforce to protect their health by working from home and the remaining 20% to go into the office with extra onsite health safety measures in place. Jeremy Shapiro, the company’s head of workforce analytics and one of several executives designing new work policies based on the data, said that employee engagement measures remained strong during a difficult time. He credited a combination of leadership and team support as critical to that success and said that hybrid office/home/third-space work settings are likely to be adopted in the future. He noted that Merck’s workforce is heterogeneous and that no single policy will be appropriate for all employees.

We suspect that many organizations will come to the same conclusion as Merck, with considerable variation in what constitutes an ideal solution from job category to job category, from team to team, and even from individual to individual. What’s best for sales may be very different from what’s best for marketing.

In an extensive early-2000s study of knowledge workforce policies, segmenting workers and giving them some choice about work environments emerged as the two most important factors in work arrangement design. As Matt Phelan, cofounder of The Happiness Index, an employee engagement platform, observed, “Some people prefer to work from home, some people prefer to work from an office, [and] some people prefer a blended approach. One hundred percent of people want the flexibility to choose.” Several of the workforce analytics leaders we spoke to for this article emphasized these factors as well but planned to conduct further testing and data analysis. Facebook, for example, is committed to both segmentation and choice but also plans to study the impact of remote work on onboarding and collaborative innovation.

As the pandemic eases, companies lacking pre-pandemic data could create experiments around these options. They could have some groups work in the office every day, some work entirely at home, and some work a mixed schedule. Some outcome measures (dependent variables in the analysis) could be measured for each group, including productivity, work satisfaction, networking frequency, and even hours worked as evidenced by computer activity. The results of these experiments could become the basis for broader workforce policies. They could be communicated and defended as data-based, which would perhaps make them less subject to resistance.

Location-based variables will also need to be tested, because we can’t assume that attitudes and accommodations taken up during the pandemic will carry into the future. This kind of testing will be important for salespeople, who haven’t been able to pitch potential customers in person since the start of the pandemic, as well as for other professionals, such as attorneys, consultants, and personal trainers. Similarly, business conferences and product exhibitions that have gone virtual will need to figure out whether their customers have grown accustomed to less time-intensive Zoom teleconferences or if there is pent-up demand for a return to in-person gatherings. They should be testing a broad range of approaches, ideally with different price points, to determine what attracts the most paying customers.

Finally, the list of great topics for experimentation and data analysis can include questions involving work hours (for instance, fewer days at longer hours), supervision approaches, and employees’ roles in determining their schedules. “We certainly think that the question of how many days to work in the office is ripe for research,” Jennifer Kurkoski, director of people analytics at Google, told us. “But there is a much bigger opportunity.” What would it take, she wondered, “to shift from bounding work by hours to bounding it by outcomes? For us, there is a recognition that there is much more that we don’t know, and we’re beginning to address questions that haven’t been open for debate before.”

In thinking about where to experiment, all senior managers should be asking themselves, “As we move past the pandemic, where can we have more understanding about how to run our business in the future?”

What It Takes to Experiment

Some organizations will have expertise in the emerging but fast-growing field of people or workforce analytics. Such groups typically have skills in research methods, social science, and policy implementation. They can greatly facilitate the efforts of management teams in conducting experiments and data analysis for post-pandemic workforce strategies.

For other companies, the “test and learn” approach can be executed in the same fashion by which companies test alternative websites, marketing approaches, or capital spending plans. Off-the-shelf software can keep track of test and control groups and help interpret results.

There are, however, factors to keep in mind when designing tests. Employees who know that their work is being monitored and measured might work harder — a phenomenon known as the Hawthorne effect, named for studies done at the Western Electric Hawthorne Works plant in the 1920s. Some factors that influence outcomes might be difficult to control, such as a requirement for social distancing among workers assigned to an office. But these confounding issues can usually be managed.

Companies can increase their chances of running these social experiments successfully if they follow some proven steps to their design:

Clarify the elements of the experiment. Articulate the questions of interest, the populations and subpopulations of interest, the dependent variables (for instance, productivity or employee satisfaction), and the independent variables (such as the number of days per week in the office). Clearly state what all the key terms mean.

Document findings. Record the process for collecting and storing the data.

Cast a wide net in the analyses. Once you start looking at the data, you may well learn things you weren’t informed enough to ask when designing the experiment.

Start early and start small. There is so much to learn. And there are so many things that can go wrong!

Ideally, COVID-19-related experiments will become part of a broader approach to experimentation to test every important decision that is lacking available data. A culture of experimentation is a major component of data-driven management, and learning from tests should become a key component of the “new normal” business environment.

The COVID-19 pandemic presents a unique opportunity for trying out new ways of working. Companies can either ignore the opportunity and ride a great wave into the unknown, or they can try to develop a deeper understanding and chart a better course. Those who take the latter approach will benefit from happier and more productive workforces, greater efficiencies, and an increased ability to respond to major changes in their environments. Work arrangements adopted after the pandemic are likely to persist for decades, so it’s worth the trouble to get them right.