Topics
Artificial Intelligence and Business Strategy
In collaboration with
BCG
In collaboration with
As chief data officer of payroll and benefits management company ADP, Jack Berkowitz has three primary responsibilities. One is to oversee the organization’s data overall, ensuring that functions like data governance, security, and analytics, are running well. Another is to build ADP’s data products, such as people analytics and benchmark tools. But the responsibility that’s of most interest to Me, Myself, and AI hosts Sam Ransbotham and Shervin Khodabandeh is Jack’s oversight of the organization’s use of artificial intelligence.
In this episode of the podcast, Jack describes how focusing on the outcomes the organization wants to achieve leads to better processes and results. He also dives into the topic of AI ethics and outlines how other organizations might consider assembling an AI ethics board.
Read more about our show and follow along with the series at https://sloanreview.mit.edu/aipodcast.
Subscribe to Me, Myself, and AI on Apple Podcasts, Spotify, or Google Podcasts.
Give your feedback in this two-question survey.
Transcript
Sam Ransbotham: When outcomes don’t motivate artificial intelligence efforts, how can they be successful? Find out how one chief data officer thinks about AI on today’s episode.
Jack Berkowitz: I’m Jack Berkowitz from ADP, and you’re listening to Me, Myself, and AI.
Sam Ransbotham: Welcome to Me, Myself, and AI, a podcast on artificial intelligence in business. Each episode, we introduce you to someone innovating with AI. I’m Sam Ransbotham, professor of analytics at Boston College. I’m also the AI and business strategy guest editor at MIT Sloan Management Review.
Shervin Khodabandeh: And I’m Shervin Khodabandeh, senior partner with BCG, and I colead BCG’s AI practice in North America. Together, MIT SMR and BCG have been researching and publishing on AI for six years, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.
Sam Ransbotham: Today, Shervin and I are excited to have Jack Berkowitz, chief data officer at ADP. Jack, thanks for joining us. Welcome.
Jack Berkowitz: Thank you. Glad to be here.
Sam Ransbotham: Let’s get started. You’re the chief data officer at ADP. Can you tell us about what that role means?
Jack Berkowitz: ADP, known as Automatic Data Processing, is the world’s largest provider of HR services, payroll, taxes, things like that. We operate in 140 countries. We have over 900,000 clients. Millions of people are getting paid from us every day.
I sort of have a two-sided job. On the one hand, I’m responsible for all of the data that flows through our systems. We’re a really big company. We have massive amounts of data, so [it involves] all the things that are classically associated with chief data officers — things about data governance, data security, usage of analytics.
The other side of my job — and it’s probably even a bigger job — is I build data products, and so my team builds people analytics, benchmarks, compensation information, all [those] types of products that our clients are using to take decisions about the world of work every day.
Sam Ransbotham: I didn’t hear the words artificial intelligence in there anywhere. How is that involved?
Jack Berkowitz: I also run that for the company as well, but we use machine learning throughout those processes — whether we’re cleaning the information, whether we’re building embedded capabilities in our HR applications or our payroll applications, whether we’re doing things like aligning job titles.
People would say, “Well, how hard can that be?” You know, in any given month, we pay about 21 million people. We have about 14 million job titles, and we crunch that down to between 6,000 and 8,000 job titles — so [there’s] an awful lot of very sophisticated natural language processing and machine learning to make that happen.
Shervin Khodabandeh: It seems like there’s three different roles that you mentioned that all come together. And I say this because at many companies, there are literally three different roles for what you mentioned — for data governance, for data products, and for AI — which creates maybe a bit of siloed-ness and a bit of maybe disconnectedness, because all these things have to work together. Comment a bit, please, on how it came about that it’s one person leading all three. That’s my first question.
And then, my second question is, is the AI involvement only in the data products, or is it a broader role that you have that you’re also supporting AI for the broader enterprise?
Jack Berkowitz: It’s a really good question. The thing to know about ADP is, yes, we’re a services company, in the sense that we provide, for example, payroll for about 1 in 6, or even more than that, people in the U.S. But we also are a SaaS [software as a service] product company, and because of that, there’s a whole bunch of different development organizations working on building SaaS products, whether it’s for the small businesses all the way up to the biggest companies in the world using our applications to do HR or recruiting or payroll or taxes, things like that. And because of that, this role emerged, really. It started as building data products, but to build data products and things like reporting, it grew the data platforms. And off the data platforms, it grew more and more capabilities in terms of doing machine learning, best practices.
We got into the ethical use of data and the ethical use of machine learning and AI, and that allowed us to be additive in terms of capabilities. The other thing about it, then, is, OK, well, where’s the extent? Because we have all of those SaaS applications, my teams will sometimes build the embedded capabilities for other applications. But we also enable those other development organizations to use the frameworks that we build.
We, for example, build a whole bunch of machine learning operations capabilities — things about bias monitoring and data shape monitoring — because that makes sense to be done once in a company and then allow other people to take advantage of it. We’ve seen a massive growth in people identifying themselves as data scientists over the past four years. We’ve been hiring people and everything else, but they don’t all have to learn how to do model deployment into production.
Shervin Khodabandeh: Very interesting. Is it fair to say that introduction and maybe scaling of AI more broadly outside of the data products that you do for your customers was sort of the data products themselves? The incubation of these data products opened the eye of the organization.
Jack Berkowitz: Yeah, exactly. It was exactly that. It was this incubation. And then, off the incubation, we started to see areas of opportunity and areas of excitement. It wasn’t really a top-down push. It was very much a bottom-up, where teams were seeing what we were achieving, and then other teams would come to us and say, “Hey, wait a second. We want to build a capability. Can you work with us?” And so it’s really become an organic growth.
Shervin Khodabandeh: I really love this story. Often, I get asked to talk to groups or [do] interviews with media around chief data officer roles, and there’s a question around, “What’s the right chief data officer role?” And I’ve always been saying that role has to be really, really linked to the use of data, not just to the governance of data and to building things with data. And I think you’re a great example of the right setup of that role and success with that role.
Jack Berkowitz: It’s interesting, because my career’s all been about product development or outcomes. It’s been about making sure that you have business outcomes. Are you building something, are people buying it, or are you building something and you’re coming out with a better capability? We bring that product mindset to even our data governance. Yes, we need to do governance, but it’s not for regulatory compliance, necessarily. It’s really about making sure that we understand the information such that somebody can build a good data product on top or a good machine learning capability on top. Otherwise, why are we doing all this?
I graduated right at the time of a recession. Sound familiar? I spent about 10 years in engineering consulting, mostly for DARPA, which is the Defense Advanced Research Projects Agency, which gets you involved in interesting things. From there, I decided to start a company with a few friends and got into the startup world, and then I had a great opportunity to join Oracle, maybe 11 years ago now, and really enjoyed my time there. And then I was able to bring it to ADP four years ago. ADP has really been the pinnacle of my career. I couldn’t have asked for a better situation in terms of combining all those learnings of how you watch a user doing something, how you start a business in these little startup companies that were VC-backed, to the data and the technology.
We have to run these systems 24-7. Companies depend on these systems to pay their employees, which is, one would argue, one of the most important things that exists in a company, particularly today, in today’s environment.
Shervin Khodabandeh: Can you share with us some uses of AI in the products that you build for your customers, as well as maybe those that are broadly used for the enterprise or for maybe core processes or more internally focused?
Jack Berkowitz: We’re in the HR space, so [that] runs a wide range of capabilities. One of the things that we’re doing right now [that] we’re really excited about is, we’ve used that job title information along with a lot of other natural language processing to come up with a skills graph — a 100% data-driven skills graph. A lot of other vendors do it with hand-cranked ontologies. Downstream from that, that skills graph shows up in a variety of places, whether it’s in recruiting applications, the employee profile inside of a company so that people can find new roles, things like that. We have a group that’s doing recommendations for people in retirement programs. We have a big retirement program for people in small businesses. A lot of times [with] small business, people aren’t offered health care or retirement [benefits]. ADP makes those services available to small-business people to offer to their employees. And there’s a capability that recommends to people to say, “Hey, people like you will actually invest more or less in their retirement program.” And so that’s a machine learning-based capability.
But then we also, just like any other company, use machine learning across the board in other areas. We do a lot of things in our sales and marketing channels. But, more importantly, we do a lot of things in our service [channels]. So we’re doing an awful lot right now to create a self-service environment for our clients. Our ability to create a better service environment for them creates a better experience, right? They get more accurate pay, or they get a better experience for their employees. In turn, that’s better business for us, so we’re spending as much money and as much time on making that service experience great as we are [on] making the core product great.
Sam Ransbotham: I know you’ve advocated for the idea of an AI ethics board. I’ll take the counter: Why is that important? What’s the benefit of that? Why bother setting up AI ethics boards?
Jack Berkowitz: We started it because we just felt that the pace of technology and the pace of data probably weren’t representing the values that we wanted to represent with our clients. We started it originally because we thought it was the right thing to do.
Where it’s gone from there has really been interesting. We’re learning a heck of a lot, both in terms of our own product development but also externally about how to educate not just our clients but even our ADP associates [and] in terms of how we evaluate where we want to do business, like biometrics or voice recognition, or even what data access rights mean. There’s also now, three years later, a big regulatory push, both in the EU, the FTC of the United States, the EEOC [U.S. Equal Employment Opportunity Commission], and so we’re not reactive to that. We know what to think about. We’re in a great position to deal with it by thinking a little bit ahead.
Shervin Khodabandeh: From a setup and accountability perspective, do you see the topic of ethics and responsible AI governed by a board or governed by a person advised by a board?
Jack Berkowitz: We’re much more in the latter, and we want to do that for a reason. We bring external experts onto our board. We have people from the HR domain. We’ll have people from the machine learning world. We’ll bring [in] ethicists.
We want that board to have freedom of thought. We have very structured product release processes, whether it’s for the products that we release to clients or whether it’s the products that we use internally. And we have governance throughout there, and security; if you can imagine, data security is at the top of our list at the moment. Also on the board is our chief privacy officer. So we want the board to have freedom of thought. And it’s an adviser. Project teams have to present to the board as part of going to market for areas.
Sam Ransbotham: It’s interesting, Shervin. You know, Jack, I don’t want to discount how important and sensitive your data is, because clearly it’s very sensitive data. But it’s interesting. Shervin and I talk with people; we’ll talk with people in medical and health, and everyone seems to have this moment of, like, “Oh gosh; oh, our data is really sensitive and important.” And I think maybe that’s ubiquitous now, that all data seems to be like that. I guess a lot of people can learn from the board set up like this.
Jack Berkowitz: It’s not a bad thing to be protecting people’s interest in their information.
Sam Ransbotham: It’s really pretty fascinating, because there are lots of sources of information about salary, and people self-report in lots of areas, but you’ve got ground truths on a lot of information about what actually hits their bank accounts. It gives great insight into really what’s going on in the economy.
Jack Berkowitz: It creates a unique capability, both to be able to provide that information to our clients or to their employees or associates, but also to treat it properly, right? We have a great opportunity to treat it properly. And so all the levels of data security, all the levels of all the great things CDOs care about — you know, data governance, providence, lineage — we have a wonderful opportunity to practice the field.
Sam Ransbotham: One area that I think you probably are interested in mentioning is this idea of comparing DEI [diversity, equity, and inclusion] metrics. That’s a great place that you’ve been able to provide benchmarking and give insight into what’s really going on versus what people would like you to think is going on.
Jack Berkowitz: Yeah. It’s a great point. The company actually started that in 2017, [when it] published its first pay equity explorer, which allowed companies to take a look at how they were doing in terms of pay equity gaps for disadvantaged groups. Now we have the benchmarking capability that allows a company to see, for their location, for their industry, for their company size, how are they doing in terms of creating a diverse environment, and then also, how are they doing not just bringing people in, but actually advancing them in their careers?
By bringing all that together, by using our benchmarking capability, by solving a problem, by looking at an outcome, we’ve had great success.
We can run multiregression analysis down to the individual — not just inside their company but against the diverse population in that local geography or industry. But then we can say, “OK, here are four or five budget scenarios.” Because it’s one thing to say, “Hey, you have pay equity issues.” But, you know, maybe the company has budgetary things, so they can make some choices about budgetary scenarios, and it tells them, “OK, if you want to close this budget, these are the people that you’re able to cover.” And so they basically can change that. They come out and then, boom, they can make those changes straight into people’s paychecks. And it’s a meaningful impact.
Shervin Khodabandeh: This is wonderful. The question I have, which would be very interesting for our audience, is, where do you get started? Because we see … I mean, I see this in my work, [I] see this with Sam when we interview and research the topic of AI deployment, there’s a question around, how much do you build capabilities before you begin to monetize or commercialize or build data products or use cases, versus how much value- and use case-driven you are. And I’m really interested in your perspective, both for ADP and any advice you have for others in the early stages of their journey.
Jack Berkowitz: The way I’ve always looked at it is, if you’re building a product, whether you’re a startup company or any other, is, build the thread from one corner of the piece of paper up to the other corner of the piece of paper. And use that thread — in other words, a use case or two — to help you define what you need in your situation with your company at that time.
You could say those are prototypes, but in my mind, a prototype is useless unless you actually try to have an impact with it, because you don’t learn about how to measure outcome. You won’t learn how to measure what you actually need.
The thing that’s been lost in machine learning and all the buzz over the past six, seven years is that all machine learning and all AI is targeted to an outcome. To me, it’s really about fielding some capability. Off of fielding that capability, you’ll learn what levels of machine learning operations you need, you’ll learn what levels of data you need. And I know that there’s 1,400 vendors. Matt Turck has the great FirstMark Capital matrix of it, and I remember when that was only 30 vendors, by the way.
I know all 1,400 vendors will tell you [that] you need to buy their stuff right away, and that’s just not true. That’s just not true. You’ve got to buy some of it, though, in order to get that initial thing fielded.
Shervin Khodabandeh: Yeah, I’m so happy you say that, because that’s, honestly, been the unlock we’ve seen both in our research, Sam, where we see that firms that just take technology first maybe get a little bit of value but there’s a big piece they don’t get. But also, in our work at BCG, that’s been the major unlock, to be value-driven and outcome-focused. And I like how you talk about the thread, because you cannot just build these things in silos, but it doesn’t have to mean that you build … the full stadium before the baseball [game] begins. You could start playing.
Jack Berkowitz: Exactly. At a company before ADP, we used to call it a “Field of Dreams business plan.” It’s like, “Nobody ever invented baseball; why are we building a baseball field now?”
The whole idea is, get that thread working. And, you know, maybe it’s not all the way connected. Maybe you still have somebody standing up with a floppy disk and running to the other computer to make it all work, but at least you have an idea. And then you can broaden out that thread over time. That’s all.
Sam Ransbotham: For our listeners, floppy disks were things that people had to put into computers to store information.
Jack Berkowitz: Yeah, thank you, Sam. Thank you, Sam. You can see it — you’ll actually see it on icons on old Macs and old PCs, so, yeah.
Shervin Khodabandeh: Jack, thank you so much. This has been so wonderful and insightful. This brings us to the next section of our show, where we ask you five questions.
Jack Berkowitz: Great.
Shervin Khodabandeh: And we expect some quick reactions to these questions. So I’ll start. What’s your proudest AI moment?
Jack Berkowitz: My proudest AI moment is when my algorithm went into production.
Shervin Khodabandeh: Very good.
Sam Ransbotham: That ties in well.
Shervin Khodabandeh: What worries you about AI?
Jack Berkowitz: I think the next reoccurrence of the AI winter. Having been through it the first time and seeing that come short. Let’s not overpromise.
Shervin Khodabandeh: Mm-hmm. Your favorite activity that involves no technology.
Jack Berkowitz: Kayaking on the Chattahoochee River.
Sam Ransbotham: Ah, been there, done that! I’ve kayaked Chattahoochee many a time. I’m actually from Atlanta, from Smyrna.
Shervin Khodabandeh: The first career you wanted: What did you want to be when you grew up?
Jack Berkowitz: I wanted to be an astronaut, like every other kid born in the ’60s.
Shervin Khodabandeh: Your greatest wish for AI in the future?
Jack Berkowitz: My greatest wish is that we help people live better lives.
Shervin Khodabandeh: Thank you. Very insightful.
Sam Ransbotham: Jack, great meeting you. I think there’s a lot here that people can learn, particularly some of the details about how you’re organized. I think that’s something a lot of people can learn from. We really appreciate your taking the time.
Jack Berkowitz: Thanks, Sam. I really appreciate the conversation.
Sam Ransbotham: Thanks for joining us. Next time, Shervin and I talk with Ameen Kazerouni, chief data and analytics officer at Orangetheory Fitness.
Allison Ryder: Thanks for listening to Me, Myself, and AI. We believe, like you, that the conversation about AI implementation doesn’t start and stop with this podcast. That’s why we’ve created a group on LinkedIn specifically for listeners like you. It’s called AI for Leaders, and if you join us, you can chat with show creators and hosts, ask your own questions, share your insights, and gain access to valuable resources about AI implementation from MIT SMR and BCG. You can access it by visiting mitsmr.com/AIforLeaders. We’ll put that link in the show notes, and we hope to see you there.
