Harnessing Data and AI for Legal Industry Innovation

In this episode, Evan Shenkman, the Chief Knowledge & Innovation Officer, and Jennifer Mendez, Senior Director of Knowledge & Innovation at Fisher Phillips break down what it really means to build a data-driven culture inside a law firm. They share how their team went from buzzwords to business impact: rolling out viral dashboards, tapping into firm-wide buy-in, and partnering with vendors to co-develop tools powered by GPT-4. Evan and Jenn also get into the limitations of case prediction, what makes a “quick win” actually work, and why firms that obsess over the perfect model are often the ones stuck on the sidelines. Their advice? Start small. Stay close to the business. Build trust through results.

• Show Transcript

  Description text goes hereTranscription

  ENTEGRATA   |   OVERRULED BY DATA   |   EVAN SHENKMAN, JENNIFER MENDEZ

  Episode Transcript

  This has been generated by AI and optimized by a human. 

  [00:00:00] Evan Shenkman: If you are at a deposition, wouldn't it be great if you had, you know, a tool that could help, you know, when the other side is saying something that contradicts their prior testimony or says something that contradicts the documents, right? Wouldn't it be great if our systems can know? Your client was, uh, motion was filed, or some sort of argument was now alleged, or, you know, a document is filed against your client.

  [00:00:21] Evan Shenkman: That's fantastic. And, and we're working on things like that. 

  [00:00:26] Tom Baldwin: My name's Tom Baldwin. This is Overruled by Data, the podcast for law firms looking to start their data journey or accelerate the journey they're already on, brought to you by Entegrata. Welcome everybody. Today's guests bring complimentary perspectives on how law firms can leverage data to drive smarter decisions from business development to operations and beyond.

  [00:00:48] Tom Baldwin: Evan Shenkman joins us. He's the Chief Knowledge and Innovation Officer at Fisher Phillips, where he has transformed how the firm uses knowledge, data, and emerging technologies like generative AI to serve clients better and faster. He's a former litigator turned data evangelist. Evan has been. A visible and vocal advocate for how innovation can elevate the practice and business of law.

  [00:01:09] Tom Baldwin: And joined with us is his partner in crime Jenn Mendez, the Senior Director of Knowledge and Management Innovation at Fisher Phillips. She brings deep experience building knowledge systems that empower attorneys and staff to work more efficiently and intelligently. Her focus is on people, process.

  [00:01:26] Tom Baldwin: Technology has helped the firm scale insights across departments and connect. Data to action in meaningful ways. Jenn and Evan have been working together now for over 10 years. They spearheaded initiatives that go well beyond traditional knowledge management, including AI implementations, cross-functional analytics projects, and new ways of measuring success and legal innovation.

  [00:01:49] Tom Baldwin: Today we're gonna dive into their journey and learn what they've learned from leading change and where they believe the legal industry is headed next. Huge intro and props to you both for joining us. Thank you so much. 

  [00:02:02] Jennifer Mendez: Thanks for having us. 

  [00:02:03] Tom Baldwin: Yeah, I appreciate it. Thank you. So it, every time we start one of these shows, I always wanna start with like that, the aha, the epiphany moment that you both had.

  [00:02:11] Tom Baldwin: Like, ooh gosh, data is actually important. When, when was that moment for each of you? Wouldn't it be stopped becoming a buzzword and started to become a cornerstone of your work? 

  [00:02:20] Evan Shenkman: Yeah. I, I think for me it goes back a bunch of years. We, we've been doing KM stuff and innovation stuff for, you know. 10, 15 years, whatever it is.

  [00:02:29] Evan Shenkman: But for me, the first time that our attorneys really appreciated the, the power of data was probably back around 2017 ish. We were, we were fortunate to have an opportunity to, to help Lex Moana roll out their employment litigation data module. So that was something where, you know, for external data, it was the first time our attorneys would be able to see.

  [00:02:50] Evan Shenkman: Here's how long cases last, and here's what tends to happen in cases, and here's the number of cases, and here's how many times this firm has appeared in a whatever case. And it's amazing how much attorneys care about seeing how often their name pops up or their firm pops up compared to other firms and so on.

  [00:03:06] Evan Shenkman: I. And it made me realize, hey, there really is something to this. Right? And, and then it went from, from there to, to companies like Trellis that did really good stuff with, with state court analytics and how long the cases tend to take and how often are judges, you know, stricken and, and all of that stuff.

  [00:03:21] Evan Shenkman: And the attorneys started asking for this more and more and more. I. And all of that, of course is, is external data, right? It's, it's firm, it's data that, uh, comes from legal research providers or data analytics companies and so on. But it started to make attorneys say, Hey, we would love this. Same sort of thing, but for our firm's own data, right?

  [00:03:39] Evan Shenkman: Don't we have all of this stuff that we're sitting on that's really valuable that we could maybe make some use out of that? We can come up with our own answers, our own predictions, information about our own clients and our own experiences to help us be better lawyers for our clients. So. It started with the external stuff, it's then moved to the internal stuff.

  [00:03:56] Evan Shenkman: Mm-hmm. But now it's snowballed and we could talk about that for, you know, 15 hours or 20 hours. But yeah, that, that's I guess to me where it started. I. 

  [00:04:05] Jennifer Mendez: I would say it's a similar thing. I think it's been a long time. I started my career in, in the law library and so, you know, legal researchers, we deal with data all the time.

  [00:04:14] Jennifer Mendez: You know, it's just in your daily activities, whether you're, you're looking to collect information about statutes or case law regulations. Don't even get me started on legislative histories. Right. You're. It's all about collecting data, finding data, organizing the data. And so early on I understood the importance of it.

  [00:04:31] Jennifer Mendez: And then I also understood how difficult it was to manage, right? Because in the early days you compiled everything pretty much, uh, in an Excel spreadsheet if you were, you know, really fancy. Yeah, fancy, 

  [00:04:43] Tom Baldwin: yeah. Yeah. 

  [00:04:45] Jennifer Mendez: And so, you know, you start to think about there's gotta be a better way to do this, to access this, et cetera.

  [00:04:49] Jennifer Mendez: And then you have the introduction of things. Like APIs where now I have access to the data, right? I have access to westlaw's data or trellis data, and now I can do different things than what they're giving me access to with their front end. So it's been an incredible journey, but it's one that I think, you know, for a long time we were hearing big data and et cetera, all the, all these, the, the general buzzwords, right?

  [00:05:13] Jennifer Mendez: And we talk now even about gen ai is is the buzzword or gentech AI and all sorts of things. They are buzzwords and so they're not until you have to do something about, and that's definitely the case with data. 

  [00:05:26] Tom Baldwin: So when we think about data, how's the awareness evolved at Fisher Phillips from leadership to frontline lawyers?

  [00:05:33] Evan Shenkman: Yeah, so I, I think I'll, I'll start and I'll turn it over to Jen, but when I, I came to Fisher Phillips about five and a half years ago, and one of the first things that we talked about when I was, you know, interviewing and onboarding and so on. Is we want you to come here and not only build our innovation function, but.

  [00:05:49] Evan Shenkman: Take advantage of and make use of this vast amount of data that we have as a firm. And the, the story that the managing partner told me in that, in, in those interviews, he said, I want to be able to, to be, you know, sitting there with an iPad or a tablet or whatever and getting ready for a mediation or whatever, and know.

  [00:06:09] Evan Shenkman: Every single time that we've had a matter just like this, what they resolved for, when the opposing counsel, what was their typical first offer, what do they typically then come as a response? When is the best time of day to then propose the counter? All of these things should be possible, right? We have all of this data, and that was a, a really wonderful way, you know, when I joined the firm to realize they really care about this.

  [00:06:31] Evan Shenkman: They have great use cases, they have great ideas. They see the power of it. So since day one, we knew that it would be a very receptive firm for great data projects. We knew that the firm had, and you know, we're a large firm that's been handling one kind of of work. For its entire history, right? So every single bit of data that we have is immensely valuable for predicting things going forward.

  [00:06:54] Evan Shenkman: So we have the data, we have the firm with an appetite, and I knew we'd be successful if we could get some of these data projects off the ground. It, timing wise, right? I, I joined this firm at the end of 2019. I very, uh, fortunately was able to get Jenn to come join me a, a month or two later. We worked together forever before that and, and then COVID happened, right?

  [00:07:14] Evan Shenkman: And COVID was the, the immediate opportunity challenge, et cetera, to do something really, really, really data focused. And that's what, that's what we built. So, Jen, why don't you talk about that, because I think that was the way that we really were able to show our attorneys who didn't know, you know, me or Jen, you know very well.

  [00:07:33] Evan Shenkman: 'cause we had just joined the firm. What the power is, what it could do for the firm. And uh, and we were off and running. So Jen, you wanna give that story? 

  [00:07:41] Jennifer Mendez: As Evan said, it, it was the early days, uh, of COVID. It was March, 2020, the world was shutting down. There was a lot going on, and there there's not a lot of good that we can talk about.

  [00:07:54] Jennifer Mendez: I'm coming out of COVID-19. But one of those things as it relates to data is that it gave us a fresh starting point, right? There was. Nothing else. So you could collect this data from, from the very beginning. And for us, you know, we found a, a dearth of, of data on COVID-19 employment litigation trends. And as an employment litigation firm, or labor and employment firm, we knew that we needed to focus on employment litigation and figure out how this was going to impact our clients.

  [00:08:21] Jennifer Mendez: So we decided to, to develop the first COVID-19 employment litigation tracker and heat map that captured a bunch of key data points on both federal and state. COVID-19 related employment lawsuits across the country. And so it gave us an opportunity to really highlight what we could do with data and to highlight what we could do just as a brand new KM department to the firm to say, Hey, you know, we can automate these crawls of all of the data, you know, of this federal and state court findings.

  [00:08:52] Jennifer Mendez: Make sure that it's. On topic for the types of cases that we handle and that our clients are going to care about. There's obviously a integration of different APIs and there's always sort of a human in the loop, right? It's not one of those things where we set it and forget it. We knew because it was such a new topic, that we wanted to make sure we were reviewing it.

  [00:09:12] Jennifer Mendez: We were manually categorizing things to make sure that they fit in the appropriate categories. But you know, once it was done, and it was one of those things where. It, it was a pretty fast turnaround. I came to Evan with this idea and very rough mockup from Power bi and he said, this is great. Show it the management committee this week, or something like that.

  [00:09:31] Jennifer Mendez: And a week later, I think we launched. But for us, you know, rather than keeping the tracker behind the paywall or a, a request law where you needed to log in. We just made it publicly available and it was one of those things that it, it went viral for us. You know, the, the firm I remember, 

  [00:09:47] Tom Baldwin: yeah, 

  [00:09:48] Jennifer Mendez: yeah. The firm immediately left to the top Google result for CVID 19 employment litigation where it stayed for like, I think even till now, five years at least.

  [00:09:58] Jennifer Mendez: It was there, you know, it was featured in CNN, in Forbes employment law 360 law.com. Where is, I mean, think about essentially every important media source. Um, both in legal and non-legal, and it was there. And so it was one of those things where it was a quick win for us and an easy way for us to show our attorneys the value of data, the importance of data, and how we could leverage it in meaningful ways to promote the firm, to elevate the firm's brand and to generate business.

  [00:10:30] Tom Baldwin: Yeah. Before we move on to the next topic, I wanna circle back on this. So. There had to have been a point where you thought somebody said, we should put this out publicly, and there must have been some contingent of the firm that was like, no way. Or was everyone like, yeah, this is a great idea. I. 

  [00:10:45] Jennifer Mendez: For the most part, I think everyone said this is a great idea.

  [00:10:49] Jennifer Mendez: Evan was in the room with me and essentially a firm leadership. There were a few additional folks. We had COVID-19 task force that we were presenting to. There's always a thought that, well, should this be behind the paywall? But we'd already sort of tackled some of this in. The documents that we had posted, we already had a knowledge bank on the firm website and we decided to go with like a freemium model, right?

  [00:11:13] Jennifer Mendez: So some things were available publicly for free that we thought everyone could get value out of, and then there were some that you had to pay a charge for. So we thought, do we pay a fee for this? And it's publicly available information, right? If if you really wanna collect it and gather it, you do have to put a little bit of effort into it.

  [00:11:31] Jennifer Mendez: But it is out there. Let's just get it out there and see what happens. And the consensus was, yes, let's make it publicly available. 

  [00:11:38] Tom Baldwin: So fast forward, you guys have got all the success. I vividly remember seeing the COVID to 19 tracker. I was like, oh my God, these guys are geniuses. Well done. And you were also one of the very first firms to test and help shape.

  [00:11:52] Tom Baldwin: In case texts co-counsel, which was built on GPT-4, you guys were early on. So what did you learn from that early collaboration? What'd that teach you about the power and also limitations of using AI inside a law firm? 

  [00:12:05] Evan Shenkman: Yeah, it gave us a lot of great insights and I think one of the cool things about having gotten access to it so early and we, we had access to it before chat, GPT came out, before anyone really knew this world was about to change, which.

  [00:12:18] Evan Shenkman: How hard was that? To not say mind, you 

  [00:12:20] Tom Baldwin: must, when you first saw it, you must have thought, oh my God. Like, I want to tell everybody. Like, it must have been so hard to keep that under your wraps. It was, it 

  [00:12:25] Evan Shenkman: was crazy. This is a, you can't tell your spouse, you can't tell your family, you can't tell anyone. We had a small group within the firm that we were allowed to tell, and, and then that group slowly expanded, expanded, expanded, because it, you literally, you know, the world's about to change in dramatic ways, not just the.

  [00:12:42] Evan Shenkman: The legal world, but the whole world is about to change and now we've seen it a couple years later how every single person's using that instead of, you know, their search browser that they would usually use. And they're using that to. You know, to get information and insights and whatever. Everything has changed and we sort sort of had a preview of that and we're helping build with it.

  [00:12:59] Evan Shenkman: So that was, that was amazing. As for early lessons learned, by having early access to it, realized that, you know, there's some things that it's not that good at, right? We were very quickly realized that it wasn't very good at math. It wasn't very good at calculating dates, it wasn't very good at figuring out timelines for dates.

  [00:13:14] Evan Shenkman: It's good at looking at documents and coming up with a timeline, but if you're saying, you know. If you're trying to have it know at least early on, what's the minimum wage in this particular state as of today? It wouldn't know that because it thought it was two and a half years, uh, ago. 'cause that was the most recent cross.

  [00:13:28] Evan Shenkman: There were certain things it couldn't do very well. It was really good at summarizing. It was really good at extracting information. It was really good at at sentiment analysis, you know. We were throwing deposition transcripts out and saying, let us know at what portion was the, the deponent flustered or angry or getting nasty.

  [00:13:45] Evan Shenkman: And it was good at that and you would never think it would be, but it was really good at that. And we learned that early on. We learned how quickly it could answer certain questions confidently, but it probably shouldn't have answered those kinds of questions that confidently. So we learned about the value of being, having human in the loop.

  [00:13:58] Evan Shenkman: It was really good early on at answering research questions. You know, this is co-counsel 'cause it was trained on case law statutes and regs. Um, but it wasn't as good at really complicated multi-part questions or multi-state questions where essentially we could ask it. What is the answer to this legal question for Colorado?

  [00:14:15] Evan Shenkman: I. Great answer. Colorado and New Jersey. And New York. Bad answers 'cause it just would get jumbled when it had three things to do at once. It's getting better at it now. But early on, these were the types of things that we learned, the types of feedback that we gave, and then they worked to resolve those issues and get it running more smoothly and, and the models kept getting stronger, the training on the backend got stronger and we're at a point now where these are just tremendous tools to use.

  [00:14:40] Tom Baldwin: So you all have been. We're really adopters in lots of ways, and, and part of it is to celebrate successes, but also, you know, with the show we're really trying to help educate folks on like some of the roadblocks, some of the potholes that, you know, if you could go back, you could tell yourself, Hey, let's not go down this path.

  [00:14:58] Tom Baldwin: What's a project, Jen, that either data or innovation or something that, that didn't go quite as planned. And what did you learn from it? 

  [00:15:06] Jennifer Mendez: Yeah, I mean, for a long time we've wanted to have a predictive analytics tool for our cases and matters. You know, tell us what the likely outcome is of this, how long it will take, how much it will cost, et cetera.

  [00:15:20] Jennifer Mendez: For various reasons, it's, it's harder than you think. We obviously know that data is messy. Hmm. And I, we always talk about Geico, right? Garbage in, garbage out. So if you, if your data isn't in order, you're not gonna get a lot of good results no matter what technology you throw at it. You know, we have busy attorneys that need to cooperate.

  [00:15:40] Jennifer Mendez: They, they need to provide information. Sometimes, um, things are delegated and it to someone that may not understand the complexity of the data that it have to enter, right. You know, when we think about things like. A single plaintiff litigation versus a class action litigation and even a settlement for class action litigation and the named plaintiff versus everyone else, et cetera.

  [00:16:05] Jennifer Mendez: Those are, you know, nuanced pieces of data. You think it'd be easy to just extract it even now with ai, but it's still really complicated. So I think we've gotten certainly much fur further along now as the technology has evolved and we're much closer, but. Trying to take that on as a whole project in the beginning I think was a bad idea.

  [00:16:27] Jennifer Mendez: Um, and this is where I tell people where, don't let. Perfect be the enemy of good enough because, you know, maybe you can't get all of those data points, you can't predict how long it's gonna take, but maybe you can predict how much it's going to cost based on all of the, the factors that you have in the case.

  [00:16:44] Jennifer Mendez: So maybe you just launch that component first and wait on the others. So those are things that we've had to, to learn, you know, just by trial and error is to say. Maybe we take this on as an iterative project, right? Multiple phases, roll it out in individual phases, you know, that's where agile versus waterfall comes into play when you come to project planning and all of that.

  [00:17:06] Jennifer Mendez: But we, we've certainly learned to be more flexible when it comes to that sort of thing. And to roll things out a little bit earlier, gain feedback, iterate, et cetera. 

  [00:17:17] Evan Shenkman: And, and we also, to add to Jen's point, you also have to look at where the data is coming from now and, and where in a perfect world the data could come from and in what format you'd like to have that data because.

  [00:17:28] Evan Shenkman: Just because we've been capturing data in one way doesn't mean that we always have to capture it that way. And there are technology solutions now that that will help us capture the data in ways that are much more appropriate that Jenn could just take and slide right into the tools in a format that we want.

  [00:17:42] Evan Shenkman: Right. So I. With some products, um, like foundation for instance, we can set up automated surveys that go out at a certain point to say, Hey, it looks like it's been six months since anything's happened. In this case, it probably can be closed. If so, answer these four questions, right, and then we'll get it.

  [00:17:58] Evan Shenkman: Or, hey, it would be much better if we captured this at the NBI stage at the new business intake stage. And we would love to have these two pieces of information that we've never captured before. And please include that, you know, we work with the NBI team, please include that. So now, when a matter's opened, we now know this, these pieces of information because it's so much easier to do that and be cleaner going forward than trying to reverse engineer or duct tape it from what we have in the past.

  [00:18:22] Evan Shenkman: And a lot of these projects, you may say, for all this stuff, we're gonna start capturing it going forward because the data, the, the legacy data is just so messy that it would take. You know, thousands of human hours or a robot could try to do it, but get it only 20%, right? Because the data is contradictory, right?

  [00:18:39] Evan Shenkman: There might be, we might have 4,000 different documents on our DMS that all pertain to the same matter, and they all are, are somewhat contradictory because, you know, that's just the way these cases sometimes go. Maybe they, maybe there were 15 amended complaints and one of them, we have them all in the system.

  [00:18:55] Evan Shenkman: Two weren't filed, three were, and it's hard to know which one is the proper one. There are a lot of, you know, things with NS codes, right? There might be one client might have 15 different things that that client does. They, they have an agricultural arm, they have a chemicals car arm, they have a manufacturing arm, they have a whatever arm, and it's hard to know.

  [00:19:14] Evan Shenkman: If someone says, we like to know, I. What the data shows for how long cases tend to last for a manufacturer company, blah, blah, blah, blah, blah. Was it opened up that way or just big company? I'm gonna say this, 

  [00:19:25] Tom Baldwin: sorry, real quick. The, the takeaway for me in all this is that if you all can't do the prediction correct, nobody can I.

  [00:19:32] Tom Baldwin: So I, I'm very saddened to hear that, but also I, you you putting eight? No, we're getting really, we're getting really 

  [00:19:37] Evan Shenkman: close. We're getting really close. God, but I'm trying to know that there are so many different variables there that, that make it a challenge, right. We're a litigation shop. A lot of our cases have 17 causes of action.

  [00:19:49] Evan Shenkman: Right. And if the case goes away. Um, you have to know, you know, it wasn't a necessarily a hundred percent win, a hundred percent loss or whatever. There might be 12 claims that were dismissed, four claims that were settled, two claims that were adjudicated, uh, then it might go on appeal. So it's not as easy as in age cases, this is what happens.

  [00:20:08] Evan Shenkman: Yeah. 'cause age was one of the 17 causes of action. All of these things have to be thought about, taken into consideration, and you have to come up with what you think is the best approach. And I'm only mentioning this because. These are, these are challenges that you have to figure out what's the best approach for your firm, and then you have to be on the same page.

  [00:20:26] Evan Shenkman: Then you go and try to tackle it. Uh, but if you try to have one module that does everything and understands all those things, a human couldn't come up with the best approach on a lot of these things. A robot couldn't, a human plus robot couldn't. And you have to just say, what is best for our firm and what's best for this project?

  [00:20:40] Evan Shenkman: And then try to tackle it. 

  [00:20:42] Tom Baldwin: Yeah. Going back to something Jenn said, which I thought, you know, lots of firms come at this with like their eyes are bigger than their stomach. Like, oh, we're gonna do a case prediction. Okay, great. So what are the building blocks for that? And I think that's the key because you may not get to that end state, but if I can say, oh, you know what?

  [00:21:00] Tom Baldwin: I can't get to the end state of predicting outcomes, but I can predict costs, or I can predict how much motions, you know, that those are wins, they're smaller and they're a little bit more achievable. And then as you build those building blocks. You might get to the point where you're close to predicting an outcome, but you don't start with that as the goal.

  [00:21:16] Tom Baldwin: I, I think that's something that people kind of hear like, oh, it's minority report and I can just swipe and things happen and GPT will do. It's not that simple. So I love that approach. Um, where do you think most firms go wrong? Where they try to quote, do data? 

  [00:21:31] Evan Shenkman: I think, and, and Jenna, I'd love your thoughts too, but I think it's when firms to our point try, try to boil the ocean and get every single thing that they could possibly get.

  [00:21:38] Evan Shenkman: You need to start with what makes the most sense. Um, if, if you're saying, you know, to the point that you made earlier, Tom. If we're trying to figure out outcomes, outcome is the hardest thing because figuring out the outcome at a case that that may go to trial depends on the credibility of witnesses. It depends on who the witnesses are, whether they're available, what the documents show, what the data shows.

  [00:22:00] Evan Shenkman: It depends much more about the, the facts and circumstances of the case will determine the outcome. It also depends on the case law and courts might, how they might rule on it. And imagine if you're at a firm that's full 

  [00:22:10] Tom Baldwin: service, that doesn't just do litigation, it's even harder, right? 

  [00:22:13] Evan Shenkman: Right. That's a challenge.

  [00:22:14] Evan Shenkman: So that I think if you're trying to have one omnibus tool that can figure out anything, you're setting yourself up for disappointment to some extent, right? Some of these things we can tackle, but it's more of a challenge, I think. So that answers your question, I guess, as to, to. What, what should we be avoiding?

  [00:22:30] Evan Shenkman: What are the, the pitfalls on things like that? But you can start with measured reasonable, appropriate steps to get the benefit of data that still is meaningful and still will help your clients be better at predicting what's likely to happen in terms of cost. What's likely to happen in terms of how long it'll take, what's likely to happen because we know the opposing counsel, we know the judge.

  [00:22:50] Evan Shenkman: All of this can help. It can all help. We'll give you the entire 360 degree view of what will happen in this case from start to finish. Of course not because it doesn't get into the actual underlying facts and the witnesses and the dispute and the, you know, all of that. But it can get you kind of, kind of along that path and certainly much far enough along where the clients are still very appreciative as to what we're able to tell them from the data and what we're able base it on.

  [00:23:14] Evan Shenkman: And then we also compliment that with the, the experiences that our attorneys have. Right? The attorneys also have experiences that are helpful and valuable, and you put that together with the data and you now you have two really. Weighty helpful things to put together to have hopefully a really good, accurate picture of a prediction as to cost, case length, duration, what we think might happen in emotion and so on.

  [00:23:37] Tom Baldwin: I was wondering if we could, it is an adjacent topic to this, what firms do wrong, but when firms just want to get started with data, again, everyone sort of has this like epiphany or this like, I want to do this huge thing, but how should firms. Especially if they're feeling a bit overwhelmed or under-resourced.

  [00:23:56] Tom Baldwin: What's your, just get started advice for firms in that position? 

  [00:23:59] Jennifer Mendez: I think in terms of firms getting started, there are, there are a number of things, as I said before, don't let perfect be the enemy of good. Don't try to boil the ocean. One thing that I always tell folks is. Leverage the data you're already paying for, right?

  [00:24:13] Jennifer Mendez: Mm-hmm. So, you know, often we get stuck on our, just the data that we have, but you can supplement your internal data with APIs and oftentimes. So not as often as I like, quite frankly. You already have access to the API via your subscription, right? Read the fine print, read the contract. Ask your vendor whether or not you have access to their API and leverage that data.

  [00:24:37] Jennifer Mendez: And if you need to pay for it, you know, make a determination, obviously assess. The API, the relationship, the content, what it is that you're looking to accomplish, and see if that's actually going to help you get to the point where you need to be. Because there are so many APIs available now that will give you meaningful data at the click of a button that it just, I think you're sort of selling yourself short if you're not leveraging APIs.

  [00:25:03] Evan Shenkman: Yeah, and, and to, to Jen's point, we do have a lot of great internal firm data on a lot of this stuff, but sometimes it's incomplete. Or sometimes it's, it's internally inconsistent because maybe the case got removed, you know, from state courts, federal court or whatever, and, and it wasn't updated in our, our initial systems to show that and so on.

  [00:25:21] Evan Shenkman: But we can now use the APIs from our vendors to be able to go and supplement and fill in the gaps or make a comparison and say it's, you know, we have a 98% accuracy between the data, but here are the gaps that we think we can fill it in. Or it's inconsistent between. If we think it's Judge Jones, your data thinks it's Judge Smith, which is it, and then we'll have the human go and check it out and then plug it in.

  [00:25:44] Evan Shenkman: So the ability to supplement, update, compliment your internal data with APIs makes the project so much easier to do than just. Throwing people at it and say, Hey, go through all of these cases that we have, and just try to make sure the judge is right. Make sure the venue is right. Make sure the opposing counsel's right.

  [00:26:03] Evan Shenkman: Make sure the attorneys at your firm are right. Make sure, blah, blah, blah. 'cause that takes forever. Mm-hmm. Uh, and it's not the kind of work that people wanna do. It's not the most scintillating work unfor Tom. I know you would, you would do that forever. You're a data guy. This is probably, you know, you're, you're, uh, you're dreaming about stuff, projects like that, but you need to use, you need to let the technology help you and the APIs just are a wonderful way to do it.

  [00:26:25] Tom Baldwin: I was on another show in talking about this. The third party data is the ultimate cheat code. To accelerate your data journey because a, even if you had people doing it, they're not gonna be perfect either, and they're gonna make mistakes. So even if the feed comes through, let's say it loops through your historical matters, where you're, you're gonna have to have someone go back, even if it gets at 80% right, that's 80% more than you had before, 

  [00:26:47] Jennifer Mendez: and it's not the push of a button as opposed to hundreds of million hours.

  [00:26:51] Tom Baldwin: Exactly. Exactly. And if it is wrong, let's say it's wrong consistently, you can easily update that too. And so I love that. I think that's, um, I, I wish I could get that, like a bumper sticker for that. If you were, and, and maybe this leans into the next thing, if there was one small high impact project you could do in let's say a month or two, what would that be?

  [00:27:10] Tom Baldwin: Abd 

  [00:27:11] Jennifer Mendez: That's such a good question. I think you have to find opportunities, identify pain points, right. I. I couldn't have told you five years ago that the COVID litigation tracker was going to go viral, and that took a week, maybe two weeks, hops to Bill. It's crazy. Zero. Do. Z zero investment on our part because we only have, 

  [00:27:33] Evan Shenkman: besides, besides the billion dollars, we pay Jenn $0 worth every penny.

  [00:27:39] Evan Shenkman: Yeah. I 

  [00:27:39] Jennifer Mendez: mean, hey, we were leveraging tools that the firm already had, right? We we're a Microsoft shop, so we had Power bi. We already had access to the APIs that I leveraged. We also supplemented with publicly available data, so it's just. Well, finding ways, and I, that's one thing that I will add beyond APIs, if you don't have access to APIs, if you don't feel comfortable working with APIs, you know, think about the, the publicly available data that you can access and leverage in different ways, whether it's on government, agency websites, state court websites, et cetera.

  [00:28:11] Jennifer Mendez: There are so many opportunities out there, you just have to find them. And so. Identify what your pain points are, identify whether or not it's a quick win. Again, don't try to boil the ocean. You know, our dashboards, all of the dashboards including the COVID-19 employment, litigation dashboard, and heat map, they've all involved.

  [00:28:29] Jennifer Mendez: They started off as one thing with a certain number of visualizations, and as we've gotten feedback and access to more data, we've iterated and added more, and added more and added more, right? And so again, be okay with iterating. Be okay with sort of a. I don't wanna say a mediocre, but you know, not your end goal.

  [00:28:49] Jennifer Mendez: Be okay with putting that out there and improving on that, and there's value in showing improvement. It gives you an opportunity to promote it right again. Right. To put a statement out there that says, by the way, our COVID to 19 employment litigation tracker now includes outcomes. So there are opportunities.

  [00:29:05] Jennifer Mendez: To make sure that you have access to, to valuable data, that you're publishing it in a way that people can consume it, and it's easy to understand and to iterate and make it better than it was when you launched it. 

  [00:29:17] Tom Baldwin: I love it. So you said mediocre and it reminded me of something. The first time I introduced the concept of MVP, someone said, oh, mediocre, viable.

  [00:29:24] Tom Baldwin: No, no, no. Mini. Each one of those words is important. Minimum viable means it works and it's a thing that, you know, all three of those words mean something. Um, I love that. 

  [00:29:34] Evan Shenkman: You know, I was just gonna add to, to what Jenn said with the COVID tracker, we, you know, we had easy readably access to data from public sources that was already clean and organized and so on.

  [00:29:45] Evan Shenkman: If you're looking for a quick win with internal data, the easiest thing to do would be see where your data is the cleanest and most amenable to a project, and then try to find a good project that works for that. Right? If you're trying to impress your firm with a cool data project, don't pick one that's gonna be a three year project.

  [00:30:02] Evan Shenkman: The data is terrible and it's contradictory, and there's not an easy API to clean it up and there's not, you know? Yes. Now Gen AI could help cleaning up some data, but. Find a product for which you have good data. If you have great financial data, then you could do, you know, some sort of great data project with your financial data.

  [00:30:18] Evan Shenkman: If you have good time and billing data that has good time entry narratives that can now that are broken down into task and phase codes, do that. If you only have task and phase codes for insured matters because that's the way your firm does it, then do a data project on insured matters. 'cause that's where you have the best data.

  [00:30:34] Evan Shenkman: Find whatever it is that will make you look like a superstar with your first data project that you could roll out in, you know, a month or so. And they'll say, oh my God, this is amazing. We want to get you a team of five more people to do things more like this. And, you know, then you're off and running. Or we wanna hire Tom to do more things like this, then you're off and running.

  [00:30:50] Evan Shenkman: But look where the easy data will be for your project. And then I'm sure you can find good data projects that will do what you said, which is, you know, get a good quick win. Mm-hmm. And you need a quick win. I know it sounds trivial to. Try to find a project that needs a quick solution for a quick win, but you need a quick win to build momentum at a firm to start to do cool stuff like this.

  [00:31:10] Evan Shenkman: And then you'll get buy-in to tackle the harder ones that take more time, that take some resources, financial or otherwise, to build them, to extract the information in a more powerful way to. Pay for an API that will do something that you can't do with your other resources and so on. But you need some quick wins.

  [00:31:26] Evan Shenkman: And quick wins are easiest where you have good data to start with. 

  [00:31:31] Tom Baldwin: I love that. And it's, um, it's something I think people forget about and, and it sounds like you all both are, uh, plugged into the business in the sense where you're hearing pain points. And I think sometimes this gets lost on folks where.

  [00:31:43] Tom Baldwin: They're trying to search for problems. The COVID-19 thing, you all jumped on it quick. You, you turned around something fast 'cause you were connected to the business. And Evan, to your point, I could build something with really cool data, but if it's not something germane to what the firm cares about, then that's the other problem.

  [00:31:57] Tom Baldwin: It's gotta be relevant and easy-ish. Right. It sounds like you all are, are connected there. So I want to pivot a little bit to talking about sort of the future of data, data and analytics and ai. Again, you all are very early adopters with ai. It's changing so fast. I mean, you're probably looking back at some of the struggles that like co-counsel had when it first came out compared to the things it can do today.

  [00:32:19] Tom Baldwin: Where do you see the biggest opportunities as you look ahead? 

  [00:32:23] Evan Shenkman: So, so one of the things that I think is, is coolest to me, and I know, you know, the big thing now, everyone's talking about agent AI and super opportunities there, and we're seeing them and we're playing with them, you know, behind the scenes at our firm and it's great.

  [00:32:35] Evan Shenkman: What I'm most excited about, and we're also seeing this now in, you know, actually in operation is sort of the, the ambient agentic ai, right? I, I, I'm really excited about AI that does not require a human to go in and prompt it for what they need. Because it is in the background. It is looking at what you're doing and it is in real time providing insights or assistance or whatever without having to ask.

  [00:32:59] Evan Shenkman: We started something like this a year and change ago, uh mm-hmm. With an automated digger system, which has been fantastic. And, and you know, we could talk about that. And it, it sort of looks at a lot of different things, like all the case filings and agency filings and agency charges and investigations and.

  [00:33:15] Evan Shenkman: Compares that to our client list and our prospect list, and it will automate an email that will go to the attorneys and say, here are all the clients that need your help and here's why they need help. And we built in a a Gen AI initial case strategy report into that too. So, you know, the attorney shows up in the morning.

  [00:33:31] Evan Shenkman: Didn't know their client or prospect was sued, but now they get an email that gives them all this information that's helpful for them to help the client really quickly and let them know, you know, we have, here's what we have, here's our thoughts, and so on. So that, that idea, right, just ambient meaning it's just doing it in the background on its own.

  [00:33:47] Evan Shenkman: It knows what to do and it's giving you what you need. I love that and, and the technology. So that project in of itself is fantastic. And you know, Jenn built like an exceptional tool that does all that and it's wonderful. What we're also seeing now are other ways that ambient AI can help. So, you know, if you are at a deposition, wouldn't it be great if you had a tool that could help, you know, when the other side is saying something that contradicts their prior testimony or says something that contradicts the documents, right?

  [00:34:14] Evan Shenkman: That's fantastic and, and we're working on things like that. Wouldn't it be great if, you know, our systems can know. Your client was, uh, motion was filed, or some sort of argument was now alleged, or, you know, a document is filed against your client. Wouldn't it be great if it would know that, and it would sort of send a message to the attorney saying, Hey, this was just filed in your case.

  [00:34:35] Evan Shenkman: Here's a proposed response. Here's a proposed email to the, to the, your client letting 'em know what's going on. Here's the timeline. Here's when it's due. Here's what could happen if you lose. Here's what could happen if you win. Here's information about the judge who's gonna be ruling on it. Here are the other cases that the judges ruled on that are something similar, and we have a nice little package that you could then have as a starting point.

  [00:34:54] Evan Shenkman: All of these things, right? The agentic part is, it's gonna go and look and figure out, okay, here's the venue, okay, now here's the judge. Okay, this must be, this, is it this the opposing counsel? Yes it is. And it'll go through that process, package it together. But what I really love is that it's doing it without an attorney having to say.

  [00:35:11] Evan Shenkman: Here was a brief that was just filed. Can you please go and run a shepherd's report and can you please go and pull the cases cited in? Can you please summarize those cases? And I'm gonna go to a different platform. Can you please give me a timeline than a different platform? Can you please write this brief for me?

  [00:35:25] Evan Shenkman: But it could. Package it all together, but starting on its own. And that to me, we are so close to that. We're actually doing things like that already, but that's where the practice of law is going. All of it will be based on data that we have internally, data that exists externally, pulling it all together and having really smart people on the vendor side, building these solutions for us and letting us build some of them ourselves.

  [00:35:49] Evan Shenkman: That to me is what's super exciting. 

  [00:35:51] Tom Baldwin: Hmm. Jen had this in her head. I'm sure the first time she got a courthouse news dinger and thought, there's gotta be a better way. 

  [00:35:57] Jennifer Mendez: Yes, absolutely. That's actually really 

  [00:35:59] Tom Baldwin: true. 

  [00:36:00] Jennifer Mendez: It, it's exactly the, those dingers are exactly what started the whole our CAM alert system and automating that process, and I, it almost makes me wanna retract to my previous statement, which is to say that that's where you should start your data project should start in business development.

  [00:36:16] Jennifer Mendez: Right. Because when you can impact the bottom line. That's a quick, easy win and no one's going to argue with you there. I think there are studies that show that something like 90% of a law firm's business comes from existing clients. So if you can tell your attorneys that their existing client has a filing against them, and here's how you can help, I mean, why not?

  [00:36:37] Jennifer Mendez: And um, it, it's been a huge win for our firm. It paid for itself. Obviously there was an investment, you know, you're leveraging APIs and there's Python scripts and there's. Click databases and all sorts of things running in the background. We partnered with a great vendor on that project. There was an investment and it essentially paid for itself in the first six months in, in just not, not just by the notifications, but that we could actually track.

  [00:37:04] Jennifer Mendez: We got these 50 new matters in the first six months that we would not have seen otherwise had it not been for this platform, and here's the amount of money they've generated in that time. And so, yeah, I mean, those are quick, easy wins. And 

  [00:37:18] Evan Shenkman: we didn't have to have the library reviewing all the new case filings and try to match it to our clients and prospects, which again, is not the kind of project that our folks in the library wanted to do.

  [00:37:28] Evan Shenkman: So they were so thrilled when we were able to have a bot handle 95% of that only send to the library folks. Ones where it had a confidence level of between 60 and 80%, I think, where they could then go and train it going forward. Yep, this is our client. Nope, that's not our client. It's just a company with a similar sounding name.

  [00:37:45] Evan Shenkman: And then it will get better and better and better. But yeah, I think we stopped tracking it because it just, the numbers got so big and we, we didn't have to to do it anymore. But the number of cases that we never would've known about with our legacy system that we now knew about there. Talk about buy-in for efforts like this when your firm sees that and then we come to them with our, our next hair brained idea.

  [00:38:05] Evan Shenkman: They're like, go for it. We trust you. Right. 

  [00:38:07] Tom Baldwin: That's a good point. The, the, the point you made earlier, Jen, sometimes that's a good starting point. Now it's a bigger effort. It's not, that's not a one week project, but Absolutely. Impact. It's not a one week project. Impact. Impact in the bottom line is huge. Yeah.

  [00:38:16] Tom Baldwin: You all have recently a appointed a director of ai and how has that shifted your priorities and opened new opportunities? I would say, 

  [00:38:25] Evan Shenkman: uh, yeah. So we, we brought on board Pritesh Patel who is, uh. Computer science Maven, ai, machine learning Maven, and he, he came from Walmart where he was doing gen AI stuff for Walmart, figuring out huge number problems.

  [00:38:40] Evan Shenkman: Like we need to have, you know, we need to order more bananas before the bananas go out, and all of our stores and, you know, X, Y, and Z and, and how can we anticipate that and the supply chain issues and so on. And, you know. But do that now for every product. So lots of big numbers in his head and great ways of going about this.

  [00:38:55] Evan Shenkman: No background in legal, but we really like that because he's coming with a completely fresh start where we could explain, you know, he could go and sit and, and process map out what we actually need and so on. So the reason we, we hired protection and I know other firms are thinking about the same sort of thing as well, and this calculus as well.

  [00:39:12] Evan Shenkman: There have been some just fantastic opportunities that our firm had to do stuff that others haven't been able to do just by. Good relationships with vendors, right? Mm-hmm. So first, you know, to get that phone call from, from Pablo. When he saw GT four back in September of 2022, was wonderful. Right. And that gave us a great opportunity and a little bit of a headstart.

  [00:39:34] Evan Shenkman: And we had similar opportunities with Trellis and with HEA and with Verit and with, you know, just a bunch since then. Right. But there's only so much that we can do partnering with vendors because it's wonderful. We get a head start, we get early access, we get a lot of great benefits, but there are some other things.

  [00:39:52] Evan Shenkman: That we wanna sort of do on our own, that are not going to immediately be a product that every other firm now can acquire. And we still, we wanna still keep working with vendors because we believe in, in that whole ecosystem working where they need law firms to help them test, evaluate, ideate. We want to have early access.

  [00:40:08] Evan Shenkman: It's a match made in heaven. But there are some other things where we wanted to build them ourselves. And have that IP and have our own attorneys and clients getting the benefit of something that other firms don't have. And for that, we knew we needed to get our own person internally to help us build some of these things, to figure out what's possible, what's not possible, and then go ahead and, and set out to build it.

  [00:40:30] Evan Shenkman: And that's where, uh, where Peresh fits in. 

  [00:40:33] Tom Baldwin: And it's interesting, I, I'm not suggesting this is a wrong approach, but you all just didn't jump right in and say, we're gonna hire a bunch of, I mean. Some firms I've seen have done that. And I'm not saying it's wrong, but like how did you come to the point where that calculus was, okay, we're coming up against some points where either we have some ideas that we don't want, um, to get folded into a product, or we just wanna own our own destiny.

  [00:40:54] Tom Baldwin: How did you come to that conclusion? 

  [00:40:56] Evan Shenkman: I don't think it was by some grand design. I'd love to say it was, but I think we were. So excited and so tied up in the moment of getting these opportunities. You know, the, the co-counsel experience was a number of months where we were working with them in secret and building it, and that was fantastic.

  [00:41:12] Evan Shenkman: We, we never would think about, let's go and hire other person. Then we went from there to working with he, working with Trellis, working with the folks at VLE now on, on Vince and AI studio, which was just announced a couple days ago. Just doing really cool projects. We, we frankly, didn't have the bandwidth to also do all these things.

  [00:41:29] Evan Shenkman: On the side, on our own. Now I think we're at the point where a lot of these things are, are working. Our team is a little bit bigger now, which is wonderful. Uh, but we're able now to focus on some of these things that we couldn't focus on before because we were doing so much already in this space. And I think it's probably easier to be relying on the vendors and working on projects with vendors where you're just testing you.

  [00:41:50] Evan Shenkman: They have their own r and d, they're building, they have their coders and so on. It's harder to have your own team. We're not, you know, law firms aren't software shops. They typically don't have these big groups, so I think we waited till the right point. Right. Two and a half, three years into this, we now know what the story is.

  [00:42:06] Evan Shenkman: What, what are the best use cases, what are not the best use cases. We had a lot of experience seeing things be very successful at our firm and in the market, you know, externally and things that were hiccups, uh, in our firm and externally in the market. And now we know, okay, these are the things that we really wanna build ourselves and we gotta the point where we were ready for it if we tried to get someone.

  [00:42:27] Evan Shenkman: You know, three months in, six months in a year in, we would've had a lot more error than, than, than success. And I think we, we waited hopefully, uh, time will tell that we waited to that the right moment. Uh, but I think we waited for the right moment. 

  [00:42:40] Tom Baldwin: Yeah, that sounds right. And then, you know, Jen, speaking of sort of building up an AI competency, you've also launched an internal generative AI training program.

  [00:42:48] Tom Baldwin: You know, I was curious like, how are you all measuring success and what would you tell other firms starting down that journey of building ai gen AI fluency? 

  [00:42:57] Jennifer Mendez: Yeah, I mean for us, what I think we've learned early on just being in this space and you know, with knowledge management initiatives comes change management and understanding that people learn in different ways.

  [00:43:10] Jennifer Mendez: So, you know what, even when it comes to gen ai, it's not just, you know, we've got on demand. Training options. We've got, uh, a long training video versus individual training videos, shorter snippets to allow for busy attorneys to just watch a three minute video on a specific feature as opposed to watching the whole thing.

  [00:43:31] Jennifer Mendez: It allows people to get that sort of just in time training, right? I only need to use this feature. I've never logged in three minutes. I can learn it, go do it myself and not have to worry about it. You know, we've got. Uh, handouts or Q Gs, quick reference guides. You know, for us, there's no one size fits all approach.

  [00:43:51] Jennifer Mendez: It's trying to meet people where they are, and I go back to that just in time training. It's something we've followed quite a bit, right? Even when piloting things, you know, things like veit, okay, well who needs this? People that are going to trial. So let's find out who's going to trial within the next three months.

  [00:44:08] Jennifer Mendez: Identify who those people are, talk to them, gauge their interest in. Learning this tool and using it and being able to provide feedback because they, they also need to commit that time as well so that we can iterate and, and confirm that it's working and doing everything that it says it's going to do.

  [00:44:23] Jennifer Mendez: And then we've also invested in just doing our, our KM road show, so to speak. Right. We've got, we've got a presence at all of our firm's retreats. So obviously we have an all attorney retreat, but now we've, we also have smaller regional retreats where we go in and talk to them about, um. Uh, all of the tools that we have access to and the things that we're doing.

  [00:44:45] Jennifer Mendez: I personally love attending those because oftentimes, you know, I take the stage, I start talking about something and someone in the audience chimes in and says, I used it and it saved me six hours on X. And, uh, it, you know, the attorneys are evangelizing these tools themselves, so it it's one of those things where.

  [00:45:03] Jennifer Mendez: One size doesn't fit all, and you have to make sure that you're, you're meeting people where they are and how they learn in terms of, uh, you know, tracking success. To answer the question in it, in its entirety for us, just having the, the usage statistics, obviously, you know, we can send out surveys and have them fill out forms and get their feedback and all that, and that's great.

  [00:45:24] Jennifer Mendez: But the usage statistics are what really jump out at us, right? Because sometimes you think I've built it, they will come and then you go and you check the usage analytics and you go, oh my God, only 10% of the firm is using this amazing tool. How do I get them to use it? And so for us, it, as we see those numbers trending upwards, and it's one of those things, even when you, you look at, you know.

  [00:45:45] Jennifer Mendez: The number of registered chat GPT users and how long it took them to get to a million, it was in no time. And, and for us, we're seeing that with the tools that we roll out, these gen AI tools in terms of actually getting people to use them, the timeframe is so small to get to, you know, 60, 80% of the firm is using this now on a regular basis.

  [00:46:07] Tom Baldwin: Okay. Wrap up questions. Y'all are amazing. Thank you. Is there a book or a podcast or idea that you've been coming back to lately? 

  [00:46:14] Evan Shenkman: Podcast, uh, LinkedIn, anything that Conor Grennan puts out, I read that immediately. Whether it's video, whether it's audio, whether it's anything else, he, he takes concepts that are complicated and makes them very easy to understand.

  [00:46:29] Evan Shenkman: And a lot of this stuff is complicated and he also gets, seemingly, somehow he gets every single big news break well before anyone else gets it. So that's who I'd say to, to stay in the, in the loop on Gen AI. Again, Conor Grennan, C-O-N-O-R-G-R-E-N-N-A-N. Awesome. And we're actually super excited. He's gonna be keynoting our firm's AI conference, which is in, in a few weeks.

  [00:46:48] Evan Shenkman: Amazing. So that'll be really nice as well. But he's, he's fantastic. 

  [00:46:51] Tom Baldwin: Jen, anybody on your, uh, radar? 

  [00:46:53] Jennifer Mendez: I'm ashamed to say that I have so many, and they're all being funneled into a folder where I'm like, I'm gonna get to these. So what I have been relying on, and it's shameless plug for my friend. Stephanie Goutos, she puts out these AI weekly newsletters on LinkedIn that give you these snippets of everything that's going on with links to the podcast or the article, et cetera.

  [00:47:14] Jennifer Mendez: And, uh, it's my guilty pleasure. I'm like, okay, I feel like I've learned something. Click on this link for the ones that I'm most interested in. And, and get that information. So if you need it in snippet format. Stephanie. 

  [00:47:26] Tom Baldwin: that's a wrap for this episode of Overruled By Data. If this podcast resonated with you, if you took one or two things away from it, you want to hear more from law firm leaders that have been there and done that hit the fall button.