Pattern matching with LLMs

When I was in law school, I wrote that I loved the exams. I'm not sure this was a view shared by many of my classmates, but I loved that they rewarded the reasoning, not just the answer. As my faculty advisor framed it, three things mattered: statutory construction, case analysis, and persuasion. The law was mostly details, and details could be looked up. We were being judged on how we broke down the problem, drew from a repository of patterns, and made the case for our analysis.

At the time, I wrote:

This really isn't very different from what most developers do on a daily basis. We recognize patterns, building them from existing, well-known solutions. We take a problem and break it down into manageable pieces. And many of us probably spend a lot of time trying to convince other people that we actually know what we're talking about and that our analysis is correct.

That connection started to feel like a universal truth, and it is one I've fully embraced as a generalist. One benefit of studying computer science at a liberal arts school was that it emphasized algorithms, approaches, and learning how to learn instead of tying me to a particular technology. That had already served me well in a field that never stopped changing. I began collecting as many patterns, solutions, and ways of thinking as I could, because each one gave me something else to pull on when the next problem showed up. The first step feeds the second.

LLMs make that generalist bet much more useful. I can recognize a shape in one domain and use ChatGPT to reach much further into another: fill in specialist detail, find related cases, and test whether the analogy holds. The model gives me access to a much wider pattern repository without requiring me to become a specialist in all of it.

Of course, ChatGPT can make a weak analogy sound completely convincing. Just as before, I have to apply judgment and taste, check the evidence, and persuade myself (and anyone else) that the pattern fits.

Thinking