How to use AI as a practice coach for knowledge work skills, not just a productivity tool
Nate B Jones of AI News & Strategy Daily argues that knowledge workers need to shift from thinking in job titles to thinking in trainable skills — and that AI makes genuine skill practice possible for the first time at scale.
Summary
Nate B Jones presents a framework for treating knowledge work skills the way athletes and musicians treat physical skills — as something to be deliberately practiced, measured, and improved. He opens by citing a 2019 blog post by economist Tyler Cowen, who observed that knowledge workers, unlike athletes or musicians, essentially never train — they only perform. Jones argues that AI now makes it possible to close that gap by providing consistent, rubric-based feedback on the artifacts knowledge workers produce. He identifies five core skills — judgment, orchestration, coordination, taste, and updating — and walks through how to build practice loops around them using real work documents, human-defined rubrics, and AI scoring. He also extends the framework to team management and hiring, arguing that the same rubric used to develop people internally can be used to evaluate candidates more honestly and consistently than traditional behavioral interviews.
Key Takeaways
FULL TRANSCRIPT
The core argument: knowledge workers don't train
Nate B Jones: We need to move from a jobs format to a skills format for our roles and our career growth, and no one's ready to talk about it. That's what this video is all about. How do you think about your job differently and think about it in terms of skills you can train and improve, preferably with the help of AI?
One of my inspirations for this was a 2019 blog post by Tyler Cowen where he talked about the idea that athletes train, musicians train, performers train, but knowledge workers really don't train. We don't train. There's no knowledge work equivalent of shooting free throws. And so I started to ask myself: what does it take to do something like a pianist practicing scales, but for knowledge work? And is there a way to start to address this in the AI age that helps us think about skills differently than we traditionally have?
Because, to be honest, traditionally our assumptions about skills have been so loaded into jobs that it's literally baked into our software. If you've ever been a hiring manager and you've ever used a software tool for hiring, for compensation estimates, for promotions — do you know what it starts with? It starts with the assumption that you need to layer specific skills into a job post. It's as if we can't imagine a world where skills might exist independently of a role. And yet that is exactly the world we're headed toward. We're headed toward a world where skills are something that we acquire because we can use them with AI to get meaningful work done. We should be measured on our outcomes, on our ability to drive with those skills — not necessarily compensated just because we have job title A or job title B, product manager or engineer.
What practicing actually looks like for knowledge workers
So in that skills world, what does practicing really look like? I know that we talk about this physically and I think that metaphor is helpful, but I want to get it into the knowledge work space because we just haven't talked about that enough.
In the physical world, you think of skills as being fractal — tiered. If you are trying to practice your fluency with the piano and you're moving your fingers and playing the scales up and down, part of that is the sub-skill of finger movement in a pattern, part of it is the sub-skill of how much pressure you place on the keys, and part of it is the sub-skill of the speed of movement. Each of those can be practiced and repeated, you can get feedback, and you can progress.
For knowledge workers, we need to find a way to get to narrow situations with repeated specific feedback that's designed to strengthen a particular pattern of recognition and response in our brain — so that we get better at our skills. Because otherwise we do our whole careers as live performance, and that's an extremely inefficient way to learn.
So what does that look like? The good news is I think we have never had a better chance to do that than we do now in the age of AI, because AI gives us the chance to have custom feedback on practice that we just never would have been able to scale otherwise. It's just that most of us aren't doing it.
Three structural barriers to knowledge work practice
It's tempting to say at this point that knowledge workers are lazy, but it's structural. I don't believe that we are lazy. I believe our environment fights against this approach to practicing our skills in three different ways.
Number one, we live in a world with fuzzy outcomes. In basketball, the ball goes in or it doesn't. You shoot the free throw and you miss or you make it — it's a clear signal. In product, or strategy, or leadership, or engineering, "good" can mix in so many different dimensions. It can be confusing — speed, quality, politics, relationships, risk. There's no single bit that flips from zero to one.
The second reason this is difficult is that we get really delayed and noisy feedback. You might make a big decision in Q1 and learn in Q3 at best whether it really paid off. Meanwhile, the market may have shifted, a competitor launched something, a key hire left. You almost never get the clean comparison: if I had written the spec differently, we would have avoided X or Y event.
The third issue is low repetition. A serious musician is going to play scales hundreds of times a week, but how many truly consequential decision docs do you have? How many product specs, strategy docs, technical architecture memos do you write in a quarter? If each one of these is entangled with real money and real people, there are no low-stakes sandboxes in traditional career pathing. The default is that most of us spend 95% or more of our so-called reps on live games. We're practicing in front of the crowd, practicing literally for our careers. That's better than nothing, but it's not the same.
The five core knowledge work skills
So the next question I wanted to ask — I wasn't satisfied with just a general challenge — was: what are some skills that are repeatable and practicable that we could talk about in the age of AI?
I would argue there are five that keep showing up.
Number one is judgment — how you frame decisions, how you define your options, how you choose when conditions are uncertain.
Number two is orchestration — how do you turn fuzzy goals into concrete workflows that humans and AI can execute together? Can you bring clarity out of ambiguity?
Number three is coordination — how do you move groups of humans through ambiguity without creating more chaos? You are still going to need the skills to coordinate. And as agents get better, you may need to learn the skill to coordinate agents and humans together.
Number four is taste — do you have a meaningful quality bar for your product, for writing, for design, for strategy? Do you have a sense of what is good, and can you talk about it and improve it like a skill?
And number five is updating — how do you change your mind as evidence and context shift without getting whipped around by the noise? What is your heuristic, your rubric? How do you think about updating your priors and changing your mind in meaningful ways?
Skills live in artifacts, not adjectives
Now, none of these really live in a LinkedIn tagline. They live in what you write and leave behind — what we could call artifacts.
Judgment can show up in your decision documents, in experiment designs, in prioritization write-ups. Orchestration can show up in handoff documents, in specs, in the way you plan a project. Coordination can show up in emails, meeting notes, stakeholder maps. Taste will show up in how your UX looks, which examples and metaphors you pick. And your ability to update will show up in how you evolve your plans over time, in the written reference to rationale.
The key is that these skills are not adjectives. We name them as adjectives and associate them with roles as adjectives, but really when you come right down to it, they're not. They're patterns in the things that you produce. And once you accept that, you stop arguing about who's strategic in the abstract, and you start looking at how people actually write, how they behave, and how they decide. This has always been the gold standard in behavioral interviewing, but we've really struggled to get to this level of clarity — especially post-AI.
What AI actually changes
So what does AI actually change? AI is not a magic brain — I say that all the time. AI is a tool that can read text. It follows instructions and it can apply a rubric consistently. This is beautiful because it gives us a wall to practice against.
Your first step, if you're serious about practicing, has nothing to do with your models. You just want to pick one artifact that matters for your team — like a decision doc for a product manager — and sit down with the people whose judgment you trust. Ask them a really simple question: when you say that a decision doc is good, can you tell me what you mean specifically? Ask gently, ask clearly, ask persistently, and push on the people in your life that you trust until you have a small, concrete list. Maybe it's: is the decision stated in a sentence? Are there at least two real options? Are the stakes and metrics explicit? Is there a clear recommendation? Are risks and trade-offs surfaced?
That's an example for one artifact. You need to look at it for all of your artifacts, the ones relevant to your discipline — whether that's architecture docs for engineering, call summaries for customer success managers, pipeline expectations for sales. There are all kinds of ways to do this. But the key is asking someone in your life what's good.
Then you turn that into a rubric — a grade. You make it clear what good looks like, and you set it out one to five. Then you pull three to five real examples and mark them up. Get a red pen out and say: this one is really good at clarity, this one is good at risks but has these weaknesses, this is the rationale.
Notice how none of this is with the AI yet. I promise we'll get there. But I want you to recognize that human skills are human skills, and I'm asking you to take some human responsibility for developing your skills.
Only then — after you've red-penned a few things — do you bring it to an LLM. You give it the rubric and your annotated examples. We are at a point where you could actually use a red pen, scribble all over the doc, and it would still work because handwriting recognition is good enough now to pick it up. And then you say in effect: when I send you a new doc, please score it like this, quote the parts you're reacting to, explain briefly why you gave each a score, and suggest edits that would move one of these dimensions up by a point or two.
Suddenly, look what that changes. Instead of a manager skimming through and thinking "that feels fuzzy, I have 15 minutes, I'm going to turn it over" — no. You get a structured critique that can be applied to every single doc of that type. This one has a two on options. That one is a four on clarity but a one on how risk is structured. This is what I need to do to change it. You get a rough, consistent view of how the skill is showing up across your real work.
We've been missing that. That is our signal. That is the basketball going into the basket. And you can actually log this — you can say, over a quarter, what are the patterns I'm starting to see in my own behavior? How are my scores changing? You can get actual scores out of five.
With that, we now have something the pre-AI world just couldn't have. When Tyler wrote this, it wasn't possible. We can do effectively film review — like athletes — on our thinking and our writing at scale, without having to hire an army of coaches. Just with some good prompts.
Turning film review into repeatable drills
The next move, once you've got that, is to turn the film review into repeatable drills that train on the patterns you care about.
Take judgment as an example. In artifact form, judgment often looks like: can I write a decision document that lets a reasonable person say yes or no without a two-hour meeting? With your rubric in place, you can create a practice exercise that looks like this. Once a week, take a real messy situation — a Slack thread, a super vague request from your manager, a fuzzy idea you had in the shower. Write a one-page decision doc that hits the pattern you've identified as good: clear decision, options presented, stakes, recommendation, and so on. Now run it through the same AI rubric you use on real docs. Compare your version to a stronger version that the model generates. Notice what you miss. That is your practice.
You can do this for orchestration, where you define what a good spec looks like in your environment — explicit goal, inputs, outputs, constraints — and create drills where people practice turning fuzzy objectives into time-bound specs. For coordination, you can define a pattern for your executive updates.
The important thing is to see the chain of behavior you need to adopt to level up. You have a skill. You identify your recurring behavior. You figure out how that maps to a recognizable pattern in the artifacts you leave behind. Then you establish a grade and start to practice. That's what it takes to go from "Tyler wrote a good thought, I'm not really changing my behavior" to "I have AI, I have a personal coach, I just need to configure it right" — and now you start to get better.
Applying this at the team level
What does this look like if you're a team lead? Very few team leads actually do this, but let's play out the operations. Suppose you run a team at a mid-size company. You decide that for the next quarter you're going to focus on a particular artifact you want to level up. You and your team define a rubric together — it's not just an individual exercise. You pull example docs that are good together. You can then wire up a team LLM so that whenever someone marks a doc as ready for review, it runs the rubric pass. This is like engineers who have code review automatically run on their pull requests — now you're having Claude or ChatGPT automatically review your docs. Same thing: it leaves comments.
You can ask any of your teammates to do two things. First, let the AI critique hit the doc before a human review — that's a management decision. And then once or twice a week, as a team, set a 10-minute timer and practice on something that the AI keeps flagging as a growth area for you individually, and report on it. Talk about it. Humans do better with goals when we articulate them. The team gets stronger and individuals progress faster because we're in a team environment.
The goal is not to demand perfection. The goal isn't even to tie this to performance ratings. It's to ask that we use small, steady habits to actually build and scale useful skills that we will need in the age of AI.
By the end of your quarter, you should be able to have a conversation with your team where you say: have we improved on our rubric for this artifact? Did the scores get higher? Are docs getting approved with fewer iterations? Are key decisions happening faster and with less "what are we deciding?" confusion? If these are moving in the right direction, what you're learning is that a practice loop changes how your team thinks and writes. That's the core. That's what you're betting on.
Using the same framework in hiring
What's interesting is that you can use the same skill set in interviews and hiring. Most companies are hiring for skills in a way that is comically indirect. We might ask, "Tell me about a time you influenced a stakeholder." We listen to the story, squint, and try to infer whether they can do the work we need done in the next couple of quarters.
If you've already done the work to define a pattern for a particular artifact, there's a much more grounded way to evaluate people. Give them the same game you play as a team and see how they do.
So instead of a traditional PM interview, maybe the candidate gets a short take-home where they write or repair a decision document based on a realistic prompt. Then there's a live session where you work through that doc and change a constraint — legal is going to block this, or the timeline shrinks — and you see how they think through it and adjust. Then there's a critique exercise where you show them a deliberately mediocre AI-generated doc and ask what's wrong with it.
The beauty of this is that you can use the same rubric you developed internally, and even the same AI model, as a first-pass scorer for consistency. The point is not to let AI decide who to hire. It's to have a shared, concrete lens on what good looks like on the work you're actually doing. And the nice side effect is that hiring and development now point at the same thing. The skills you test for in candidates are the skills you help them practice once they're inside the door. It's not: we hired them for their strategic thinking and they're bad at Jira tickets. These are the skills we tested for, and these are the skills we work on as a team.
AI use is not the concern — the underlying skill is
I want to call something out here. None of this presumes that you cannot use AI to get better. You are going to be using AI. Anthropic has noted that something like two-thirds of AI usage is shadow AI usage — people not reporting it. People aren't incentivized to report it right now. This doesn't mean you have to hide your AI use. You can be open about using AI and still get better at these skills, because the goal is the outcome.
If your interviewee is using AI, you're going to find out real quick whether they have a healthy relationship with AI when they turn something in and then you give them a live constraint and they fumble and can't handle it. You're going to see where the edges of those skill sets are.
The practice loops I'm describing are designed to reinforce the kinds of skills we humans need in the age of AI. They're going to push people to clarify decisions, surface risk, articulate trade-offs. If someone uses AI for a pass at that, great — but you're going to catch them if they haven't done the heavy thinking. What's freeing about this is that you're enabling a real evaluation through live conversation where people talk through their choices and how they respond. The interview and the development conversations feel very similar. It's not about trying to catch people cheating with AI. All you're trying to do is see if they have a stable pattern of thought that remains visible even when their ability to just tab-tab-tab through a screen is gone.
If they're having a conversation and you change some dynamics and they just stumble because they're not in front of a screen, you're going to know. Whereas if they can articulate the trade-offs and start to point those skills in the right direction — and yeah, maybe AI helped them get there faster — that's fantastic. Now you can measure it.
Limits and how to start
I don't want to over-romanticize this. There are going to be real limits. Rubric scores will be noisy. I would not treat them as precise numerical representations. I would not treat them as a basis for promotions. I don't want people to feel like there's a surveillance risk where every single document is scored. The goal is to get better. The goal is to become useful. And I don't want program fatigue to eat this.
So instead of trying to start really big, I would strongly recommend that you start small. Pick one little thing, a short change in habit, start to practice, and just start to feel into it. Because really the goal is to get in the habit of being athletes about our knowledge work. How do we intentionally name a skill, measure a skill, see what good looks like, and use the power of AI to train and get better?
If we go after that — if we have that sort of focus and goal, whether as an individual or a team manager — we are going to be in good shape. We are going to be in a position where we can actually answer Tyler's question. Part of why Tyler Cowen wrote what he did back in 2019 is that we didn't have AI. AI couldn't be there to coach us. It was too expensive for most people to get coached. Well, not anymore. Now we have AI. AI can help each of us individually and help our teams to actually grow in our skill sets. And that's really exciting.