The Art of Asking Good Questions

experiments, shipping insights, and automating reports. Your stakeholders praise your work. Your dashboards are beautiful. Your analyses are rigorous. But if the roadmap doesn’t change based on your work, are you really driving impact?

The answer often comes down to one critical skill that separates strategic data scientists from tactical ones: the ability to not just to ask better questions, but to frame them.

Good questions generate impact. And impact—not just technical excellence—is your best defense against AI replacing your role.

The Three Levels of Data Science Impact

Before I talk about what makes a great question, I want to describe three levels of performance for a data scientist. With these levels defined, we’ll then be able to see why better questioning is the key differentiator across levels.

As most data scientists advance in their careers, they progress up the levels when it comes to product influence. The lower level is where AI could be a threat, but the upper level sees it as an opportunity.

Level 1: Reactive Executor You answer questions after decisions have been made. Most of your work comes from Jira tickets or Slack requests. You focus on delivering clean analyses quickly, but you rarely know the broader strategic context for your work.

Level 2: Informed Collaborator You participate in product planning meetings and run experiments, asking clarifying questions to better understand immediate problems. While still primarily reactive, you inform choices rather than frame them.

Level 3: Strategic Partner You shape what gets prioritized. You identify problems worth solving before anyone thinks to ask. You surface strategic questions. Your work directly influences what gets built, shipped, or funded.

Moving up to Level 3 behavior, regardless of your actual seniority in your org, is what will insulate your career from potential AI impacts (and also set you up for a strong promotion!).

The takeaway here is that at Level 1, you’re answering questions others have already framed. At Level 2, you’re clarifying questions to solve the immediate problem. But at Level 3, you’re asking the strategic questions that reshape how entire teams think about problems.

As you advance up the levels, you shift from answering questions to asking better ones. But what makes a question ‘better’? What transforms a routine data request into strategic insight? The answer lies in five specific attributes that define high-leverage questions.

The Anatomy of Questions That Drive Decisions

Not all questions are created equal; some generate busy work, others surface insights, but the best questions generate decisions. I’ve identified five attributes that separate high-leverage questions from low-leverage ones, that delineate the strategic questions from the busy work:

Decision-Linked: Great questions have clear lines of sight to action. When you answer them, someone knows what to do next.

• Instead of: “What’s our user engagement rate by cohort?”
• Ask: “Which cohorts should we target for our retention campaign, and how much budget should we allocate?”

Ambiguity-Reducing: They clarify fuzzy or risky product bets. They take situations where smart people disagree and provide a framework for moving forward.

• Instead of: “How many users tried the new feature?”
• Ask: “Is the new feature cannibalizing our core usage, or is it expanding overall engagement?”

Directional: They don’t just describe, they guide. They point toward specific actions and reveal important tradeoffs.

• Instead of: “What’s the correlation between feature usage and retention?”
• Ask: “If we focus on this feature, what’s the maximum retention improvement we could expect?”

Scalable: They create frameworks for solving similar problems in the future or across teams, building organizational capability rather than just individual insights.

• Instead of: “What happened to conversion rates last week?”
• Ask: “What early indicators should we monitor to predict conversion rate changes before they impact revenue?”

Non-Obvious: They surface tradeoffs that weren’t being considered and challenge existing assumptions.

• Instead of: “Are users engaging with our recommendations?”
• Ask: “Are our recommendations making users more autonomous or more dependent on our platform?”

The Question Ladder: Your Path to Strategic Impact

Now that you know what great questions accomplish, how do you apply that to the request you just received from your PM? I’ve developed a sequence of questions I like to ask—either just to myself, or to my PMs or engineering teams—that helps me take a tactical request and ensure it has strategic results. I call this the Question Ladder. It’s a practical tool that transforms how you engage with stakeholders.

Instead of diving immediately into analysis, you work through each rung of the ladder to ensure you’re solving the right problem:

1: Observation: Something changed, or someone’s curious

2: Clarification: What exactly are we trying to learn?

3: Relevance: How does this tie to product or business goals?

4: Decision: What decision or action would this enable?

5: Prioritization: Is this worth prioritizing over other work?

As you progress through the Question Ladder, keep the Five Attributes from the previous section in mind. After each question you ask, use the five attributes of good questions to determine what should be asked next.

Here’s how this works in practice. A product manager approaches you with what seems like a simple request: “Can you analyze our feature adoption rates?”

Instead of immediately writing queries, you climb the ladder:

Observation: “What specific change or pattern prompted this request?”

Clarification: “Are we trying to understand which features are most popular, which users are most engaged, or which features drive retention?”

Relevance: “How does feature adoption connect to our current product strategy? Are we evaluating existing features or deciding what to build next?”

Decision: “What decisions about product development are waiting for this analysis? What would we do if adoption rates were much higher or lower than expected?”

Prioritization: “What other projects would we pause to act on insights from this analysis? What level of confidence would we need to change our development priorities?”

This process usually takes 10-15 minutes of conversation, but it transforms a generic reporting request into a strategic analysis with clear implications for action.

From Executor to Strategic Partner

The Question Ladder also helps you identify when not to do analysis. If you can’t find clear answers at levels 4 and 5, that’s a strong signal that the analysis isn’t worth doing yet. Better to spend time clarifying the strategic context than to produce insights that won’t be used.

I’ve seen data scientists go from the PM’s request directly to writing code, then wonder why their beautiful analysis gets ignored. The ladder forces you to do the strategic work upfront, when it’s most valuable and least expensive.

This is how you move from being a “ticket-taker” to a strategic partner. An important differentiator from a Level 1 data scientist and a Level 3 one is that at Level 1 you’re a strong contributor, but at Level 3 you’re up-leveling the entire team. Using the ladder won’t just improve your questions, it will improve your stakeholders’ questions too.

The frameworks in this post—the three levels, the five question attributes, and the Question Ladder—aren’t just tools for better analysis. They’re a critical skill for transforming how you show up as a data scientist. Master them, and you’ll find that the right questions don’t just improve your work—they elevate everyone around you.

Did this post spark your curiosity about becoming a more strategic data scientist? I expand on these frameworks—and many others—in my new book The Strategic Data Scientist: How to Level Up and Thrive in the Age of AI (Affiliate link). I wrote it because throughout my career I’ve seen too many data scientists struggle to connect their technical work to the business problems that actually matter. If there’s one takeaway, it’s this: the best defense against AI isn’t more code, it’s a different way to think.

If you’d like to go deeper, you can pre-order the ebook now on Amazon—and start putting these frameworks into practice. Paperback and hardcover versions will be available on September 30th.