The age of unlimited building

Technology leaders have spent years dreaming about faster product development.

Backlogs would shrink. Product roadmaps would accelerate. Teams would spend less time wrestling with implementation and more time delivering value.

AI appears to offer exactly that future.

Yet there is a catch.

"The friction used to be waiting for an engineer to build something," Katie explained. "But now that everyone is a builder, organizations think, 'Oh, we can just build more faster.'"

That shift sounds empowering on the surface. In practice, it can be surprisingly destabilizing.

Scarcity has always served as a hidden quality control mechanism. When engineering resources were limited, organizations had to ask difficult questions. Which customer problem matters most? Which feature deserves investment? Which opportunity aligns with strategic goals?

When the cost of building approaches zero, those questions don't disappear. They simply become easier to ignore.

Many organizations are discovering that AI doesn't automatically create better products. It creates more products. More experiments. More features. More workflows. More opportunities to distract customers with things they never asked for.

The result resembles a city that suddenly eliminates all zoning laws. Construction becomes easier. Buildings appear everywhere. But growth without planning quickly becomes chaos.

Katie has observed this pattern repeatedly while working with organizations experimenting with AI adoption.

"They're not actually checking to see what their customers want," she said. "And they're not checking in with customers to make sure that those decisions that they're making are for the customer."

For leaders focused on customer experience and product management, that observation should feel uncomfortable. It suggests that AI product development is not reducing the need for customer understanding. It is increasing it.

Why AI agents are changing the rules

Much of the current excitement around AI revolves around AI agents.

Unlike traditional large language models, which primarily respond to prompts, AI agents can act autonomously, maintain context, perform tasks, and make decisions over time.

That capability changes the product development lifecycle in profound ways.

Tasks once owned by humans can increasingly be delegated to software. Research synthesis. Competitive analysis. Documentation. Requirement creation. Workflow automation. Customer support routing.

The promise is undeniable.

The responsibility is less frequently discussed.

Katie offered a comparison that captures the challenge perfectly.

"You wouldn't hire a junior developer to come into your organization and hand them all the keys to your repositories and say, 'Okay, go nuts,'" she said.

Most companies would never do that with a human employee. Yet many are doing precisely that with AI.

They deploy agents before defining governance. They automate workflows before establishing accountability. They grant access before creating guardrails.

The assumption seems to be that because AI is software, it should behave predictably.

But AI agents are not deterministic systems in the traditional sense. They require training. Evaluation. Oversight. Context.

Like a new employee, they need management.

This reality is creating an entirely new category of work within product teams. Someone must decide how agents should behave. Someone must determine whether their outputs are useful. Someone must intervene when they make mistakes.

Those responsibilities don't disappear simply because the work is automated.

They shift.

Customer discovery becomes more important, not less

One of the most interesting arguments Katie made during the conversation runs directly against much of the current AI narrative.

Conventional wisdom suggests that AI reduces the importance of discovery because teams can iterate so quickly.

Katie believes the opposite is true.

"Discovery is much more important than the actual building phase," she said.

That statement may sound counterintuitive until you consider how AI systems learn.

Traditional software follows instructions.

AI systems absorb context.

If the context is flawed, the outputs will be flawed. If the assumptions are wrong, the errors can multiply rapidly.

Discovery becomes the foundation upon which everything else rests.

Imagine constructing a skyscraper. Advances in construction technology might allow workers to build floors faster than ever before. But if the foundation is unstable, accelerating construction only increases the magnitude of the eventual failure.

AI product development works much the same way.

Organizations that skip customer discovery aren't simply risking a misguided feature release. They risk training systems on incomplete assumptions about customer needs, behaviors, and expectations.

This is why customer insight is becoming a strategic asset.

The organizations most likely to succeed with AI won't necessarily have the best models or the fastest engineers. They'll have the deepest understanding of the people they serve.

That understanding comes from listening.

From research.

From observation.

From customer feedback.

In other words, from the very disciplines some executives assumed AI would make less important.

The rise of human-in-the-loop evaluation

For all the discussion about automation, one phrase appeared repeatedly throughout the conversation: human-in-the-loop.

The concept is simple.

Humans remain involved in evaluating AI outputs.

But the implications are significant.

Many organizations still view AI-generated content as answers. Katie views it as a starting point.

Research findings should be reviewed.

Recommendations should be challenged.

Insights should be validated.

"The human needs to direct the agent," she explained. "The human needs to direct the AI into doing what the human wants it to do, and not the other way around."

This distinction may become one of the defining competitive advantages of the next decade.

The organizations that blindly accept AI outputs will move quickly.

The organizations that thoughtfully evaluate AI outputs will move intelligently.

Those are not the same thing.

Human-in-the-loop evaluation also creates an opportunity to improve AI systems over time. By providing feedback, correcting mistakes, and defining what success looks like, teams gradually teach agents to perform more effectively.

The process resembles coaching more than programming.

And that requires a very different mindset from traditional software development.

A looming challenge for product leadership

Perhaps the most thought-provoking portion of the conversation focused not on technology but on talent.

Historically, product leaders developed strategic judgment through experience.

They conducted interviews.

They analyzed research.

They reviewed support tickets.

They spent years immersed in customer problems.

That friction created expertise.

AI threatens to remove much of that friction.

A junior product manager can now generate summaries instead of reading dozens of customer interviews. They can review synthesized insights rather than performing analysis themselves.

The efficiency is undeniable.

But what happens to learning?

Katie posed a question that many organizations have yet to answer.

How do future leaders develop strategic thinking if AI performs much of the tactical work that once taught those lessons?

The concern extends beyond product management.

Researchers. Designers. Marketers. Analysts.

Any profession that develops expertise through pattern recognition faces a similar challenge.

Knowledge has always required effort.

If AI reduces the effort, organizations must become more intentional about developing judgment.

Katie's recommendation was not to reject AI. It was to remain actively engaged with it.

Have people validate outputs.

Compare human conclusions with machine conclusions.

Use AI as a collaborator rather than a substitute.

The goal is not efficiency at all costs.

The goal is capability.

Why everyone should build an AI agent

Near the end of the discussion, Mike asked Katie how non-technical professionals should begin engaging with this rapidly changing landscape.

Her answer was refreshingly direct.

Build something.

Not because everyone needs to become an engineer.

Not because every employee should launch a startup.

But because firsthand experience changes understanding.

"I really think that everyone in an organization, from the CEO to a junior HR person, should be building something," Katie said.

Building an AI agent forces people to confront realities that theoretical discussions often obscure.

You learn how much context matters.

You discover how easily systems can misinterpret instructions.

You experience firsthand why governance and evaluation are necessary.

Most importantly, you stop thinking about AI as magic.

It becomes a tool.

A powerful one, certainly. But still a tool.

And like any tool, its value depends on the clarity of the person using it.

The future belongs to better decision-makers

For years, product development has been constrained by execution.

Today, execution is becoming abundant.

The bottleneck is moving elsewhere.

The organizations that thrive in the age of AI agents will not be those that build the most features, generate the most code, or deploy the most automation.

They will be the ones that make the best decisions.

They will understand their customers more deeply.

They will invest in customer discovery.

They will maintain human oversight.

And they will recognize that AI's greatest value lies not in replacing human judgment, but in amplifying it.

Technology has always rewarded speed. The next chapter may reward discernment even more.

As Katie put it, "Just because you can build something doesn't mean you should build something."

Episode links

• Katie Robblee on LinkedIn
• Mike Mace on LinkedIn
• Nathan Isaacs on LinkedIn
• The future of insight: How information workers leverage AI + human understanding to drive smarter decisions: This on-demand webinar explores how leading teams combine AI capabilities with human insight to make better decisions—a direct parallel to Katie Robblee's emphasis on human-in-the-loop evaluation, AI governance, and customer-centered decision-making.
• Human insight for the AI-driven product development process: This guide is perhaps the closest companion piece to the episode. It examines how AI is transforming the product development lifecycle, why customer discovery must become continuous, how AI changes product team roles, and why human insight is becoming more critical—not less—as development accelerates.
• Natalie Nixon on creativity, intuition, and AI as a co-creator: While focused on creativity rather than product operations, this episode explores a similar theme: AI is most powerful when paired with human judgment, curiosity, and strategic thinking. Both conversations challenge the idea that AI should replace human expertise and instead position it as a tool that amplifies it.
• The responsible path to AI-accelerated customer insights: This blog post aligns closely with Katie's perspective on AI adoption. It argues that AI should accelerate insight generation while humans remain accountable for interpretation, decision-making, and outcomes. The piece also discusses agentic workflows, transparency, and the risks of accepting AI-generated outputs without inspection.
• Best practices for great AI experiences: This resource page supports one of the episode's core arguments: organizations should focus less on building AI features quickly and more on identifying the right customer problems to solve. It covers validating AI opportunities, understanding customer reactions to AI, and evaluating trust and usability before investing in development.