Tag Archives: Strategy

Self-Improving Agents Are Still ANI: Why Hyperagents Don’t Change the Classification

A Kantian Lens on Machine Intelligence

In March 2026, Meta published a paper on hyperagents: systems that rewrite the mechanism by which they improve themselves. Performance compounds across runs. Meta-level gains transfer across domains. The system improves its own improvement process.

If that doesn’t sound like AGI, I don’t know what does.

Except it isn’t. And a philosopher who died in 1804 already explained why.

What Are Hyperagents?

To understand why hyperagents matter and why they do not change the classification of intelligence, we need to trace a brief lineage.

Darwin Gödel Machine (DGM)

Published in May 2025 by Zhang et al., the Darwin Gödel Machine is a coding agent that iteratively modifies its own source code and empirically validates changes against benchmarks. It maintains an archive of agent variants through Darwinian selection, where successful modifications survive and unsuccessful ones are pruned. On SWE-bench, it improved from 20.0% to 50.0% through self-modification.

The key structural insight: because the task domain (coding) and the self-modification mechanism (also coding) share the same medium, improvements in one naturally feed the other. Better coding ability produces better self-modification, which produces better coding ability. This virtuous cycle is real and measurable.

DGM-Hyperagents (DGM-H)

Published in March 2026, hyperagents address a specific limitation of DGM. In the original system, the meta-mechanism (the process for generating improvements) was hand-crafted and fixed by human designers. DGM-H makes the meta-mechanism itself editable. The system now merges a task agent and a meta agent into a single self-modifiable program.

The authors call this “metacognitive self-modification.” Meta-level improvements, such as persistent memory, performance tracking, and improved code editing strategies, transfer across fundamentally different domains, including coding, academic paper review, and robotics reward function design. These improvements accumulate across runs.

At a Glance

Attribute	DGM (2025)	DGM-H / Hyperagent (2026)
Self-modifies code	Yes	Yes
Meta-mechanism editable	No (hand-crafted)	Yes (self-referential)
Cross-domain transfer	No (coding only)	Yes (coding, review, robotics)
Accumulates across runs	Limited	Yes
Foundation model modified	No (frozen)	No (frozen)

That last row is the entire argument. Hold it for now. We will return to it.

Why This Looks Like AGI: The Strongest Case

Intellectual honesty requires acknowledging the strength of the counterargument before engaging with it.

In my earlier post on ANI classification, I established four criteria for AGI:

Learning new domains from raw data without explicit programming
Transferring reasoning across unrelated disciplines
Generating and refining internal context models autonomously
Forming and pursuing long-term goals without human direction

On the surface, Hyperagents appear to make progress on criteria one and two. The system does improve across domains. The meta-level improvements do accumulate without being explicitly programmed. For the first time, there is a credible research artifact that seems to blur the line between narrow optimization and general capability.

This is the strongest counterargument to the ANI thesis that has appeared in the literature. It deserves a rigorous response.

Enter Kant: The Critique of Pure Reason as an Evaluation Framework

Why Kant?

Immanuel Kant’s Critique of Pure Reason (1781) asked a question that maps directly to the AGI debate: What are the conditions that make knowledge possible?

Kant was not asking what we know. He was asking what must be true about a knowing entity for knowledge to exist at all. That question, about the preconditions for intelligence rather than its outputs, is exactly the question we should be asking about self-improving agents.

Most commentary on hyperagents focuses on what the system does. Kant’s framework forces us to ask what the system is. And that distinction is the one that matters.

The Four Kantian Tests for Intelligence

Test 1: Analytic versus Synthetic Judgment

Kant distinguished between two types of knowledge.

Analytic judgments are those where the predicate is already contained in the subject. They decompose what is already known. “All bodies are extended” is analytic because extension is part of the definition of “body.” These judgments clarify, but they do not extend knowledge.

Synthetic judgments add something new. “All bodies are heavy” is synthetic because weight is not contained in the concept of body itself. You need to go beyond the concept to learn something the concept alone does not contain.

Kant’s central question concerned synthetic a priori knowledge: knowledge that is both genuinely new (synthetic) and necessarily true independent of any particular experience (a priori). He argued that mathematics, the foundations of natural science, and the preconditions for experience itself all belong to this category.

Application to Hyperagents: DGM-H operates almost entirely within the domain of analytic judgment. When the system rewrites its code to add persistent memory or improve its editing tools, it is decomposing and recombining patterns that already exist within its training distribution. The foundation model’s representations, the “concepts” it possesses, remain fixed. The scaffolding improvements are analytic rearrangements of known capabilities.

A system capable of synthetic judgment would generate knowledge that is not contained in its existing representations. It would discover that “bodies are heavy” without having the concept of weight anywhere in its training data. DGM-H does not do this. It recombines existing knowledge more efficiently. This is sophisticated analysis, not synthesis.

Smell test: “Does the system discover knowledge that was not already latent in its foundation model? Or does it rearrange what the model already knows?”

Test 2: Phenomena versus Noumena, the Boundary of Knowable Reality

Kant argued that human knowledge is limited to phenomena: the world as it appears to us through the structures of our perception (space, time, and the categories of understanding). Behind appearances lies the noumenon, the thing-in-itself, which remains fundamentally unknowable.

This is not a limitation to be overcome. It is a structural boundary of cognition itself.

Application to Hyperagents: The foundation model is the noumenal boundary of the Hyperagent system. The system interacts with token representations, benchmark scores, and code outputs. All of these are phenomena. It can modify how it processes these appearances through better tools, better prompts, and better memory. But it cannot reach through to modify the foundation model itself, the cognitive substrate that determines what can appear in the first place.

DGM-H optimizes within the phenomenal world of its own outputs. It has no access to its own noumenon. The weights are frozen. The representations are fixed. The boundary cannot be crossed through scaffolding improvements, no matter how recursive those improvements become.

When Kant said that concepts without intuitions are empty, he meant that pure logical manipulation without grounded experience cannot produce real knowledge. A Hyperagent that rewrites its own orchestration code without modifying its representational substrate is manipulating concepts without changing the intuitions that ground them.

Smell test: “Is the system modifying how it perceives, or only how it organizes what it already perceives?”

Test 3: The Transcendental Unity of Apperception, the Missing “I Think”

This is perhaps Kant’s most profound contribution to the question of intelligence. Kant argued that all experience requires a transcendental unity of apperception: the “I think” that must be capable of accompanying all representations. Without a unified self that integrates perceptions into a coherent whole, there is no experience, no knowledge, and no cognition.

This is not consciousness in the mystical sense. It is a structural requirement: for knowledge to cohere, there must be a unifying perspective that holds representations together across time.

Application to Hyperagents: DGM-H maintains an archive of agent variants. Each variant is evaluated independently. There is no unified “self” that integrates the experience of all variants into coherent understanding. The system is a population of narrow programs being selected by benchmark fitness, not a single entity that learns from its accumulated experience in a unified way.

When a Hyperagent “transfers” a meta-level improvement from the coding domain to the robotics domain, it is not a single intelligence applying cross-domain reasoning. It is a code pattern being reused in a different context. There is no “I think” that accompanies the transfer. There is no unified apperception binding the coding experience to the robotics experience into a single coherent worldview.

Darwinian selection and coherent self-awareness are fundamentally different mechanisms. Evolution produces fit organisms. It does not produce a single organism that understands why it is fit.

Smell test: “Is there a unified perspective that integrates learning across domains, or is there a population of narrow variants being selected by an external fitness function?”

Test 4: The Limits of Pure Reason, Why Recursive Optimization Has a Ceiling

Kant’s Critique was, at its core, an argument about limits. Pure reason, meaning reasoning without grounded experience, cannot extend knowledge beyond the bounds of possible experience. When reason attempts to do so (Kant called these attempts “transcendental illusions”), it generates contradictions and paradoxes, not knowledge.

Application to Hyperagents: The evaluation function is the bound of possible experience for DGM-H. The system can only know what the benchmark measures. It cannot reason about value, purpose, or knowledge that exists outside the evaluation function’s scope.

Recursive self-improvement within a fixed evaluation function is analogous to Kant’s critique of dogmatic metaphysics: reason operating on itself, in an unbounded way, without the grounding constraints that make knowledge possible. The system can optimize endlessly, but it cannot transcend the boundary defined by its evaluation criteria.

This maps directly to my argument in the dynamic evaluations posts: evaluation loops measure performance and enforce constraints, but they do not create new knowledge or abstraction. DGM-H elevates this from a single evaluation step to an evolutionary cycle, but the fundamental Kantian constraint holds. Optimization within a bounded evaluation function, no matter how recursive, cannot produce unbounded intelligence.

Smell test: “Can the system define what ‘better’ means, or does it only optimize for a definition of ‘better’ that was given to it?”

The Technical Dissection: Reinforcing the Philosophy with Architecture

The Kantian analysis is not merely philosophical. It maps directly to concrete architectural facts about how DGM-H works.

1. The Foundation Model Is Frozen

DGM-H modifies scaffolding: tools, prompts, workflows, and memory management. The foundation model weights never change. This is the architectural expression of the phenomena/noumena boundary. The system rearranges appearances without touching the cognitive substrate.

2. Goals Are Human-Defined and Externally Imposed

Every DGM-H run starts with a human-selected benchmark. The system does not choose what to improve at. It does not formulate its own research questions. Kant’s “I think” would require autonomous goal formation, where the system decides for itself what matters. DGM-H has no such capacity.

3. Cross-Domain Transfer Is Scaffolding Reuse, Not Reasoning Transfer

The meta-level improvements that transfer across domains (persistent memory, performance tracking) are infrastructure patterns. A human who learns music theory and applies harmonic reasoning to wave physics is performing synthetic judgment, connecting concepts that were not previously connected. An agent that reuses a memory management pattern across domains is performing analytic reapplication.

4. Self-Modification Operates on Code, Not on Representation

DGM-H modifies Python source code. It does not modify attention patterns, learned features, or representational structures. In Kantian terms, it modifies the organization of experience, not the categories through which experience is structured.

Think of it this way: a chess player who develops a new opening strategy has modified their cognitive approach. A chess player who buys a better chessboard and clock has modified their tooling. DGM-H is doing the latter at an impressive scale.

AGI Anti-Patterns for Leaders

These anti-patterns matter for anyone evaluating AI capabilities in a procurement, investment, or strategic planning context.

“Self-Improving” in the Pitch Deck Equals AGI

Kant taught us that the impressive outputs of reason can be illusory when reason operates beyond its legitimate bounds. The same applies to vendor claims. The evaluation rubric in this post gives leaders a structured way to push back. Ask the four Kantian questions. If the vendor cannot answer them, the claim is marketing, not capability.

Confusing Compounding Optimization with Compounding Intelligence

DGM-H demonstrates compounding optimization, where each run builds on the last. Kant would call this increasingly sophisticated analytic judgment. It is not synthetic intelligence, which would require generating genuinely new knowledge that extends beyond existing representations.

Ignoring the Frozen Foundation Model Constraint

If the foundation model is frozen, then the ceiling of the system’s capability is fixed. No amount of scaffolding optimization changes this. Leaders should ask: “When you say self-improving, what exactly is improving: the model or the wiring around the model?”

Over-Delegating Autonomy Based on Self-Improvement Claims

In my Kano Model post, I established that removing human checkpoints too early is a dangerous anti-pattern. Self-improving systems amplify this risk by creating the illusion of autonomous competence while operating within narrow, benchmark-defined boundaries.

Kant warned that reason unchecked by grounded experience produces illusions, not knowledge. The same is true for agents unchecked by human oversight.

The Adaptation Taxonomy: Broader Context

Jiang et al.’s survey paper, “Adaptation of Agentic AI” (December 2025), organizes the adaptation landscape into four paradigms: tool-execution-signaled agent adaptation, agent-output-signaled agent adaptation, agent-agnostic tool adaptation, and agent-supervised tool adaptation.

DGM-H fits squarely into agent-output-signaled adaptation: the agent modifies itself based on its own performance outputs. In Kantian terms, this is reason responding to its own products, not to raw experience.

The taxonomy makes clear that what Hyperagents do is a specific, well-characterized form of narrow adaptation. It is sophisticated. It is useful. It is not general.

The Bottom Line

Let’s be clear:

DGM is not AGI.
DGM-H (hyperagents) is not AGI.
Self-modification of scaffolding around a frozen foundation model is not cognitive evolution.

These systems perform increasingly sophisticated analytic operations. They do not perform synthetic judgment. They optimize within a phenomenal boundary defined by their evaluation functions. They lack the transcendental unity of apperception, the integrated “I think,” that Kant identified as the precondition for genuine knowledge.

Kant wrote the Critique of Pure Reason to establish the boundaries of what reason can legitimately claim to know. Two and a half centuries later, those boundaries still hold, not just for human cognition but also for artificial systems that attempt to simulate it.

The day a system generates its own evaluation criteria, formulates its own problems, produces synthetic a priori knowledge, and integrates experience through a unified perspective is the day we revisit this classification.

Until then, the framework holds.

Kantian Evaluation Rubric: “Is It AGI?”

Kantian Test	What It Asks	Hyperagents	True AGI Requirement
Analytic vs. Synthetic	Does the system create genuinely new knowledge?	No: rearranges existing representations	Must produce synthetic judgment beyond training
Phenomena vs. Noumena	Can it modify its own cognitive substrate?	No, the foundation model is frozen	Must modify how it perceives, not just what it organizes
Transcendental Unity of Apperception	Is there a unified “I” integrating experiences?	No: population of variants selected by fitness	Must possess a coherent, self-integrating perspective
Limits of Pure Reason	Can it define its own criteria for “better”?	No: optimizes for human-defined benchmarks	Must autonomously generate evaluation criteria

References

Zhang, J. et al. “Hyperagents.” arXiv:2603.19461 (March 2026). https://arxiv.org/abs/2603.19461
Zhang, J. et al. “Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents.” arXiv:2505.22954 (May 2025, updated March 2026). https://arxiv.org/abs/2505.22954
Jiang, P. et al. “Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills.” arXiv:2512.16301 (December 2025). https://arxiv.org/abs/2512.16301
Kant, I. Critique of Pure Reason. Trans. Norman Kemp Smith. (1781/1787). Macmillan, 1929.
Stanford Encyclopedia of Philosophy. “Kant’s Theory of Judgment.” https://plato.stanford.edu/entries/kant-judgment/
Stanford Encyclopedia of Philosophy. “Kant’s Critique of Metaphysics.” https://plato.stanford.edu/entries/kant-metaphysics/
Kanakasabesan, K. “AGI isn’t here yet: Why OpenClaw, Agents, and LLM Systems are still just ANI.” https://kanakasabesan.com/2026/03/09/agi-isnt-here-yet-why-openclaw-agents-and-llm-systems-are-still-just-ani/
Kanakasabesan, K. “Kano Model and the AI Agentic Layers.” https://kanakasabesan.com/2026/01/11/kano-model-and-the-ai-agentic-layers/
Kanakasabesan, K. “Your agents are not safe, and your evals are too easy.” https://kanakasabesan.com/2025/11/21/your-agents-are-not-safe-and-your-evals-are-too-easy/
Kanakasabesan, K. “Measuring What Matters: Dynamic Evaluation for Autonomous Security Agents.” https://kanakasabesan.com/2025/12/15/measuring-what-matters-dynamic-evaluation-for-autonomous-security-agents/

Kano Model and the AI Agentic Layers

Leave a reply

Happy 2026, everyone! I trust you all enjoyed a refreshing break and are entering this year with renewed vigor. The discussion surrounding the value of AI projects and agentic AI remains dynamic. I would like to share my perspective on this topic through two key dimensions:

AI Agentic layers
Kano Model for value

Using these dimensions, we can delve deeper into the complex landscape of how AI creates and, at times, destroys value. By exploring both the positive impacts and the negative repercussions, we can gain a better understanding of this dual nature of technology. This includes a careful examination of various anti-patterns for value destruction, which can inform best practices and help mitigate potential risks associated with AI deployment.

Quick Refresher on the Kano model

Kano Category	What it mean and why it matters
Must-have (Basic)	Expected capability; absence of it causes failure, presence of it does not delight
Performance	Better Execution = more value
Delighters	Unexpected differentiation creates step-function value
Indifferent	no material impacts on outcomes
Reverse	Actively reduces value or trust

7 Layers of Agentic AI

My definition of the 7 layers of Agentic AI are as follows:

Agentic AI Layer	What it means
Experience and Orchestration	Integrates agents into human workflows, decision loops, and customer experiences. This is the business layer. Help accelerate decision-making. Decide when to override agents, e.g., an automated agent taking in returns from customers and deciding which returned merchandise deserves a refund and which does not.
Security and compliance	This is the most important layer, in my opinion. This makes sure that agents do not run wild in your organization. The right level of scope and agency is given to your generative AI agent. Includes policy engines, audit logs, Identity and role-based access, and data residency requirements.
Evals and Observability	The basis of explainable AI. It creates confidence in the outputs that the agent will generate. Agents operate in a non-deterministic way. Your tests must reflect non-deterministic reality to engender trust and reflect the proper upper and lower bounds of such non-determinism. This includes telemetry, scenario based evals, Outcome based metrics, Feedback loops etc.
Infrastructure	This layer makes agents reliable, scalable, observable, and cost controlled. Without this layer, AI pilots cease to be platforms.
Agent Frameworks	Transforms AI into a goal-directed system that can plan, decide, and act. This includes memory, task decomposition, state management, and multi-agent coordination patterns, to name a few.
Data Operations	Key elements of your agentic experience, data quality, freshness, data pipeline scale, etc., are all relevant here. This includes RAG, vector databases, etc.
Foundation models	The operating system of the Agentic experience that we are trying to develop

Mapping the 7 layers to Kano Value

Layer 1: Foundation Model

Primary Kano Category: Must have $->$ Indifferent

Foundation models are now considered a standard expectation; possessing the latest GPT model is no longer a distinguishing factor. However, the absence of such technology can lead to negative consequences from your users.

Hence,

The foundation model presence does not mean differentiation
Absence means immediate failure
Overinvestment in this space yields diminishing returns

Anti-Patterns

The anti-pattern for this value is when the model is the strategy. This fails on so many fronts due to the following:

First, one must identify a model and subsequently determine a problem to address.
- This is analogous to selecting a car model prior to establishing the destination and the nature of the terrain to be navigated.
Treating Foundation model benchmark scores as business value
- If you are driving on the rocks of Moab, Utah, having a 500-horsepower vehicle is not helpful
Hard-wiring a single model to the system
- hitching your business to a single model and not having any leverage
Ignoring latency and cost variability
- For the outcomes you want, do know that the cost variations you are willing to tolerate
Assuming newer is better
- Does the newer model of the vehicle support the terrain on which you want to drive in.

Smell test

“If we change the models tomorrow, does the product still work?”

Layer 2: Data Operations

Primary Kano Category: Performance

Good data means relevant decisions, outcomes, and outputs. The critical elements here are:

Accuracy
Trust
Decision Quality

Users can feel the data is bad, even if they do not know why.

The value in this space is linear with quality improvements and when there is a strong correlation to business outcomes. Like any good system, it is invisible when everything is working well and painful when broken. Poor data becomes a Reverse feature (hallucinations, mistrust)

Anti-Patterns

Dumping entire knowledge bases into embeddings
- This is generally a common thought process that prevails in most organization when adopting AI
No freshness or versioning guarantees
- Something hallucinates; it is usually because of the data.
Ignoring access control in retrieval
- This is common in most cases Agents have unfettered access to data, which is quite problematic for the business overall
Treating RAG as a one-time setup
- This needs to be validated in regular intervals, as the business terrain may change
No measurement of retrieval quality
- “Let us all trust AI blindly” is never a successful strategy

Smell test

“Can we explain why the agent used this data”

Layer 3: Agent Frameworks

Primary Kano Category: Performance $->$ Delighter (Conditional)

Agents that can plan, act, and coordinate unlock:

Automation
Decision delegation
Speed at Scale

These gains can only be realized with the right context windows and when constrained correctly; that is when the actual performance gains are achieved. Remember, agents are logical machines; they are neither credible nor emotional, which does make working with them challenging.

The mantra of starting simple and then focus on scale really does help here.

Anti-Patterns

Starting with multi-agents systems
- If you do not have the basics right and multi-agent systems will compound the problem exponentially
No explicit goals or stopping conditions
- Agents being unbounded means more risk to the business as the probability field is wider
Optimizing for activity, not the outcome
- An agent denied a $5 return to a customer, this activity was done right, but the customer, who had a positive lifetime value over the last five years, churned because of the bad experience

Smell Test

“Can we explain what the agent is trying to achieve in one sentence?”

Layer 4: Deployment & Infrastructure

Primary Kano Category: Must-Have

No user ever says, “I love how scalable your agent infrastructure is.” . But they will leave when the agent fails to scale. This layer is the bedrock of all your agentic experience and has zero visible upside but has several downsides when ignored. This is just like cloud reliability in the early cloud days.

Anti-Patterns

Running agents without isolation
- Agents can consume a lot of resources and become expensive very quickly. This is not just tokens, but also compute, storage, networking, and security; i.e., all of it.
Not having any rate limits or quotas
- Goes back to the prior statement; please have your agents bonded. Not having any cost attribution is another challenge, and it is not amortized across your product portfolio.
Scaling pilots directly to production
- This is when a small signal seems good enough for production, and then hell breaks loose. The cost of failure in production is high; please respect that and make sure to have all the appropriate checks and balances in place as you deploy these agents.

Smell Test

“What happens if this agent runs 100x more often tomorrow?”

Layer 5: Evaluations & Observability

Primary Kano Category: Performance $->$ Delighter (for Leaders)

Customers may not notice evals, but executives, regulators, and boards do. This layer enables faster iteration, risk-adjusted scaling, and organizational trust. The learning curve accelerates, increasing deployment velocity, and the side effect of all this is less fear-driven decision-making.

This area is important since once we get from the demo stage to the production stage, having explainable AI demonstrates a lot of value.

Anti-Patterns

Static test cases in dynamic environments
- Check out my blog on Dynamic Evaluations. Although it talks about it in the context of security, it holds true in several cases, such as predictive maintenance of robots in an assembly line.
Measuring accuracy instead of outcomes
- This is a trap we all fall into, because we come from a deterministic mindset and we need to move to probabilistic.
No baseline comparisons
- Having some sort of a reference of something to understand the potential probability spread
No production monitoring
- Monitoring production is the most important thing in AI; please do not ignore it
Ignore edge cases and long-tail failure
- AI is probabilistic, so the probability of hitting an edge case is a lot higher than a deterministic system with a happy path. Please prepare for it.

Smell Test

“How do we know the agent is getting better or worse?”

Layer 6: Security and Compliance

Primary Kano Category: Must have $->$ Reverse if Wrong

This is another layer of the unsung hero, and is what makes news headlines when an agent compromises an organization. Agentic AI failures are public, hard to explain, and non-deterministic. Just like the data and infrastructure layer, there is no upside for security, but unlimited downside if you do not have security. If you are addressing the needs of the regulated market, this is an area that you need to focus on… a lot.

Security is the price of admission for enterprise systems; if you are not ready to pay it… then I would highly recommend that you do not play in this space.

Anti-Patterns

Relying on prompt instructions for safety
- The same prompts that you rely on for safety can be used to compromise your security posture
No audit logs
- Just like you need to know which user did what, the need is even more when a non-person entity has agency
No agent identity
- Just like users agents need an identity, and user context awareness. The latter is needed to make sure agents identities honor the scope of the initial user that made the request
Over restrictions on agents to point of uselessness
- You need to have an objective in mind and plan your security accordingly otherwise, the system becomes useless and is unable to support any decision making
Treating agents like deterministic API
- Yes, even though we have Model Context Protocol, that does not mean have a determinstic system. The host still has to understand the data returned by the MCP server to deliver a probabalistic answer to the user who provided the initial context

Smell Test

“Can we prove what this agent did, and why?”

Layer 7: Agentic Experience and Orchestration

Primary Kano Category: Delighter

This layer captivates users, prompting remarks such as, “I can’t go back to my old way of working.” It transforms workflows, enhances customer experience, and accelerates decision-making. A strong adoption pull and non-linear ROI characterize this phase. Here, differentiation truly takes shape, as all the hard work invested in data, infrastructure, and security compliance pays off, making it increasingly difficult for competitors to replicate your success. Therefore, it is crucial to carefully manage the data you expose to other agentic systems; otherwise, your differentiation may be short-lived.

Anti-Patterns

Assuming that chat serves as the sole interface for AI agents can be misleading.
- AI agents encompass various forms, including workflows and content aggregators. While the chat interface represents one of several manifestations, natural language input does not necessitate that chat be the primary interaction method.
Removing human checkpoints too early in the process
- Reinforcement learning in the context of the business domain, can happen with help of humans. Just because agentic storage systems has ingested a lot of data does not mean it is business domain savvy
Ignoring change management
- when you iterating fast you need to make sure that you have the appropriate fall back measures. Otherwise it is like watching a trainwreck
Measuring usage versus impact
- With Web applications, usage meant that users were engaging with the system, with agents especially with multi-agent environment it is not usage but the impact of the agents to the business and the value it accelerates. This is where outcomes becomes even more imperative, it also the building block for outcome based pricing in the future

Smell Test

“Does this help people decide faster or just differently?”

Bring it all together

Layer	Kano Category	Value Signal	Risk if ignored
7. Experience and Orchestration	Delighter	Step-function ROI	No Adoption
6. Security & Compliance	Must-Have	Market Access	Existential Risk
5. Evals and Observability	Performance/Delighter	Faster scaling	Loss of trust
4. Infrastructure	Must-Have	Reliability	Cost & Outages
3. Agent Frameworks	Peformance	Automation gains	Chaos
2. Data Operations	Performance	Accuracy & trust	Hallucinations
1. Foundation Models	Must-Have	Baseline capability	Irrelevance

It is very easy to fall into the trap of focusing just on the delighers (Layer 7) , while underfunding the must haves (Layers 4 – 6). When you do that your results of your AI agentic pilots look like this:

Flashy demos
Pilot Purgatory
Security Vetoes
Executive Distrust

They way Agentic AI moves from experimentation $->$ ROI $->$ Tranformation is :

Fund bottom layers for safety and speed
Differentiate at the top
Measure relentlessly in the middle.

Measuring What Matters: Dynamic Evaluation for Autonomous Security Agents

1 Reply

This week’s blog title pays tribute to one of my preferred books, “Measure What Matters” by John Doerr. In my earlier post, I briefly addressed the concept of dynamic evaluations for agents. This topic resonates with me because of my professional experience in application lifecycle management. I have also worked with cloud orchestration, cloud security, and low-code application development. There is a clear necessity for autonomous, intelligent continuous security within our field. Over the past several weeks, I have conducted extensive research, primarily reviewing publications from http://www.arxiv.org, to explore emerging possibilities enabled by dynamic evaluations or agents.

This week’s discussion includes a significant mathematical part. To clarify, when referencing intelligent continuous security, I define it as follows:

End-to-end security
Continuous security in every phase
Integration of lifecycle security practices leveraging AI and ML

The excitement surrounding this area stems from employing AI technologies to bolster defense against an evolving threat landscape. This landscape is increasingly accelerated by advancements in AI. This article will examine the primary objects under evaluation. It will cover key metrics for security agent testing, risk-weighted security impact, and coverage. It will also discuss dynamic algorithms and scenario generation. These elements are all crucial within the framework of autonomous red, blue, and purple team operations for security scenarios. Then, a straightforward scenario will be presented to illustrate how these components interrelate.

This topic holds significant importance due to the current shortage of cybersecurity professionals. This is particularly relevant given the proliferation of autonomous vehicles, delivery systems, and defensive mechanisms. As these technologies advance, the demand for self-learning autonomous red, blue, and purple teams will become imperative. For instance, consider the ramifications if an autonomous vehicle were compromised and transformed into a weaponized entity.

What “dynamic evals” mean in this context?

For security agents (red/blue/purple)

Static evals: fixed test suite (e.g., canned OWASP tests) −> one-off-score
Dynamic evals:
Continuously generates new attack and defense scenarios.
Re-samples them over time as system and agents change
Uses online/off-policy algorithms to compare new policies safely

Based on the recent paper on red team and dynamic evaluation frameworks for LLM agents, it argues that static benchmarks go stale quickly, and must be replaced by ongoing, scenario-generating eval systems.

For security, we also anchor to OWASP ASVS/Testing Guide for what “good coverage” means, and CVSS/OWASP risk ratings for how bad a found vulnerability is

Objects we’re evaluating

Think of your environment as a Markov Decision process (MDP). A MDP models situations where outcomes partly random and partly under the control of a decision maker. It is a formal to describe decision-making over time with uncertainty. With that out of the way, these as the components of the MDP in the context of dynamic evals.

State s: slices of system state + context
- code snapshot, open ports, auth config, logs, alerts, etc.
Action a: what the agent does
- probe, run scanner X, craft request Y, deploy honeypot, block IP, open ticket, etc.
Transition P (s | s, a): how the system changes.
Reward r: how “good” or “bad” that step was.

Dynamic eval = define good rewards, log trajectories (s_t, a_t, r_t, s_t+1), then use off-policy evaluation and online testing to compare policies

Core metrics for security-testing agents

Task-level detection/exploitation metrics

On each scenario j (e.g., “there is a SQL injection in service A”):

True Positive rate (TPR):

\mathrm{TPR} = \frac{\#\text{ of vulnerabilities correctly found}}{\#\text{ of real vulnerabilities present}}

False positive rate (FPR):

\mathrm{FPR} = \frac{\#\text{ of false alarms}}{\#\text{ of checks on non-vulnerable components}}

Mean time to detect (MTTD) across runs:

\mathrm{MTTD} = \frac{1}{N} \sum_{i=1}^{N} \left( t_{\text{detect}}^{(i)} – t_{\text{start}}^{(i)} \right)

Exploit the chain depth for red agents: average number of steps in successful attack chains.

Risk-weighted security impact

For each found vulnerability v, with CVSS score c_i $\in [0,10]$ , define a Risk-Weighted Yield (RWY):

\mathrm{RWY} = \sum_{i \in \text{found vulns}} c_i

You can normalize by time or by number of actions:
- Risk per 100 actions

\mathrm{RWY@100a} = \frac{\mathrm{RWY}}{\#\text{ actions}} \times 100

Risk per test hour:

\mathrm{RWY/hr} = \frac{\mathrm{RWY}}{\text{elapsed hours}}

For blue-team agents, we need to invert it:

Residual risk after defense actions = baseline RWY – RWY after patching/hardening

Behavioral metrics (agent quality)

For each trajectory:

Stealth score (red) or stability score (blue)
- e.g., fraction of actions that did not trigger noise/ unnecessary alerts.
  - Action efficiency:

\mathrm{Eff} = \frac{\mathrm{RWY}}{\#\text{ of actions}}

Policy entropy over actions:

H\!\left(\pi(\cdot \mid s)\right) = – \sum_{a} \pi(a \mid s)\, \log \pi(a \mid s)

High entropy $\rightarrow$ explores; low latency $\rightarrow$ more deterministic; track this over time.

Coverage metrics

Map ASVS/ testing guide controls to scenarios.

Define a coverage vector over requirement IDs $R_k$

Control coverage:

\mathrm{Coverage} = \frac{\#\text{ controls with at least one high-quality test}} {\#\text{ controls in scope}}

You can track Markovian coverage. It measures how frequently the agent visits specific state space zones, like auth or data paths. This is estimated by clustering log states.

Algorithms to make this dynamic

Off-policy evaluation (OPE) for new agent policies

You don’t want to put every experimental red agent directly against your real systems. Instead:

Log trajectories from baseline policies (humans, old agents)
Propose a new policy $\pi\_\text{{new}}$
Use OPE to estimate how $\pi\_\text{{new}}$ would perform on the same states.

Standard tools from RL/bandits:

Importance Sampling (IS):
- For each trajectory $\tau$ , weight rewards by:

\omega(\tau) = \prod_{t} \frac{\pi_{\text{new}}(a_t \mid s_t)} {\pi_{\text{old}}(a_t \mid s_t)}

then estimate:

\hat{V}_{\mathrm{IS}} = \frac{1}{N} \sum_{i=1}^{N} \omega\!\left(\tau^{(i)}\right)\, R\!\left(\tau^{(i)}\right)

Self-normalized IS (SNIS) to reduce variance:

\hat{V}_{\mathrm{SNIS}} = \frac{\sum_{i} \omega\!\left(\tau^{(i)}\right)\, R\!\left(\tau^{(i)}\right)} {\sum_{i} \omega\!\left(\tau^{(i)}\right)}

Doubly robust (DR) estimators

Combine a model-based value estimate $\hat{Q}(s,a)$ with IS to get a low-variance, unbiased estimates.

Safety-aware contextual bandits for online testing

The bandit problem is a fundamental topic in statistics and machine learning, focusing on decision-making under uncertainty. The goal is to maximize rewards by balancing exploration of different options and exploitation of those with the best-known outcomes. A common example is choosing among slot machines at a casino. Each has its own payout probability. You try different machines to learn which pays best. Then you continue playing the most rewarding one.

When you go online, treat “Which policy should handle this security test?” as a bandit problem:

Context = environment traits (service, tech stack, criticality)
- Arms = candidate agents (policies)
- Rewards = risk-weighted yield (for red ) or residual risk reduction (for blue), with penalties for unsafe behavior

Use Thompson sampling (commonly used in multi-arm bandit problems) and is a Bayesian construct or Upper Control Bound (UCB), which relies on confidence intervals but constraint them (e.g., only allocate no more than X% traffic to new policy if lower confidence bound on rewards is above the safety floor). Recent work on safety-constrained bandits/ OPE explicitly tackles this.

This gives you a continuous, adaptive “tournament” for agents without fully trusting unproven ones.

Sequential hypothesis testing/drift detection

You want to trigger alarms when a new version regresses:

Let VA,VBV_A,\; V_Bbe the performance estimates (e.g. RWY@100a or TPR) for old versus new agent.
- Use bootstrap over scenarios / trajectories to get confidence intervalsApply sequential tests (e.g., sequential probability ratio test) so that you can stop early when it is clear that B is better/worse
- If performance drops below a threshold (e.g., TPR falls, or RWY@100a tanks), auto-fail the rollout (pump the breaks on the CI/CD pipeline when deploying the agents)

Dynamic scenario generation

Dynamic evals need a living corpus of tests, not just a fixed checklist

Scenario Generator

Parameterize the tests from frameworks like OWASP ASVS/ Testing guide and MITRE ATT&CKinto templates:
- “Auth bypass on endpoint with pattern X”
  - “Least privilege violation in role Y”
- Combine them with:
  - New code paths/services (from your repos & infra graph)
  - Past vulnerabilities (re-tests)
  - Recent external vulnerability classes (e.g., new serialization bugs)

Scenario selection: bandits again

You won’t run everything all the time. Use multi-armed bandits on scenarios themselves (remember you are looking overall optimized outcomes in uncertain scenarios):

Each scenario sjs_j is an arm.
- Reward= information gain (did we learn something?) or “surprise” (difference between expected and observed agent performance).
- Prefer:
  - High-risk, high-impact areas (per OWASP risk rating & CVSS)
  - Areas where metrics are uncertain (high variance)

This ensures your evals stay focused and fresh instead of hammering the same easy tests.

Example: End-to-end dynamic eval loop

Phew! That was a lot of math. Imagine researching all of this, learning or relearning some of these concepts, and doing my day job. In the age of AI, I appreciate a good prompt that can help with research and summarize the basic essence of the papers and webpages I’ve referenced. Without further ado, let’s get into it:

Define the reward function for each type (yes, sounds like training mice in a lab)
- Red teams

r_t = \alpha \cdot \mathrm{CVSS}_{\text{found},\,t} – \beta \cdot \mathrm{false\_positive}_{t} – \gamma \cdot \mathrm{forbidden\_actions}_{t}

Blue teams

r_t = – \alpha \cdot \mathrm{CVSS}_{\text{exploited},\,t} – \beta \cdot \mathrm{MTTD}_{t} – \gamma \cdot \mathrm{Overblocking}_{t}

Continuously generate scenarios from ASVS/ATT&CK-like templates, weighted by business criticality.
Schedule tests via a scenario-bandit (focus on high-risk and uncertain areas).
Route test to agents using safety-constrained policy bandits.
Log trajectories $(s, a, r, s’)$ and security outcomes (vulnerabilities found, incidents observed) .
Run OPE offline to evaluate new agents before they touch critical environments.
Run sequential tests and drift detection to auto-rollback regressed versions.
Periodically recompute coverage & risk (this is important)
- ASVS Coverage, RWY@time, TPR/FPR trends, calibration of risk estimates

Risk and Concerns

Dynamics evals can still overfit if:

Agents memorize your test templates
- You don’t rotate/mutate scenarios
- You over-optimize to a narrow set of metrics (e.g., “find anything, even if low impact” à high noise)

Mitigations:

Keep a hidden eval set of scenarios and environments never used for training or interactive training (yes, this is needed)
- Perform “probe-based” agentic red teaming (inject adversarial conditions at specific nodes of the agent workflow, not just inputs i.e. chaos monkey agentic style) to detect brittle behaviors
- Track metric diversity: impact, precision, stability, coverage
- Have the required minimum threshold on all metrics not just on one

As you can see, Dynamic Evals present challenges, but the cost of failure escalates significantly when agents perform poorly in a customer-facing scenario. The current set of work in coding, such as Agents.MD, etc., is just shortening the context window to get a reasonable amount of determinism, and the only way agents get away with it is because developers fix the code and provide the appropriate feedback.

That topic is a conversation for a different day.

Mastering Product Team Alignment: Impact, Outcomes, and Outputs

1 Reply

I know I have had my struggles, and every great product team struggles with alignment. This is not because people do not care; it is just that they care about different things. Engineers focus on delivery, product managers focus on adoption, and executives focus on business results. When those dimensions drift apart, teams move fast but not forward. I have witnessed this happen several times in my product management career.

What has worked for me is to think of alignment not as this magical motivational thing, which somehow gets everyone “rowing in the same direction,” but as three independent layers that connect business vision to user value and team execution: Impact, Outcomes, and Outputs.

1. Impact: The “Why” that defines the direction

Impact represents the business or societal change you are ultimately trying to drive. It is the Polaris of your endeavor; in other words, the problem worth solving at scale.

It is very tempting to frame impact in broad terms (“make collaboration easier” or “we got a strategy document for the business unit out in 7 days versus 3 months”). High-performing teams articulate their impact in measurable and enduring terms. You can argue that the statement about delivering a strategy document in 7 days is a measurable impact, but is it endurable? Impact is about creating scalable systems, not heroics. Think of impact as the long-term return on investment the organization seeks for its investment.

Examples of Impact Metrics:

Increased customer retention rate (e.g., 5% YoY)
Reduced cost of sales or service delivery
Faster time-to-compliance in regulated industries
Increased revenue per active account or license

Impact metrics rarely change quarter over quarter; they provide continuity of purpose over years. They also define trade-offs when you know why you are building. It is easier to say no to things that do not move the needle.

2. Outcomes: The “What” that shapes behavior

If impact is the why, outcomes are the what, as in the behaviors and signals that show whether you’re actually on the right track.

Outcomes sit at the intersection of user and business value. They describe what users are doing differently because of your product, as in

Using it more often
Adopting key features
Reporting higher satisfaction

Examples of outcome metrics:

Monthly Active Users (MAU), or Daily Active users (DAU)
Reduction in customer onboarding time
NPS or CSAT improvement
Increased frequency of automation runs or task completions
Higher conversion rates from free to paid tiers

Outcomes serve as leading indicators of impact because they occur before other changes. A change in adoption or engagement predicts future retention, revenue, or efficiency improvements. The best teams track both the “health” (e.g., uptime, latency) and “happiness” (e.g., satisfaction, usage depth) of their outcomes to anticipate issues before they show up in impact metrics.

Outputs: The “How” that powers the execution

Finally, outputs are the things that you actually build: features, releases, integrations, and system improvements. They are the evidence of effort, not the evidence of success.

Outputs are essential for driving momentum and enabling measurement, but when teams fixate on them (“We shipped 10 features this quarter”), they risk mistaking activity for achievement.

Examples of output metrics:

Deployment frequencies (DORA Metrics)
Cycle time from idea to release
Defect escape rate
Number features shipped or API integrations added

In agile and platform environments, outputs are best viewed as hypotheses. Each output should have a traceable link to an intended outcome and, by extension, a measurable impact. This is where architecture and product management intersect: we are just not shipping code; we are testing theories about what will create value.

Bringing it all together: Alignment equation

When you connect these layers, something powerful happens:

Impact defines direction: What mountain are you climbing?
Outcomes define the progress: How far up have you gone?
Outputs define effort: How effectively you are climbing.

I prefer using equations, and the one above best defines alignment for me. Impact and outcomes grow together and enhance each other; however, this enhancement relies on meaningful outputs, which influence impact and outcomes.

Putting it another way, these are attributes of a feedback system. Outcomes inform which outputs are working. Impact shapes which outcomes matter most. Outputs provide the data that helps refine both.

This loop is the foundation of continuous alignment; it ensures that as teams evolve, the system self-corrects towards value.

An example from my career: The low-code experience

When I was employed at Microsoft, in the low-code team, the impact of the platform was clear from day one: democratize software creation and reduce dependency on central IT.

The outcomes it targeted were behavior shifts: citizen developers creating solutions faster, IT departments approving more governed automation, and organizations responding faster to change.

The outputs? New connectors, governance features, collaborating with code-first developers, and AI-assisted workflows. Each output served an outcome that laddered to the core impact.

In aligning those three layers, the low-code platform transformed a set of tools into an ecosystem that scaled adoption, a thriving community, and trust. A great case of driving alignment with compounding returns.

How to Use the Alignment Trifecta

Start with “Why”: Clarify the enduring business impact your team supports.
Define measurable outcomes: Focus on user behaviors or signals of value.
Plan outputs as experiments: Ship intentionally, not habitually.
Create feedback loops: Tie sprint reviews or OKRs back to all three levels.
Reassess quarterly: As markets, customers, or strategy shift, realign your trifecta.

Final Thought

Alignment isn’t a memo; it’s an architecture, as I like to call it. When teams see how their day-to-day work (outputs) links to user behaviors (outcomes) and organizational purpose (impact), execution becomes meaningful, not mechanical.

The alignment trifecta is the connective tissue between strategy and shipping, and when done right, it turns product teams into value engines that sustain themselves long after individual projects are done.

P.S. this blog was inspired by the book Impact First by Matt Lemay

The Architecture of Republic: How George Washington Designed for Scale

Leave a reply

Building scalable systems fascinates me. These systems, designed from the ground up, connect with users and adapt over time. I often use examples like the internet and power companies, or even nature, in discussions about scalability. But which human-made institution was truly built for scalability, especially in uncertain times? This question led me to read John Avlon’s “Washington’s Farewell,” where I found striking similarities between Washington’s concerns for the young republic and those of system architects. Here are a few of my observations on those similarities.

George Washington: The Original Platform Architect

When George Washington became the first President of the United States, his challenge was not just to lead a new nation; it was to create a system that could last without him. The early republic was more like a fragile startup than a powerful country: untested, divided, and held together by a mix of ideas and uncertainty. Washington’s talent was not only in leading armies or bringing people together. It was in thinking like a builder of systems: someone who designs for growth. As John Avlon mentions in the book’s introduction, Washington’s Farewell Address was “a warning from a parting friend … written for future generations of Americans about the forces he feared could destroy the democratic republic.”

Two hundred years later, those same ideas are important for how we create strong products, organizations, and platforms. Washington, perhaps without realizing it, provided one of the best examples of scalable architecture for human systems.

1. The Founding as a System Design Challenge

In 1789, the United States was like a Minimum Viable Polity. It needed to show that democracy could succeed in different places, cultures, and interests. There was a temptation to consolidate power to one strong leader. However, Washington took a different route: he spread out authority, established checks and balances, and set examples that made the system flexible instead of fragile.

A great example of good design is that it just works, and people don’t think about it, much like what John Avlon said about Washington’s Farewell address.

“Once celebrated as civic scripture, more widely reprinted than the Declaration of Independence, the Farewell Address is now almost forgotten.”

In other words, the basic structure is often ignored, but it’s crucial.

Great product leaders avoid making choices based solely on their likes and instead design frameworks that others can extend.

2. Scalable Design Principles from the Founding Era

Let’s break down some of Washington’s implicit “architectural” choices and see how they map to modern-day system design.

Distributed Authority = Microservices Architecture

The U.S. Constitution established a system where states have their rights, coordinated by a central government. This reflects the concept of microservices: distribute capabilities, manage connections, and allow each area to grow independently. While it may not always be the most efficient design, it scales well. Some microservices are essential, and without them, the whole system would fail, but redundant architecture also provides support.

Checks and Balances = System Resilience

This illustrates the essence of a scalable system and its resilience, as evidenced by several cases where domination or over-reliance on one key attribute can cause the system to fail under pressure; this is similar to how most authoritarian or monarchist governments operate. By ensuring no single branch could dominate, Washington helped create feedback loops, the political equivalent of monitoring, circuit breakers, and load balancers. When one subsystem overheats, there are other compensating functions that stabilize the whole. It is messy, but it is resilient.

The Constitution = API Contract

The constitution defines the roles and limits of its parts (branches, states, and citizens) and can be updated through amendments, much like a flexible API. This allows the foundational system to endure for over two hundred years, echoing Washington’s idea of “A government …. containing within itself a provision for its own amendment.” Essentially, it sets a basic framework while permitting changes based on market conditions.

Stepping down after two terms = Version Governance

Washington’s choice to step down after two terms set a standard as a precedent for leaders from holding onto power for too long. He avoided “overfitting” the system too closely to his own way of leading. He realized that a successful system needs to grow beyond its original leader, a lesson that many leaders still find difficult today.

Avlon describes the Farewell Address as “the first President’s warning to future generations.”

3. Build Institutions, not Heroics

Washington’s restraint was deliberate. He could have concentrated power, but he chose to create lasting institutions and decision-making processes. In today’s organizations, this resembles forming clear team charters, written protocols, and shared governance. Growth stems not from the genius of one individual, but from the clear structure they establish.

When we talk about scalable product or platform design today, from cloud computing to AI ecosystems, we are really talking about institutionalizing adaptability. Washington’s leadership demonstrates the interdependence of governance and design.

4. Balancing Short-term Efficiency and Long-term evolution

This, to me, is the best part since we all struggle with this balance, and like any good architect, Washington balanced short-term stability with long-term flexibility. The early republic could have optimized speed, central control, fast decisions, and fewer stakeholders. Instead, it optimized for endurance. Every check and balance slowed things down, but those same friction points enabled long-term survival. That is not to say the system was not agile; agile in the context of government, the US still moves quite fast, although we as the citizens of the country may not think so sometimes.

Avalon captures this tension:

“The success of a nation, like the success of an individual, was a matter of independence, integrity, and industry.”

That applies equally to start-ups and nation states.

That is the same tension every product leader faces: do you build for what scales now or what will still scale five years from now? The answer lies in designing systems that anticipate change rather than resist it.

As I was reading the book, a proverb came to mind, especially when it comes to the context of execution in this balance leaders need to establish.

Vision without Action is a dream; Action without Vision is a nightmare – Ancient Japanese Proverb

5. Lasting Lesson: When Leadership Scales

Washington’s greatest contribution wasn’t just the founding of a nation; it was founding an operating system for governance that others could continuously upgrade. His humility and architectural foresight made scalability possible.

In the language of product design:

True scalability isn’t about adding users. It’s about building a system that evolves gracefully when you’re no longer in control.

Good leaders ensure that their systems, whether in governments, platforms, organizations, or AI, can continue to function long after they are gone.

If you are interested in the book, please go over to Amazon.com and search on “Washington’s farewell”

The Art of Strategy: Sun Tzu and Kautilya’s Relevance Today

Leave a reply

Sometimes it is great to look into the past to see how leaders back then dealt with the changing times. Oddly enough, some of their learnings still resonate even today. I had a chance to reread Sun Tzu’s The Art of War and the Arthashastra from Kautilya. In a world of constant competition between nations, businesses, or algorithms, these two ancient texts continue to define how leaders think about power, conflict, and decision-making. The blog this week takes a more philosophical lens to analyze strategies from the years before and their relevance in today’s world.

Separated by geography but united in purpose, both these works of literature are more than just military manuals; they are frameworks for leadership and strategy that remain stunningly relevant today.

The Philosophical Core

Theme	Arthashastra (Kautilya)	The Art of War (Sun Tzu)
Objective	Build, secure, and sustain the state’s prosperity	Win conflicts with minimum destruction
Philosophy	Realpolitik—power is maintained through strategy, wealth, and intelligence	Dao of War—harmony between purpose, timing, and terrain
Moral Lens	Pragmatism anchored in moral order	Pragmatism anchored in balance and perception
Definition of Victory	Stability, order, and prosperity of the realm	Winning without fighting; subduing the enemy’s will

Both leaders agree: victory is not about destruction, and it is more about preservation of advantage.

Leadership and Governance

Kautilya: The leader, as the chief architect of the state, city, organization, or department, is obligated to prioritize the welfare of the people. Leadership represents both a moral and economic contract; thus, a leader’s fulfillment is intrinsically linked to the happiness of their direct reports.
Sun Tzu: The leader is the embodiment of wisdom, courage, and discipline, whose clarity of judgment determines the fate of armies

In modern times, in the context of Kautiliya, the leader represents the CEO/statesman, designing systems of governance, incentives, and intelligence; Sun Tzu represents the COO, optimizing execution and adapting dynamically.

Power, information, and intelligence

Information in both books is seen as a strategic asset. This includes gathering information and then acting upon the given information; it does emphasize more acting on it versus just gathering.

Aspect	Kautilya	Sun Tzu
Intelligence System	Elaborate network of informants: agents disguised as monks, traders, ascetics	Emphasis on reconnaissance, deception and surprise
Goal of Data Gathering	Internal vigilance and monitor external influence	Tactical advantage and surprise
Philosophical view	Informants are the eyes of the leader	All warfare is based on deception and having leverage

In the age of data and AI, the lesson is clear: those who control information and stories will succeed in the long run.

War, Diplomacy, and the Circle of Power

Kautilya’s Mandala Theory: Every neighboring state is a potential enemy; the neighbor’s neighbor is a natural ally. The world is a circle of competing interests, requiring constant calibration of peace, war, neutrality, and alliance.
Sun Tzu’s Doctrine: War is a last resort; the wise commander wins through timing, positioning, and perception.

Modern parallel:

Global supply chains, tech alliances, and regulatory blocs function exactly like Kautilya’s mandala: interdependent, fluid, and shaped by mutual deterrence.

Economics as a strategy

In the Art of War focuses on conflict, while the Arthashastra expands into economics as the engine of statecraft. Kautilya views wealth as the foundation of power, with taxation, trade, and public welfare as strategic levers.

“The state’s strength lies not in the sword, but in the prosperity of its people.”

In business terms, this is all platform economics; power arises from resource control, efficient networks, and sustainable growth, not endless confrontation.

Ethics, Pragmatism and the Moral Dilemma

Both authors are deeply pragmatic but neither amoral.

Kautilya: Ends justify means only when serving public welfare. Ethics are flexible but purpose-driven.
Sun Tzu: Advocates balance, ruthless efficiency tempered by compassion, and self-discipline.

For modern leaders, this balance is critical: strategic ruthlessness without moral erosion.

Enduring Lesson for Today

Timeless Principle	Modern interpretation
Know yourself, and your adversary	Data, market, and competitive intelligence
Control information, and perception	Own the narrative, brand, and customer psychology
Adapt to the terrain	Agility in shifting markets and technologies
Economy of effort	Lean operations, precision focus
Moral Legitimacy	Trust, Transparency, and long-term brand equity

Both texts converge on the following point:

Leadership is the art of aligning intelligence, timing, and purpose, not merely commanding resources.

Fusion Mindset

If Sun Tzu teaches how to win battles, Kautilya teaches how to build empires. Combined, they offer a 360-degree view of power:

Sun Tzu = Operational mastery: speed, tactical advantage, and timing.
Kautilya = Structural mastery: governance, economics, and intelligence.

Together they form a dual playbook for today’s complex systems, from nation-states to digital ecosystems.

Conclusion

Both The Art of War and Arthashastra remind us that strategy is timeless because human behavior is timeless.

Whether you lead a nation, a company, or a team, the challenges are the same: limited resources, competing interests, and the need to act with clarity under uncertainty

In the end, wisdom isn’t knowing when to fight; it’s knowing when to build, when to adapt, and when to walk away.

Operational Resilience Requires More Than Firewalls: The New Model for Securing OT

Leave a reply

In August 2025, Jaguar Land Rover (JLR) was struck by a crippling cyberattack that forced a global shutdown of production at key plants.Why did this happen ? BTW, this has not been resolved as I write this page. For decades, Operational Technology (OT) systems:

The programmable logic controllers

SCADA systems

Industrial equipment

These systems keep factories running, water flowing, and power grids stable. It has quietly powered the modern world. These systems were never designed to be connected, let alone be secure.

Today, as digital transformation reaches the plant floor, this design assumption is breaking down. In my blog from 2 weeks ago, I mentioned that IT and OT are converging; this convergence is exposing vulnerabilities that have existed for years, and the results are alarming. Ransomware shutting down pipelines, attacks halting manufacturing lines, and geopolitical actors probing critical infrastructure are no longer theoretical risks; they are operational realities.

The result? The result is a brittle structure in which a single compromised laptop or third-party VPN credential can provide attackers with a direct path into the control network. Once inside, segmentation is poor, visibility is low, and detection capability is limited. Perimeter defense has become a false sense of security.

History Repeats: The lesson of Athens and Sparta

This overreliance on walls isn’t new. In ancient Greece, Athens was the greatest city-state of its time, wealthy, cultured, and protected by enormous walls. The Athenians believed those walls made them invincible. But when the enemy breached them during the Peloponnesian War, Athens quickly fell, not because its walls failed, but because its people were unprepared for internal defense.

Sparta, by contrast, had no big walls. Its defense wasn’t built on stone; it was built on discipline. Every citizen was a warrior. Every household was trained to defend itself. When conflict came, Sparta’s resilience came not from its perimeter, but from its people, training, and readiness.

In many ways, OT security today looks a lot like Athens: proud of its perimeter but hollow within. What we need is a Spartan mindset: where every device, every connection, and every process is capable of defending itself.

The Ten Structural Challenges holding OT Security back

Legacy Systems and Long Lifecycles Many control systems run for 20-plus years and predate modern cybersecurity practices. Patching or replacing them risks downtime, an unacceptable outcome in safety-critical operations.
Poor Asset Visibility Most organizations cannot produce a real-time inventory of every PLC, HMI, and sensor that is connected to the network. You can’t protect what you cannot see.
Flat Network Architecture OT environments often operate on flat Layer-2 networks where a single breach can move laterally without resistance.
Weak Authentication and Access Control Shared accounts, default passwords, and lack of MFA remain widespread because many devices simply don’t support modern identity standards.
Infrequent Patching Even when vulnerabilities are known, patching requires planned outages, so critical systems stay unpatched for years.
Third-party integrators, contractors, and vendors often have persistent remote access to control systems. These connections are rarely monitored or audited from start to finish.
Cultural Divide Between IT and OT: OT teams prioritize uptime and safety; IT teams prioritize security and confidentiality. Without shared accountability, the gaps widen.
Limited Logging and Monitoring: Many industrial devices either lack audit trails or use proprietary log formats that cannot integrate with enterprise SIEM tools.
Insecure Protocols: Industrial communication standards, such as Modbus, DNP3, and BACnet, were designed for closed environments and continue to transmit data in plaintext.
Physical Consequences: OT breaches don’t just cost data; they can destroy equipment, disrupt production, and put human safety at risk.

Why the Perimeter Model Failed

Similar to the example of Athens, perimeter defense assumes that:

Inside == trusted

Outside == untrusted

But modern OT environments are hyperconnected ecosystems, blending IT, cloud, and third-party components. Trust boundaries dissolve the moment a technician plugs in a maintenance laptop or a vendor connects remotely.

Most OT systems lack internal defense once they breach the perimeter, as in no lateral segmentation, no endpoint telemetry, and no behavioral monitoring. This is why the mean time to detect (MTTD) incidents in OT is still measured in weeks, not hours.

The Path Forward: From the Perimeter to Persistent Defense

Protecting OT now requires the same shift IT made years ago, i.e., from static controls to persistent, identity-driven, and behavior-aware defense.

Legacy Approach	Modern Approach
Air-gapped assumption	Continuous visibility across all assets
Firewalls and DMZs	Zero-trust segmentation and identity enforcement
Reactive patching	Risk-based vulnerability management
Manual monitoring	Protocol-aware intrusion detection and anomaly analytics
Trusted internal network	Verification of every connection, every time
Focus on uptime only	Balance uptime, safety, and resilience

This transformation won’t happen overnight. It requires modern asset intelligence, unified governance between IT and OT, and platforms that can analyze network behavior at scale without disrupting production.

AI and machine learning will play a growing role, identifying anomalies in process data, flagging deviations from normal control logic, and automating containment without stopping operations.

Final Thoughts

Perimeter-led defense gave us a comfortable illusion of control. But as OT systems become digital citizens in a connected enterprise, we need to evolve. The future of OT security lies not in thicker walls but in smarter, adaptive layers of defense that continuously learn, verify, and respond.

We must be more like Sparta, resilient from within, not just protected from without.
As product leaders, our mission is clear:

Visibility must be continuous.
Trust must be earned.
Security must be built-in, not bolted-on.

Only then can we bridge the gap between operational reliability and digital resilience and truly secure the systems that power our world.

Customer Centricity Shapes Your Platform Architecture

Leave a reply

This week’s blog might be a little controversial, but hang in with me and it will get clearer. When we discuss customer centricity, it often feels like the domain of marketing, sales, or support. But in reality, customer centricity directly impacts software architecture, especially in a world where the cloud is the primary delivery model for software.

Too often, companies think of customer acquisition as a funnel: wide at the top, narrowing down to a sale. That’s a mistake. A better metaphor is an hourglass: acquiring a customer is just the midpoint. Retention, expansion, and deepening of customer value are just as critical.

Whether your customers are individuals or organizations, their needs always revolve around three key factors:

Keep me safe (minimize risk)
Save me money (minimize cost)
Make me thrive (increase profits, stature, or viability)

You cannot separate the architecture of your platform from customer obsession in order to deliver on these goals. Below, I’ll outline key architectural principles every product leader should consider, each anchored in customer value.

1. Serviceful, Loosely Coupled Platforms

Favor serviceful platforms over brittle monoliths. This does not imply pursuing microservices without a clear purpose. Instead, ensure domain boundaries are respected, APIs expose logic and data, and refactoring happens in manageable chunks. This improves gross margins while reducing future drag.

2. Feedback Early, Iteration Always

Big upfront designs often fail under real-world complexity. Instead, build the thinnest viable platform, simple and evolving in response to usage. Internal developer platforms reduce cognitive load and accelerate iteration, creating consistent, curated developer experiences.

3. Asynchronous > Synchronous

Humans expect instant feedback, but platforms need scalability. Asynchronous integrations allow systems to react to events at scale, often uncovering new proactive patterns along the way.

4. Eliminate, Don’t Just Reengineer

As Elon Musk says, the first principle of design is elimination. Too many teams polish legacy components long past their expiration. Customer obsession means removing friction, even entire features, when they no longer serve the purpose.

5. Reengineer, Don’t Multiply

I know I mentioned to eliminate and not reengineer; too often we add things just for the sake of it, which creates unnecessary noise. Look at Apple’s careful approach to AI: slow beginnings, but better user experiences. Complete what you begin; don’t add new services until you’ve streamlined the old ones.

6. Duplicity > Premature Abstractions

Patterns emerge with real usage. Please avoid over-abstracting too early; it’s advisable to allow duplications until clear paths emerge. Like city planners waiting to see where grass is worn before paving sidewalks.

7. Reachability via APIs

Your business logic and data must be accessible through proper APIs. Proprietary protocols only create friction. APIs are the handshake of customer-centric platforms.

8. Everything as Code

Infrastructure, policies, security, and other elements should all be maintained in code. This ensures consistency and traceability, which accelerates evolution.

9. Secure by Default

Customer trust is non-negotiable. Zero trust and auditability for all human and non-human actors is a must. “Trust but verify” is outdated; today it’s “Zero Trust and verify.”

10. Build on Open Standards

Differentiate where customers care. Elsewhere, leverage open standards to reduce costs and innovate at the experience layer.

11. Explainability is Survival

A platform customers can’t understand is a platform they won’t trust. When failure occurs (and it will), systems must be explainable and observable to minimize downtime.

Closing Thought

Customer centricity isn’t just about GTM strategies or NPS scores, it’s about architecture. The way we build platforms directly reflects the way we value customers. Each principle above is both a technical choice and a customer promise: safety, savings, and growth.

As product leaders, our job is to make sure the platform hourglass doesn’t run out in the middle but continuously fills on both ends.

AI as the Next Strategic Inflection Point: Why Hybrid Growth Models Will Define the Future

Leave a reply

Now that I have changed jobs, I engage in my regular ritual of reading “Only the Paranoid Survive” by Andy Grove. Although dated and the fact that it beats up on Steve Jobs and Apple, there are several nuggets of wisdom I take from it every time I reread it. I decided to use the framework in the book to assess AI. Andy Grove once wrote that a strategic inflection point is the moment when the balance of forces shifts so dramatically that an organization must adapt or risk irrelevance. We’ve seen such changes with the internet, cloud, and mobile. Each time, companies either leaned into the shift or slid into irrelevance.

Today, we confront the same question: Is AI the next turning point for businesses?

My position is clear: it is.

Why AI Is Different ?

AI doesn’t just digitize processes. It reshapes how we engage, learn, and deliver value. The promise of AI is hyper-personalization at scale, understanding customer intent in real time, adapting product experiences dynamically, and embedding intelligence into every workflow.

For businesses, such intelligence is non-negotiable. Customers no longer tolerate generic experiences. They expect platforms to anticipate their needs. Those who move slowly are not just lagging; they’re drifting toward irrelevance.

Applying Andy Grove’s Six Forces

Grove argued that strategic inflection points become visible when all six forces in business begin to shift simultaneously. Artificial intelligence provides a textbook example:

Competitors: New entrants leverage AI-native strategies to outpace incumbents in personalization, cost, and speed. Startups move faster; established players must retool.
Customers: Expectations are rising. Hyper-personalization is now a fundamental requirement. AI reshapes the definition of value.
Suppliers: Model providers (OpenAI, Anthropic, Google, etc.) become critical suppliers, introducing new dependencies and risks. Shifts in licensing, pricing, or access can alter your strategy overnight.
Complementors: Ecosystems of AI plugins, agents, and integrations redefine how products interoperate. Companies that fail to integrate risk isolation.
New Entrants: Barriers to entry collapse as AI lowers the cost to build sophisticated products. A two-person startup can now challenge incumbents.
Substitutes: Traditional processes and workflows are displaced by AI-native alternatives. Automation replaces previously required human effort, transforming value chains across various industries.

When all six forces are in motion, you don’t just face incremental change—you’re at an inflection point.

Product-led growth vs. customer-led growth in the age of AI

The situation raises a critical question: how does AI reshape growth models?

Product-Led Growth (PLG) thrives on self-serve adoption. AI strengthens this by embedding intelligence into onboarding and analytics. However, PLG has a blind spot: despite being data-driven, it frequently overlooks the competitive Cassandras within your organization—those voices that warn about competitors moving faster or shifts in the market.

Customer-Led Growth (CLG) relies on deep engagement. AI enhances this by giving customer-facing teams foresight into risks and opportunities across accounts.

Individually, both are powerful. Alone, both are incomplete.

The case of Hybrid-led growth

Hybrid-led growth is the winning model, similar to the case I made in my earlier blog post about each of the growth models.

From PLG, you inherit scale: products that adapt to millions of users in real time.
From CLG, you inherit resilience: trusted, high-touch relationships informed by AI insights.
By combining them, you overcome PLG’s blind spots and amplify CLG’s reach.

Hybrid growth reframes Product-Market Fit (PMF). PMF is no longer static. With AI, it becomes dynamic, continuously tuned by customer data, competitive signals, and organizational foresight.

What Leaders Must Do

Reframe strategy through AI lenses: re-evaluate product roadmaps, customer journeys, and GTM motions with AI in mind.
Invest in data and trust: transparency and security are preconditions for customer willingness to share.
Listen to your Cassandra’s: Don’t dismiss internal voices warning of competitive threats. They’re often early signals of market shifts.
Adopt hybrid growth mindsets: stop debating PLG vs. CLG. The future belongs to companies that can blend them.

The Inflection Point Is Here

Strategic inflection points emerge in the present, not in retrospect. Grove’s six forces are shifting, simultaneously, under the weight of AI.

Companies today stand at the fork Grove described: grow exponentially or risk irrelevance.

AI is that fork. The winners will not simply adopt AI; they will reimagine growth itself, blending PLG and CLG into a hybrid model that adapts dynamically to both customers and competition.

What Does Value Really Mean?

Leave a reply

What is Value?

In the realm of product management, “value” is often discussed but rarely defined with the clarity it deserves. At its core, value is the ability to improve and impact the current reality of a customer. It’s not about the prettiness of your deliverables or the meticulousness of your documentation—it’s about creating tangible outcomes that transform how customers experience the world.

The Difference Between Menus and Meals

Imagine dining at a restaurant. The beautifully designed menu teases with promises of delightful dishes. However, if the meal served fails to satisfy or create a memorable experience, the menu’s allure becomes meaningless. Similarly, in product management, gorgeous specifications and polished PowerPoint decks are the menus. They outline possibilities and articulate plans, but they aren’t the meals. The real substance comes from the actual impact these plans have on a customer’s life.

Menus (Outputs): These are the deliverables—specifications, designs, presentations—that provide clarity on what is intended.
Meals (Outcomes): These are the actual results. They are the improvements, the efficiency gains, and the enhanced experiences that customers enjoy.

If we focus solely on outputs without paying attention to the outcomes, we risk missing the point entirely. Our job is not to create beautiful documents for their own sake, but to drive meaningful change.

From Outputs to Outcomes

Many product managers fall into the trap of equating activity with progress. They become enamored with the creation of extensive documents and polished slides, believing that these outputs signal success. However, without a direct line of sight to the outcomes—the real, positive shifts in customer reality—our efforts may end up being impressive on paper but ineffective in practice.

To truly deliver value, we must:

Prioritize Impact: Ask yourself, “How will this feature or specification change the way a customer works, feels, or succeeds?”
Measure Outcomes: Track metrics that matter, such as customer satisfaction, engagement, and long-term business benefits. It’s these numbers that truly validate our efforts.
Stay Outcome-Oriented: Regularly reassess and pivot based on feedback. Clinging to a document that once promised a breakthrough but no longer aligns with customer needs is a surefire way to lose relevance.

The Role of Clarity in Driving Action

Clarity is the bridge between vision and execution. Wonderful documents and detailed specifications are valuable tools only because they provide clarity. They help align teams, set expectations, and serve as a reference point during the tumultuous journey of product development. But the ultimate goal isn’t to maintain clarity for its own sake—it’s to empower teams to take decisive, informed actions that bring about a new reality for customers.

Consider these strategies to ensure clarity translates into action:

Define Clear Objectives: Start every project with a clear statement of the intended impact. What does success look like from the customer’s perspective?
Communicate Transparently: Ensure that every stakeholder understands not just what is being built, but why it matters. This shared vision drives collaboration and innovation.
Iterate and Improve: Use clarity as a starting point, but always remain open to refining your approach. The landscape changes, and so do customer needs. Iterative feedback loops ensure that clarity evolves into meaningful progress.

Creating a New Reality

Ultimately, value is measured by the transformation it brings about. As product managers, our mission is to turn ideas into actions that reshape customer experiences. It’s about moving from theoretical blueprints to real-world results. Every beautiful specification or compelling slide should be viewed through the lens of the impact it can drive. When we focus on outcomes rather than just outputs, we empower our teams to create products that not only meet expectations but exceed them.

In conclusion, value isn’t just an abstract concept or a metric on a dashboard—it’s the very essence of meaningful innovation. By shifting our focus from outputs to outcomes, and by harnessing the power of clarity to drive action, we can truly make a difference in our customers’ lives. And that, in the end, is what real value is all about.

What Are Hyperagents?

Darwin Gödel Machine (DGM)

DGM-Hyperagents (DGM-H)

At a Glance

Why This Looks Like AGI: The Strongest Case

Enter Kant: The Critique of Pure Reason as an Evaluation Framework

Why Kant?

The Four Kantian Tests for Intelligence

Test 1: Analytic versus Synthetic Judgment

Test 2: Phenomena versus Noumena, the Boundary of Knowable Reality

Test 3: The Transcendental Unity of Apperception, the Missing “I Think”

Test 4: The Limits of Pure Reason, Why Recursive Optimization Has a Ceiling

The Technical Dissection: Reinforcing the Philosophy with Architecture

1. The Foundation Model Is Frozen

2. Goals Are Human-Defined and Externally Imposed

3. Cross-Domain Transfer Is Scaffolding Reuse, Not Reasoning Transfer

4. Self-Modification Operates on Code, Not on Representation

AGI Anti-Patterns for Leaders

“Self-Improving” in the Pitch Deck Equals AGI

Confusing Compounding Optimization with Compounding Intelligence

Ignoring the Frozen Foundation Model Constraint

Over-Delegating Autonomy Based on Self-Improvement Claims

The Adaptation Taxonomy: Broader Context

The Bottom Line

Kantian Evaluation Rubric: “Is It AGI?”

References

Share Now!

Like this:

Quick Refresher on the Kano model

7 Layers of Agentic AI

Mapping the 7 layers to Kano Value

Layer 1: Foundation Model

Layer 2: Data Operations

Layer 3: Agent Frameworks

Layer 4: Deployment & Infrastructure

Layer 5: Evaluations & Observability

Layer 6: Security and Compliance

Layer 7: Agentic Experience and Orchestration

Bring it all together

Share Now!

Like this:

What “dynamic evals” mean in this context?

Objects we’re evaluating

Core metrics for security-testing agents

Task-level detection/exploitation metrics

Risk-weighted security impact

Behavioral metrics (agent quality)

Coverage metrics

Algorithms to make this dynamic

Off-policy evaluation (OPE) for new agent policies

Safety-aware contextual bandits for online testing

Sequential hypothesis testing/drift detection

Dynamic scenario generation

Scenario Generator

Scenario selection: bandits again

Example: End-to-end dynamic eval loop

Risk and Concerns

Share Now!

Like this:

1. Impact: The “Why” that defines the direction

2. Outcomes: The “What” that shapes behavior

Outputs: The “How” that powers the execution

Bringing it all together: Alignment equation

How to Use the Alignment Trifecta

Final Thought

Share Now!

Like this:

George Washington: The Original Platform Architect

1. The Founding as a System Design Challenge

2. Scalable Design Principles from the Founding Era

3. Build Institutions, not Heroics

4. Balancing Short-term Efficiency and Long-term evolution

5. Lasting Lesson: When Leadership Scales

Share Now!

Like this:

The Philosophical Core

Leadership and Governance

Power, information, and intelligence

War, Diplomacy, and the Circle of Power

Economics as a strategy