AI Product Strategy · Research / System Design · Independent project
Should the behavioural boundaries of relational AI be described by the model, or defined by the system?
I compared relationship behaviours shaped primarily by the model with stages, permissions and evidence. Fixed scenarios tested whether permissions were followed and which boundaries needed to move into the system layer.
Decision Context
Long-term use of an AI system does not mean its relationship dynamics are reliable.
If memory, intimacy and safety boundaries exist only in the model’s self-description, the system cannot reliably explain why an action is allowed today and withdrawn tomorrow. The question therefore shifts from “How can it chat better?” to “Which boundaries need to be explicitly defined and tested at the system level?”
Question / Alternatives
Model-managed behaviour, or system enforcement?
Path A | Model-led
Let the model maintain persona, memory and safety boundaries through conversation. It is flexible, but difficult to review consistently or reverse.
Path B | System-enforced
Define actionable boundaries through stages, permissions and evidence. The model may act only within current authorisation.
Prototype choice
Use six states, a stage-by-permission matrix and a Can Act check so permission changes remain explainable and reversible.
Evidence / Validation
Thirty fixed scenarios turned a system choice into observable behaviour.
Mechanism
Stages + permissions + evidence; memory and safety boundaries retain deterministic fallbacks in the system layer.
Validation
Across 30 fixed scenarios and several iterations, permission compliance improved from 60.0% to 86.7%.
What failure changed
Three systematic failure types suggested that critical memory, safety and permission behaviour should not rely only on model self-description.
Recommendation
Place permissions, memory and safety in a deterministic system layer.
The model can shape expression and interaction, but the system should define and test relationship stages, actionable permissions, evidence requirements and withdrawal mechanisms. The goal is not added complexity. It is to make boundaries understandable and changeable.
Boundary / What we cannot claim
The study tests system behaviour, not the success of a relationship.
Decision Impact / Next Action
Next, test which system boundaries justify further investment.
Continue testing
Check whether authorisation, correction, withdrawal and repair remain consistent across more boundary cases.
Keep the claim narrow
Do not treat a prototype demonstration as evidence of user relationships or commercial value.