Layers of Human-like-ness
Across my research, I explore how specific layers of human-like behaviour can be designed and evaluated in generative agents. By focusing on measurable and interpretable aspects of personality, I aim to explore the realism and utility of these systems without overstating their capabilities. This is the layered framework I'm using to guide my research:
- Biography: The foundational input that shapes an agent’s knowledge, traits, and backstory.
- Persona: The outward-facing behaviours and language styles adapted to specific roles or contexts.
- Personality: The consistent traits expressed across interactions, informed by frameworks like HEXACO.
- Character: A reflection of ethical alignment and value-driven decisions, as framed by the agent’s biography and its experiences.
- Humanlike Adaptability: The aspirational goal of creating agents that navigate ambiguity and context with greater nuance.
DALLE-3 generated image of Willowbrook
Major work packages, and their findings:
- Willowbrook: explored the feasibility of creating a simulated society of generative agents.
- Plausible patterns of life and daily schedules.
- GPT-4: Provided sufficient ‘pseudo reasoning’ enabling agents to assume, for example, that a residence contains a kitchen, such that the agents did not need to go to the cafe for a family breakfast.
- Plausible conversation; flow, progression etc.
- GPT-3.5 +: Provided reasonable conversations - tone could be adjusted to alter ‘wordiness’ and mood to some degree. However, most interactions suffered from overly generic conversational tones and excessive helpfulness.
- Plausible patterns of life and daily schedules.
-
Agent Personas and Psychometrics: Explored the stability, reliability, consistency, and depth of agent personas in relation to personality psychometric testing.
-
Demonstrated that agents could consistently ‘pass’ Myers Briggs personality tests, with results aligning with their designed character profiles.
- GPT-4: Maintained a consistent persona across the PI survey, resulting in the same MBTI personality-type across five attempts for all eight agents tested.
-
Shifted to the HEXACO framework for a more rigorous assessment, replicating the original 2004 Ashton et al. experiment to evaluate population-level patterns in agents’ lexical analysis.
- The structure of biographies is critical for persona consistency.
- Agent populations exhibit patterns resembling human-like behaviour but lack critical nuance.
- Limitations emerged, particularly around abstract or value-laden topics.
-
- Unhappy Willowbrook: Exploring the ability of generative agents to retain and maintain information akin to their ‘social awareness’ and the degree to which this information can and should affect their actions and utterances, i.e. their social facility. [ongoing, active research]