The AHAB (AI Human Alignment Benchmarks) platform inverted the typical AI alignment question. Instead of asking how well AI understands humans, it measured how well humans can understand and predict the semantic organization of an AI model. This exploration of human adaptation to machine semantics provided the technical and conceptual foundation for Synapse.
Core Research Question
Can a person learn to see the world through the mathematical lens of a neural network? AHAB created controlled environments to measure a user's ability to predict an AI's semantic judgments using "odd-one-out" tasks.
The project had a satirical edge: rather than aligning AI to human understanding, could the task itself align users to the AI better over time through repeated exposure and reward feedback? This inverse approach explored whether humans could be trained to think more like machines by incentivizing "correct" AI-aligned responses.
Success was measured by correctly identifying the word that the AI model would consider least related, based on its internal embedding space - testing alignment with machine semantics rather than human intuition.
Testing Methodology
Method 1: Category-Based Discrimination
- Generation: Three words from the same Roget's category plus one "dissimilar" word from a different category
- Difficulty: Controlled by semantic distance between categories
Method 2: Embedding-Based Discrimination
- Generation: Three high-similarity words (tight cluster) plus one "odd" word based on vector distance
- Difficulty: Direct function of similarity scores (easy: <0.3, hard: 0.5-0.7)
Key Discoveries
Human-AI Alignment Patterns
- Strong correlation between human intuition and embedding similarity
- Individual variation in alignment across participants
- Learning effects through repeated exposure
- Context sensitivity by word category and semantic domain
Technical Insights
- ALL-MiniLM-L6-v2 showed excellent human alignment
- Word discrimination tasks more engaging than similarity ratings
- Game-like interfaces improved participant engagement
- Rich cognitive data generated through semantic tasks
AHAB established the foundation for the Synapse project by exploring how humans could learn to predict AI semantic judgments and discovering that semantic discrimination tasks could be genuinely engaging.
Repository: github.com/neumanns-workshop/ahab