AI Research Engineer — Cognitive and Multimodal Foundation Models

The role

Investigate computational models that can learn from behavioral and multimodal observations. The focus is measurable prediction and careful validation rather than an impressive demonstration alone.

The work

• Investigate and reproduce relevant cognitive-modeling, behavioral prediction and multimodal learning research.
• Build training and inference pipelines around open-source language and multimodal foundation models.
• Design models that incorporate person-specific priors, interaction histories and new observations.
• Explore Bayesian inference, probabilistic prediction, representation learning, preference modeling and uncertainty calibration.
• Investigate multimodal fusion across text, behavior, audio, video and physiological signals.
• Develop evaluation frameworks using held-out future behavior rather than retrospective reconstruction alone.
• Compare complex models against strong non-neural and general-purpose baselines.
• Build live demonstrations that translate scientific results into useful intelligent interfaces.

What matters

• Strong fundamentals in machine learning, deep learning, probability and statistical inference.
• Excellent Python and PyTorch experience.
• Experience working with transformers, LLMs, multimodal models, fine-tuning and model evaluation.
• Familiarity with representation learning, sequence modeling and statistical experimentation.
• Ability to read research papers critically, reproduce experiments and make independent technical decisions.
• Strong interest in cognitive science, computational neuroscience or probabilistic models of behavior.

What good work looks like

An independently evaluated model with held-out tests, strong baselines, and calibrated uncertainty.

Why this work matters

Make meaningful predictions about human cognition and test them against reality.

Get in touch

Send us your CV, your personal website or examples of work you have done, particularly a machine learning project you evaluated carefully, including what did not work. Include an informal handwritten statement of purpose, with no prescribed length, telling us why this mission matters to you, what difficult work has taught you, and what you would like to take responsibility for at Walnut. This mission will demand scientific rigor, patience, independent thinking and a willingness to test ideas against reality. Write the statement entirely in your own words, without using generative AI. We will not consider statements written using AI. A photograph or scan of the statement is sufficient, and a typed version is welcome if handwriting is difficult. A reference from someone who knows your work is welcome, but not required.