dataset creation for AI in design

My main objective is to create training datasets for AI models that create and edit computer aided designs. These research projects involve a range of techniques including bespoke dataset creation, annotation pipelines, data augmentation, and synthetic generation methods.

multimodal instruction following

Designers want to do more than prompt a model. Naturalistic design communication depends on the precision and richness of visual communication, and interfaces that support this. My collaborators and I recently developed a dataset of multimodal CAD editing instructions (sketches and text) paired with corresponding edits to 2D CADs. This work highlights how language and drawing complement one another in guiding design, and paves the way for AI systems that understand instructions in more natural and expressive ways.

benchmarking human–AI creativity

Benchmarks are a key driver of AI development, but for me the real measure of progress is whether AI helps people work more creatively and productively. Towards this goal, I develop benchmarks of AI-augmented human creativity, to measure the improvement of AI as a useful tool, and drive the development of models that help people build better things.