Some of the projects and explorations I’m most proud of.

Research

Currently working in the Isola lab with Minyoung Huh, Tongzhou Wang, Brian Cheung on in context learning and inner optimization in transformers. Trying to submit to ICML.

Papers:

• The Quantization Model of Neural Scaling. Eric J. Michaud, Ziming Liu, Uzay Girit, Max Tegmark. NeurIPS, 2023. Fun work on modeling emergence at scale as a product of structure in the task distribution of language - personally worked on finding a method to cluster inputs based on the mechanisms they elicit in the model - thereby finding semantically meaningful clusters in natural language data.

• Lower Data Diversity Accelerates Training: Case Studies in Synthetic Tasks. In Submission. Suhas Kotha*, Uzay Girit*, Tanishq Kumar*, Gauran Ghosal, Aditi Raghunathan.

• Between the Bars: Gradient-based Jailbreaks are Bugs that induce Features. Kaivalya Hariharan*, Uzay Girit*. Accepted to NeurIPS ATTRIB, Redteaming Adversaries, currently being polished to put on arXiV.

In the past, I did some exploratory work on safe neurosymbolic RL algorithms inspired by Dreamcoder, and have another ongoing project related to explicit world modeling and model-human collaboration for science.

Software

• Archivy, self-hostable and extensible knowledge management software. Along the main software I also built a set of packages and extensions like a static site generator, a git integration, and an NLP tool.
• Dust. I worked at Dust in summer 2023, mentored by Stanislas Polu. At the time I arrived there were only 4 other people working there, and as of writing this they have grown 4x and are doing really well! Motivated by my other projects, I wanted to build power user data interfaces where you have extensive control of different workflows and capabilities on your data. That thread is now a pretty core part of the Dust product, and I’m excited to see what they become!
• Espial, an engine for automated organization and discovery of personal knowledge. See the demo here.
• AdiosCorona, a general resource on COVID guidelines built with a group of French scientists. Really neat illustrations and great advice for our difficult situation.
• More on my GitHub.