2023 in Review

life, review

30 Dec 2023

22 minute read

Skip table of contents

Table of Contents #

If you want to talk to me about any of this, chat, or have any recommendations on anything, email me or DM me on Twitter.

2023 was. 2023 was… Settling into college in Boston, settling into a lot of things, picking the stable path in many ways, but finding new strength and resolve within; notably in my perspectives on community and what I want to accomplish. These are big questions that for me right now lie split between understanding the world and making it better.

Lot of firsts; doing my first real job this summer at Dust, where I explored product design and coded with really competent people… Seeing a bit of the world, traveling alone, in Tokyo actually, unexpectedly – there’s something vulnerable and uncomfortable about walking around a city, 15 km a day, alone, and seeing what it’s like, that really brought out a lot of good things for me… Teaching a class for the first time, on abstract algebra to high schoolers… Going to my first rave…

It feels like on a higher level a lot of this year has been about noticing conflicts – France vs US, grind vs pure aesthetics, impact vs pure aesthetics, exploration vs exploitation (sounds cliche but actually), me vs world, math vs software, safety vs innovation, ambition vs nonambition (actually) etc.. . and feeling that these crutches actually explain a lot of my internal confusion about key questions. It’s been an existential year. I’m realizing more and more that I have existential tendencies, and that this is also the lens I use to understand and interact with people. My reading this year reflects this!

In a few words though, it was also a year of mathematics, of grinding it and getting better; of philosophy and getting clearer frames for what I want from/for the world; of writing, reading and poetry; of climbing; of software and of AI…

Goals #

My goals are always kind of shooting upwards, and it’s easy to find lists of things intimidating, but remember the process is often not as cool as it sounds.

Last year’s #

2024 Goals #

Projects #

What I wanted to do #

What I did #

Dust #

This summer I interned at Dust, a startup focused on developing AI tools for enterprise work. This fits into my interest in using AI to help humans come up with ideas and learn with technology, and I was particularly interested in how insights in my work could be applied to how individuals work, even outside of organizations. I was given a lot of freedom and started with some fixes and building some infrastructure (like a web extension) for personal indexing. Then I got interested in building a new interfadce and started working on Gens, a power user tool that would allow users to integrate relevant documents, ideas, and notes from different data sources to build workflows with their AI assistants. I worked closely with the technical cofounder, Stan to develop the core functionality using React, TypeScript, and building on their Rust backend. My goal was to craft a writing interface where users could pull insights from their data sources to assist the creative process. On top of a bunch of coding, I also found myself thinking a lot through user workflows, testing prototypes, and gathering feedback to refine the user design.

Firelight #

This is the learning community I’m working on at MIT! We did some stuff mostly spring semester, but I need to kick things up a bit more next year.

We threw a blogathon which was very successful where people wrote something new every 30 mins, [see submissions here], did a learnathon where people learned/taught each other stuff, and we’re doing a weekly story reading group.

I also taught abstract algebra to a bunch of highschoolers in 2 hour classes for [Splash], which was fun and it seems we actually managed to give them the big ideas of groups, group actions, subgroups, quotients/homomorphisms!

Research #

This year started with me exploring some ideas in January related to developmental interpretability and how gradients of neural networks can be used to find clusters in input space that correspond to mechanisms in how the model solves separata sub tasks. I sort of ambled a bit and then the method lead to something that worked on LLMs, allowing us to find clusters of inputs for different subtasks the LLM was solving, like completing sequences of numbers for example (this was a cluster). This work was my contribution, particularly in the toy setting of sparse parity, to the Quantization hypothesis of neural scaling paper, that has a lot of cool ideas and proposes a model for LLM scaling and emergence.

I am quite interested in deep learning theory and think understanding generalization/inductive biases/the effect of regulariation is a key/super cool research question I want to work on. I want to try more mathy types of research given that I’ve done a lot of coding.

In the later third of the year I’ve been looking more at program synthesis, particularly after being impressed with Dreamcoder. I think there’s interesting stuff at the intersection of program synthesis and intepretability, and I also think it’ll be important for reasoning, but I’m not working on that.

I want to get better models of research and fields. It’s been hard! Which is to be expected, but I am making slow progress. Motivation and interest need to be constantly maintained which is difficult, but I want to feel like I am doing research that I own, that I am leading and fully owning up to, and I don’t think what I’ve done for now is at that point. Excited to try to get there.

What I want to do #

At MIT I’ve been feeling like I don’t really enjoy CS classes in a formal context with exams, probably because of my self-taught background, and I think I’d rather just implement things and develop CS knowledge like that. I’ve also been enjoying Rust, so I think implementing primitives in networking, databases, concurrency, etc… in Rust or another low-level language would be good.

I’ve been toying around with some better interfaces for language model x knowledge stuff for cyborgism. I think there are more useful ways of organizing data at smaller scales for personal workflows. I’m particularly interested in better self-organizing setups with automated recall and very high quality connections, but doing something here would be nice.

I also want to consolidate my Anki setup and review past courses, I think otherwise I easily lose a lot of context that would be useful if I want to build up a web of knowledge.

I want to do a research internship in industry this summer but getting one has been hard. Generally looking to work on hard open problems that matter at least a little, let me know! Otherwise I’ll just do my own thing.

Learning #

What I wanted to learn #

What I learned #

I’m thinking of changing the format for this section up. A lot of this entire post is about learning, so I’ll just be going into things with some more detail.

Dust #

This internship working at an early stage startup taught me a lot. I learned that independence requires accountability and clear planning to effectively deliver, especially in environments where things are moving fast and no one really knows what product the world wants. I want to be very independent but it is also pretty hard when you don’t know what you’re doing in a given context.

On another note, I find it kind of hard and excruciating to do the last 5% that turns a prototype into something a bit polished, and that can be key in some contexts. Gens was somewhat of a power user interface with a lot of complexity and to deliver in that scenario is precisely where you need a lot of product design. Product design is so important and hard! You can put so much thought into building designs and interfaces that naturally give your users the intuitions/reflexes to use your tools.

Working at a startup in the future seems fun. The founders were playing with a lot of constraints in terms of technical output and social considerations around funding and team dynamics/etc… and are doing well. I made a lot of good memories and am excited to see what they get up to.

On the technical side:

Courses #

The spring semester became a semester somewhat purely focused on my courses, as I grew a bit overwhelmed with a pretty heavy courseload, and didn’t have much time for anything else. I think I had to do this to feel what it was like and decide not do it again, probably (gotta live your ego!).

For the fall, I was trying to take it more light, but ended up going still a bit too hard on courses, albeit much more chill than the spring.

See post on classes.

For next semester, I’m planning on taking a lighter load and actually really focusing on my projects. It’s just hard to because it feels like there’s always so much to learn, but everything is a tradeoff and it feels like AI is just blazing ahead. Nonetheless, here is the long list of things that seem interesting for next semester. Bolded are those I am trying to learn in 2024.

Math:

Physics:

CS:

Misc

Humanities (need CI H):

Too many things to do in one life…

For the fall, I’m also interested in:

Philosophy #

I’ve been reading a lot and thinking a decent amount about time/choice/commitment and existentialist ideas in general, and this effort is giving me a better idea of what some type of philosophical work/question asking look like. I think I want to maybe attempt to write something about this soon, but on a high level this year studying some philosophy has been useful for me to get some exposure to other systems of meaning, and see how philosophers think about what values they are living their lives to, in a moral relativist fashion, for example in Beyond Good and Evil.

I think EA is one such system, but it also doesn’t truly resonate and reading this stuff helped me realized that by staring down into the abyss.

I’ve been noticing a lot related to the conflict between the idea of life as a continuous series of choices leading to a continuous series of becomings vs a perspective of letting yourself flow into what will happen and feel the world as it turns you into what you become. [College and choice is related], but more upcoming on this.

More stuff I want to learn for 2024 #

Completes other sections.

Writing #

What I wanted to write #

~ I forgot to write this last year.

What I wrote #

I wrote more this year. This market also motivated me a bit. I wrote so much I became an MIT admissions blogger! I think I have a much better grip of how to write about myself since this year, mostly through just sheer grinding and reading. The application to the MIT blogger job helped a lot, and my travels. One of my goals for next semester is to now get better at technical writing and pedagogy, which I’ve started a bit but want to improve at.

Posts: (in reverse order)

I also wrote a lot of poetry, in and outside of classes at MIT. This is more personal so you can ask to see it if you know me, and I’d recommend my poetry post, but it’s been a great experience in terms of letting myself accept forms of writing that are intentionally diffuse and allow you to sink into memories and emotion.

What I want to write #

Concrete things:

Content I liked #

Books #

What I read #

This year was kinda bathed in existentialism, which makes sense. A good year for books.

Literature #

I need to write up reviews for a lot of stuff, will do that next week and update.

Non fiction #

What I want to read #

From Goodreads, some highlights:

Articles #

In no particular order except the order my search found stuff in. These are all, in my own opinion, quite good. Probably missing some, but going back to my bookmarks, I’m happy to see how people are just writing good stuff!

What should I read next year?