For the love of the game

I’m Surya, a designer and technologist. I used to be an architect. I lived and worked in a tiger reserve for a bit. I started a product design studio and left to come to New York for an art and tech grad school program.

I’m a very pessimistic person, and the way technology is being built is difficult to grapple with. So a lot of my projects are really a way to show myself there’s a better way it can be done.

I was running a product design studio prior to this, and I came to the belief that designers have a much bigger role in shaping the future, especially coming from architecture.

When I saw AI happening, I was still in India, and I was blown away. I was surprised that people around me weren’t shocked, and I knew I wanted to be closer to where this technology was being built. So I decided to go back to school and I knew NY was the creative capital of the world. I chose an art program, ITP, because I think artists are best positioned to work with new innovations even more than engineers. I felt like I had to think like an artist to really imagine the ways this new technology would interact with society, culture and real people. So I figured I’d come here, do stuff, and figure it out.

Most people were experimenting with it and using it. But there were also people who had a distaste for it, or valid criticisms, which I largely agree with. But I wanted to be more pragmatic. There are no Luddites in power. We’re not going to roll back technological progress that’s already in place.

My uncle told me something when I was a kid that stuck with me. He said, “Surya, if you feel like a parade is going off track, you don’t scream from the crowd, because no one’s going to hear you. Get in front of the parade and show them the right way.”

I think that’s why I prefer to make things, and show myself and others alternatives to ideas I believe are misguided and leading us astray.

That project is a collaboration with one of my good friends, Cameron. He’s a free spirit and a true artist.

At the time I had this grievance with generative AI image models. The only way to change the images they produced was by prompting again and again, and they always have this synthetic smell that I disliked. My very first coding language I learned was p5, a JavaScript library where you write some code, press play, and an image comes up. So to me it was obvious that code makes images. If I ask a model to make an image this way, I can edit the code, and the code becomes something the model and I can both participate in.

So I spoke with my friend Cameron and told him, “What if we RL’d a model to draw better images by simply writing code in p5?” He said, let’s do it, and this is why I love him, he’s the only one that took the idea seriously.

Most AI image models take your words and give you pixels. It gives you a JPEG file that is not malleable. It’s like you go to a website, you ask for a recipe, they give you a photograph of the dish, and that’s it.

But when you ask a coding model to make images by writing code: you get both — the image and the code behind it. You can look at it, or you can open the code and change a colour, move something, adjust a brushstroke.

Yeah it’s interesting because a lot of this was just me looking at resources online and free books like Nathan Lambert’s book on RLHF. I’d read the chapters and have Claude quiz me.

I had never done any ML or heard of RL before January this year. I think there are only two things that matter. How curious are you about these things, and how much time are you willing to give to your curiosity?

And I also relied on people in my life (who are a lot smarter than I am) asking a ton of questions and by annoying the hell out of them. That’s how I learned.

The people I worked with were all alumni of Recurse Center, a programming retreat, and it’s the best place to learn and grow in New York, in my opinion. When I started to code and work in software, Max, Sean, and Cameron, who were alumni, never said “This is not how you do it.” It was always, “Code can do anything you want, so what do you want?”

I often find that what we’re doing on Twitter is very out of touch with how this technology meets the average person in their daily life. I keep asking myself, is this good? Does this technology empower human agency?

One example project I did recently was with this wet lab in Manhattan. My friend Fabri and I grabbed coffee with one of the lead researchers and asked, have you guys tried AI? They said no so we built them a group chat where they could talk to AI in it. They told us it saved them 10 to 30 percent of their time. They’re doing research for leukemia drugs, and it’s 10 years to get the drugs out. If you can save one to three years, that’s net good, net really good.

Then there’s my friend Kim, who runs a coffee shop called Fasan on Allen Street. Max and I helped him set up an AI tool for his business. The best part wasn’t the time he saved — it’s that he started updating a journal on his website and made a Substack, because now he had the ability and time to write about his life. He writes about how much he loves his wife and his daily interactions with customers. That’s what this is for. You’re doing this so people have their life back.

Being at WebSim has been a life-changing experience. Max (CTO of Websim) was my professor at ITP. The folks there never said an idea wasn’t possible. Instead they asked, why? Is it fun? Was it creative? Was it interesting? That reframed the way I think about software.

WebSim has a very young audience, and when you’re making stuff for kids, you can’t not be creative. Making tools for empowering a creative audience rewires the way you think. Now what I’m most interested in is what it means for people to collaborate with AI agents and other people wielding AI agents, especially for fun. How do you make the internet feel creative again?

I have conversations with people where they say something so inspiring that I get nerd sniped. Then I’ve got to go make that feeling into something.

BrushArena was the same. Cameron and I are doing the RL residency at Prime Intellect and we thought we should show people how different models paint.

People seemed to be enamoured with the aesthetic of the paintings but the core of it for us is that we’re very interested in a deeply effortful, collaborative future where humans and machines produce something better than what we could do by ourselves. And I appreciate that people like the outputs, because that means you like the output of a person and a machine collaborating together.

I’m not searching for utility. A brand guideline and a logo have to check certain boxes. You need a wordmark, it needs to be in this aspect ratio. None of those rules apply to me. It’s just me and a model, and we’re going to make something together that moves us, hopefully for other people or with other people.

And I think that’s a broader problem — everyone is trying to use AI purely for utility. What I love recently is the vibe coding game stuff, where people are mixing and mashing old playstation games together. Let’s use AI for fun too.

I’m making weird, odd things that don’t always serve a purpose. There’s often no utility in my work. That’s what makes it fun. You’ve got to do some things for the love of the game.

I think designers have an important role to play in the age of AI. We need to be shaping how these models are trained, helping form their sensibilities, their constraints, what they’re rewarded for, all of which is a very creative act.

What it means to be a designer is changing. That’s the thing designers are having a hard time accepting. I think engineers are held to a more deterministic standard of their work that designers sometimes are not, and that’s why the seat at the table is precarious. Does the software load? Is it fast? Does the database work well? With designers it’s like... is it delightful? Does it feel intuitive? Perhaps that’s harder to measure, but it either is or it’s not, and we need to overcome this problem somehow. Shaping models, I think, is prime surface area for bringing those qualities of work in and helping shepherd this technology.

When AI came along and it was producing slop, I think designers felt like, yes, we’re back in charge. Now the models are getting better and their outputs are getting more tasteful. When you say you’re going to shape these models to not produce slop, that’s a polarizing thing to say. But you can think about it as the layer of work changing, that we’re not being replaced but rather we’re going to be shepherding these machines into producing things that make sense. And of course taste is a moving target, but it’s the continual act of becoming, and just when you have become, the needle has moved on. But that’s ok. It’s how design and good design has always been.

Criticism is valid. I just don’t think you can sit on criticism. You have to do something about it.