“Fractured Skies”, Nilima Shah. Exhibited at MoMA from September 27th. This exhibition will have her ruminations on Kashmir - an Indian state riddled with violence, displacement, loss, and the enduring power of human connection.
My motivations have been weathering a storm this past year, owing to the general post-teenage-tradeoff of “what I am good at” versus “what I believe to be important”. I began this year, with a gallant quantum-computer-shaped puff in my chest, ready to work on quantum algorithms and complexity theory. I realised later my universe was not written by Frank Herbert, and we were understanding “intelligence” before interstellar travel. I decided to work on human-AI interactions, how they could disturb the social structures we have grown. Quickly after, my close friend urged me to publish in machine learning after discussing with me his work. A few raggedy papers and a research fellowship later, METR released findings from an investigation into “rogue” AI behaviour, urging me to reconsider my work. I write this post after a weeklong delirium induced by rationalist thought experiments, OpenAI’s twitter handle, and papers from 2017.
This blogpost was not written by, proofread by, ideated via, or polished through LLMs. As a result, it may contain typos, expletives, and perhaps unsavoury references to internet culture. I urge the reader(s) to think of these artifacts as features, not bugs :D
Am I a crackpot?
When you live in a country which is far behind the frontier (but close enough to try and emulate it), it is rational to distrust news coming from the west. AI safety incidents from the west landed in my inbox around the same time as my peers. We work in some of the only India-native machine learning firms as researchers and engineers. We never had these problems, nowhere close to it - in fact, we’re still kind of dealing with our models being dumber than us. This tension between news we hear and the models we work with was one of the fulcrums of my muddled thought process - “Am I a crackpot? Am I a target for some elaborate marketing scheme? Is my internet dead1? I’d really like it to be alive…”.
I decided to shake these evidently adolescent thoughts off my head by reading up things I had very high confidence in being human generated (I did not want to deal with the fight-or-flight response whenever I caught an em-dash in a research paper), and so I looked up papers written before LLMs were widely adopted as thought-conduits.
I don’t mean to be a fear-mongering hysteric, but what I read wasn’t funny or interesting. What follows is a brief writeup of my opinions, fears, and aspirations that stem from these readings.
Roko, your basilisk is a stretch!
Soon after I came across The rocket alignment problem, I reached out to a friend of mine to discuss it. They shrugged me off, and said (verbatim) - “Bro just discovered 2018”. Not very long after, I found myself reading up on the canon for alignment - Roko’s basilisk, Grimes, and a sneaky re-read of The Last Question, and a few types of inductions. All of these reads are admittedly more existential and “foundational”, than a person like me2 would like. Lucky me, I also tend to love well written stuff. I believe that a lot of the reasons why people get interested in things, and pursue them, is in order to tell a story thats never been told before. I believe that Man is a Storyteller - and in the pursuit of these Stories we find seafarers and scientists, but what do I know.
Post-METR report Nishit is a different version of pre-METR Nishit. I have been reading sci-fi for almost as long back as I can remember reading, and maybe that is what pre-disposes me to stand in awe of the times we live in. I have played absolutely no hand to play in the state of AI today, but it is awesome. We have come up with so many egotistical definitions of self - be it cognition, intelligence, adaptation, etcetera, over Man’s introspective past. And now we have systems which are checking boxes in that list. It is wonderful. It is also the first chapter of most science fiction books where things go very wrong /s3.
I am not very sure of where this blog is headed. I expect to return to this blog and update it often, as I mature. Hopefully a super-intelligent model doesn’t kill me for anthropomorphising it in the meantime.
Footnotes
-
[1] The dead Internet theory is a concept that asserts that the Internet consists primarily of bot activity and automated content manipulated by algorithmic curation. Originally conceived as a conspiracy theory alleging that the phenomenon is a coordinated effort to control the population and reduce genuine human interaction, the concept is also employed colloquially to describe the impacts of generative AI and emphasize only the core observations without speculating on the driving forces ↩
-
[2] Trained on physics, validated on workshop paper admits, and paid for capabilities research. ↩
-
[3] Tone indicator for sarcasm - the preceding sentence need not be taken seriously, it is a joke. ↩