ChatGPT Memory, Boundaries and AI Companions, Auren – Stratechery by Ben Thompson

Good morning,

On last Thursday’s Sharp Tech, Andrew and I discussed the then-current state of Trump’s trade war, including the history of Apple manufacturing and how and when tech built up its supply chain in Asia. Plus, at the end, I give my trip report from my vacation visit to the Japan Grand Prix.

On to the Update:

ChatGPT Memory

From Ars Technica:

OpenAI today announced a significant expansion of ChatGPT’s customization and memory capabilities. For some users, it will now be able to remember information from the full breadth of their prior conversations with it and adjust its responses based on that information. This means ChatGPT will learn more about the user over time to personalize its responses, above and beyond just a handful of key facts.

Some time ago, OpenAI added a feature called “Memory” that allowed a limited number of pieces of information to be retained and used for future responses. Users often had to specifically ask ChatGPT to remember something to trigger this, though it occasionally tried to guess at what it should remember, too. (When something was added to its memory, there was a message saying that its memory had been updated.) Users could enable or disable this feature at will, and it was automatically off for specific chats where users chose the “Temporary Chat” option—sort of ChatGPT’s version of incognito mode. The new improvements announced today go far beyond that.

Now, where there was once a checkbox in ChatGPT’s interface to disable or enable memory tracking, there are two checkboxes. “Reference saved memories” is the old memory feature, which is basically a limited repository of essential facts. The second is the new feature: “reference chat history.” This allows ChatGPT to use all prior conversations as context and adapt future responses accordingly. Unlike the older saved memories feature, the information saved via the chat history memory feature is not accessible or tweakable. It’s either on or it’s not.

There are expected to be a host of OpenAI announcements this week, including GPT-4.1, but I think memory is much more interesting. The feature seems pretty basic: OpenAI didn’t release any details about how it works, but it seems to be some sort of RAG search over structured summaries of previous chats. It also seems to have been building up those summaries since mid-February or so; I couldn’t trigger any sort of recollection of conversations before then.

Basic, though, is fine, given the obvious utility: while there are certainly downsides to your AI getting over-indexed on your questions, the gain is in how I described the downside: your AI. ChatGPT has now imposed a cost on every conversation that happens with any other chatbot, which is that that conversation will not be in memory. This is a pretty basic moat — to go along with the basicness of the feature — but that doesn’t mean it won’t be effective.

It was also all but announced by Sam Altman in a Stratechery Interview last month:

Where I think there’s strategic edges, there’s building the giant Internet company. I think that should be a combination of several different key services. There’s probably three or four things on the order of ChatGPT, and you’ll want to buy one bundled subscription of all of those. You’ll want to be able to sign in with your personal AI that’s gotten to know you over your life, over your years to other services and use it there. There will be, I think, amazing new kinds of devices that are optimized for how you use an AGI. There will be new kinds of web browsers, there’ll be that whole cluster, someone is just going to build the valuable products around AI. So that’s one thing.

There’s another thing, which is the inference stack, so how you make the cheapest, most abundant inference. Chips, data centers, energy, there’ll be some interesting financial engineering to do, there’s all of that. And then the third thing is there will be just actually doing the best research and producing the best models. I think that is the triumvirate of value, but most models except the very, very leading edge, I think will commoditize pretty quickly.

ChatGPT memory is a part of that first thing, and it’s an interesting twist on the question of identity. The value of owning identity is something I’ve talked a lot about over the years, both in the consumer and enterprise contexts, but what is interesting about OpenAI’s gambit here is that the identity they are seeking to own is not your identity but rather the identity of your AI.

Boundaries and AI Companions

This is a subtle difference, but I think it’s a profound one. Consider this objection from Ethan Mollick:

I could not agree more in the context of personal identity. In 2020’s Social Networking 2.0 I explained why I maintained multiple Twitter accounts:

Identities — plural — referred to the many users of Twitter, but a second thing that is interesting about my Twitter group is that @benthompson is not a member; my alter-ego, @notechben is. I created that account — which, I will tell you right now, is pretty annoying to follow — so that I could tweet freely during basketball games without losing followers from my primary Twitter account. After all, just because you like my takes on tech, it does not necessarily follow that you like my takes on sports.

What I increasingly realize, though, is that separating my identities on Twitter does not mean a lesser experience, but a far superior one; social interaction in any medium is always a balance between self-expression and the accommodation of others, which means that in the analog world it is a constant struggle to strike a balance between being myself and annoying everyone around me at some point or another. The magic of the Internet, though, is that you can be whatever you want to be.

In this view group chats were a defining product of social networking 2.0:

This is where messaging is a much more natural fit, and, as far as the depth of your network is concerned, messaging services are just as much a threat to v1 social networks’ connectivity as TikTok is to Facebook’s hold on attention: I can simultaneously be a Bucks fan with the Fiefdom, be a tech enthusiast with my Slack group, explore ideas with my WhatsApp group, and talk politics with my trusted friends. The fact that I am not my whole self in any of these groups is a feature, not a bug, and one that is uniquely made possible by digital.

Even group chats, however, have the key limiting factor of traditional social networks: other people. The good thing is that those other people are, presumably, trustworthy; the bad thing — and not bad in a normative sense, but rather in terms of necessitating thought and consideration — is that you still need to maintain some sort of inner guardrail in terms of what you discuss, what your relative relationship with other members of the group are, inescapable status questions, fears of looking dumb, etc.

This illuminates one of the biggest potential consumer use cases of AI: it’s the “perfect” companion. I put “perfect” in quotes because I am skeptical that an AI can truly meet one’s emotional needs like another human can; then again, that is why perfect is the right word choice, because a human can never be perfect (that’s just another way of making the same observation). An AI companion is focused only on you, or better, focused on the same things you are focused on; it is an extension of yourself, an advocate or councilor in all of your dealings, not a counterparty. I made this drawing for that Social Networking 2.0 Article:

Humans are in those colored ovals; AI is the only entity — other than myself — that can plausibly live exactly in the middle: it’s precisely because it is not human that you can have a conversation about all the different aspects of your life. Mollick is right that boundaries are good, but you can make the case that when it comes to interactions with others the question of boundaries — or violating them — is more about recognizing an indelible reality; it’s not necessarily the case that said reality applies to AI, and there are likely to be a lot of interesting use cases that flow from that fact. If, of course, you always use the same AI.

Auren

I tried out an interesting new chat app over the last few weeks called Auren; the introductory thread from X user @nearcyan is here. Auren is compelling in that instead of trying to build something scalable it quite deliberately is very expensive — $20/month for 2500 messages — in an attempt to build the best possible chat companion.

I found Auren pretty impressive (although I immediately chose to interact with Soren, the more aggressive and challenging alter-ego), although I never lost the sense that I was in fact interacting with an AI, not a person (which again, is arguably a feature, not a bug). I tried to use it according to its intended purpose, like some sort of lifecoach or therapist, and I thought Auren/Soren did a good job of making me think about a particular decision in different ways.

Now I do have a shortcut in my interactions with new AIs: given that I am a public figure with lots of information on the Internet, simply telling AIs who I am gives them a lot of context; even so, there is still the therapist challenge, which is spending a lot of time describing all of the various details surrounding the question at hand, even before you get to the interesting parts of the discussion.

To that end, I was reminded of Altman’s defense of the OpenAI API in our interview:

SA: I really believe in this product suite thing I was just saying. I think that if we execute really well, five years from now, we have a handful of multi-billion user products, small handful and then we have this idea that you sign in with your OpenAI account to anybody else that wants to integrate the API, and you can take your bundle of credits and your customized model and everything else anywhere you want to go. And I think that’s a key part of us really being a great platform…

I don’t think we’ll be a platform in a way that an operating system is a platform. But I think in the same way that Google is not really a platform, but people use sign in with Google and people take their Google stuff around the web and that’s part of the Google experience, I think we’ll be a platform in that way.

The carry around the sign-in, that’s carrying around your memory and who you are and your preferences and all that sort of thing.

SA: Yeah.

So you’re just going to sit on top of everyone then and will they be able to have multiple sign-ins and the OpenAI sign in is going to be better because it has your memory attached to it? Or is it a, if you want to use our API, you use our sign-in?

SA: No, no, no. It’d be optional, of course.

Altman’s “No, no, no” was referring to the requirement that you must use OpenAI’s sign-in with their API; the interesting part is the idea of knowing that if you sign in with OpenAI, you will get your AI’s identity with the product you are using. Well, to the point above, it would have been handy to bring that context to Auren! And then, presumably, the details of our conversation could then flow the other way to my AI identity.

Anyhow, this is still all quite basic, and there are a lot of details to be worked out, particularly around privacy; at the same time, you can also surely see why there is a potentially fascinating advertising play here (which, admittedly, makes the privacy questions even more pressing). Ultimately, though, the final takeaway is that nothing in this Update was really about model quality: memory is a product feature; by extension, moats are often downstream from product choices. OpenAI, to that end, is the accidental consumer tech company, so this is — along with models, of course — exactly what they should be working on.

This Update will be available as a podcast later today. To receive it in your podcast player, visit Stratechery.

The Stratechery Update is intended for a single recipient, but occasional forwarding is totally fine! If you would like to order multiple subscriptions for your team with a group discount (minimum 5), please contact me directly.

Thanks for being a subscriber, and have a great day!