The Cheapest ChatGPT Will Ever Be 🤑

Welcome to edition #39 of the No Longer a Nincompoop with Nofil newsletter.

Here’s the tea 🍵

• AI “friends” on the rise 🫂 

• OpenAI

  • New free model 🆕 

  • Hacked 🖥️ 

  • Revenue 💵 

  • AGI Progress Tracker 📊 

  • OS advancements 💾 

A world of virtual lovers

Things are just weird now.

If you’ve been reading my newsletter for sometime, you would have seen me write about Character.ai (CAI). This website allows you to chat with fictional characters via text or call, and you can even create your own characters.

CAI was founded by Noam Shazeer, one of the authors of the famous research paper from Google, “Attention is all you need”. It was in this paper that researchers showcased the potential of the transformer, the very architecture that now powers the most powerful AI models on the planet.

Last week, Shazeer shared a new research blog. In it, they casually detail some of the most advanced technical details regarding LLMs and using them at scale.

The key here is scale.

I mean, CAI was founded in 2021, a mere 3 years ago.

How much scale could they possibly need?

CAI does just under 1/5th of Google’s entire search volume.

CAI serves 20,000 queries per second while Google does 105,000.

Yes. You read that right. A website that lets you chat with fictional characters does 20% of the global Google search volume. This number is only going to increase as AI gets better and they roll out more features, like voice chat.

Before releasing voice chat, they tested it among 3 million users. They combined for 20 million calls. To put this into perspective, that’s a whopping 7 calls per user.

When was the last time you called 7 people?

I definitely can’t remember the last time I did…

They see ~2 Billion queries a day, the website has 250 Million visitors every month, and the actual platform has 20 Million monthly active users.

If that’s not crazy enough, the average time spent on the website itself is almost 30 minutes…

If you want to understand just how bad it really is, all you have to do is look at their subreddit. It goes ballistic anytime the site is down.

This isn’t something new. There are hundereds of people talking about their addiction.

And don’t think this is just lonely guys. It’s a 50/50 split between men and women, and some even suggest that there are actually more women than men. There is no way to reliably verify this though.

It’s no wonder Meta is also trying to create digital clones of famous celebrities.

It’s also no wonder that literally 3 days after the CAI blog went up, reports came out that Google is also looking into deploying a similar product, although the more likely scenario is that Google simply buys CAI.

Big tech is going to wrangle every cent out of the loneliness epidemic.

OpenAI

New model

OpenAI announced their new free-tier model, GPT-4o mini. It scores 82% on the MMLU Dataset and does quite well on others. This seems like OpenAI’s answer to Claude Haiku and Gemini Flash.

I wouldn’t put all my faith in benchmarks though. If you’ve actually seen the kinds of questions in some datasets, this wouldn’t be all that impressive. I don’t take benchmarks into consideration anymore.

The model has a 128K token context window and supports 16k output tokens which is 4x the amount GPT-4o and GPT-4 can output. In an early release on the LLM Leaderboard, it ranked 4th, beating GPT-4 Turbo.

But it’s the price that really sells this model.

15 cents per 1 Million input tokens.

60 cents per 1 Million output tokens.

To put into perspective how many output tokens this is, you could generate like ~2000+ pages of text for 60 cents…

The bots are going to run wild.

This officially replaces GPT-3.5 in ChatGPT. If you compare the price of this model with models OpenAI released two years ago, there is a 99% reduction in cost.

The cost of intelligence is bee lining to zero.

Now, if you were hoping for the release of the voice capabilities they demonstrated a while back, I have some good news. They’ve stated that the alpha starts later this month and the full-roll out is coming in about a months time.

We’ll see if that actually happens.

• Some people have been testing GPT-4o mini and they’ve found it be unbelievably good, like, better than GPT-4o good. This is suspicous. Upon inspection, it turns the model answers questions and also gives a tonne of extra completely random and useless info to appease the benchmarks [Link]. I’m always hesistant when I see a small model perform so well, and I suspect mini isn’t as good as it may seem.

• On another test, GPT-4o mini performed the same as the original GPT-3.5 [Link].

• The rather interesting thing about the announcement is that it says you’ll be able to provide video as input AND video output. I don’t know if I trust video output, but, even video input would be quite impressive.

Hack

OpenAI was hacked April last year and it was never reported to the public or law enforcement. It wasn’t a large hack, no customer data or model data was stolen, but comms between employees were comprimised.

There are constant calls for the national risk of China stealing advanced models, techniques and secrets from top American AI labs like OAI and Anthropic.

Two points:

Research is the reason everyone is here in the first place. OpenAI, Google, Anthropic, every AI lab out there, can build their models and products because research was conducted and shared with the wider public.

Researchers collaborated and worked together to synthesise new ideas and work on refining old ones.

The reality of the situation is that most of what you need to build top level AI models is in research that is already out there. It’s simply a matter of providing the man power and resources to make it work. When it comes to China building their own advanced models, it’s not a matter of if, but when.

China has already shown how capable they are when it comes to building AI models. DeepSeek-Coder-V2 is a Chinese model that ranks 6th on the AI Leaderboard and is a very, very capable open source model.

Will China steal secrets from American AI labs?

Maybe.

As of right now, it seems like they might not even need to. Have you seen their new video generation model Kling? (it’s crazy good).

You see, AI advancement comes with a double edged sword.

On one hand, you want progress. This progress can only come with collaboration. Collaboration can only come with sharing ideas and research.

On the other hand, you don’t want adversaries to gain a lead, so you feel compelled to not share secrets. But by not sharing any ideas, you slow down progress.

Here lies the problem.

It’s very similar to regulation.

Regulation is important and is absolutely necessary.

Failing to regulate AI, even at this stage, would be very dangerous.

Regulating it would be even worse.

So what do you think, what should we do?

OAI recently bought on the former NSA chief onto their board. Since then, their next frontier AI model went from being just around the corner to 18 months away.

Last week it was announced that OpenAI is blocking access to their models in China. The funny thing is that Microsoft, which provides access to the same models through their cloud platform Azure, is not doing the same.

According to Microsoft, “OpenAI, being an independent company, makes its own decisions”. They’re also planning to block access in other countries like Russia, Iran and North Korea.

This probably means they’re going to release something new soon, although, they absolutely don’t need to.

According to SimilarWeb, ChatGPT is the 10th most visited site on the planet. Just look at that increase from the GPT-4o launch in May. No matter how good competitors get, OpenAI is chilling.

Also, the fact that Google’s Gemini and Character.ai are at the same level is hilarious. So much for Google’s distribution moat.

Revenue

OpenAI’s annualised revenue has doubled in the last six months, increasing to $3.4 Billion.

The biggest surprise to me is how little the API makes. There are tens of thousands of startups and businesses using their API, yet it only accounts for 15% of their revenue.

The sad part for Microsoft is that OpenAI makes more from selling it’s API than Microsoft does selling it on Azure.

A question I would have for someone in sales or marketing:

Is 7.7M paying customers a good ratio for being the 10th most visited site in the world?

Regardless, these are just estimates.

The funny thing is that although this is very impressive, Accenture added $3.6 Billion runrate bookings in the last quarter!

This is what you get when you employ 55,000 “AI practitioners”. They’re aiming for 80k in the next two years.

Will they even need humans to do that kind of work with the type of AI models we’ll have in two years?

Guess we’ll find out.

Curiously, if AI capabilities keep trending the way they are, it will completely nuke consultancies (I think, could be wrong).

Then again, AI will nuke a lot of industries, not just consulting.

AI Progress Tracker

OpenAI has released a new AI “progress tracker”. Simply put, it’s a great marketing tool to make people think they are leading the race to AGI.

Their list goes:

1. Chatbots - ChatGPT etc

2. Reasoners - AI that can reason and problem solve like a human

3. Agents - systems that can take actions

4. Innovators - AI that can aid in inventions

5. Organisation - AI that can do the work of an organisation

There aren’t any more details.

They say that we’re on level 2. I’ve been building products with LLMs since ChatGPT came out and I see what they mean. I’m not sure we can classify them at human-level problem solving when it comes to math though.

Yeah, this might be a tokeniser issue, as in how ChatGPT breaks up the words and translates them into machine language, but, why does that matter?

Wrong is wrong. The average person isn’t going to know or care about why it’s wrong.

There is speculation that OpenAI is releasing a lot of material like this to make it seem like they are far apart from everyone else. I agree with this. Other models, some even open source, can do much of what ChatGPT can do right now.

The general consensus right now is that Claude 3.5 Sonnet is the best AI to use right now.

The reason why they’re doing this though, is very interesting.

Some suggest that OpenAI is planning to IPO soon.

How much do you think a share in OpenAI would be?

Side Note: This is Google DeepMind’s Level’s of AGI Table [Link]

Working with computers, not on them

OpenAI has acquired Multi, a platform that let people remotely control computers and allows for multiplayer use of a computer. Imagine giving the ChatGPT desktop app the ability to use your computer and giving it a task.

“Book that meeting by emailing this person”, and it just goes and does it.

As I’ve said before, the way we use computers will fundamentally change. Why use so many different apps when an AI abstraction can use all of them for you, a JARVIS of sorts.

Apple is already laying the groundwork for this on iPhones. I wouldn’t be surprised if they were looking at doing it for Macbooks. They probably wouldn’t like Macbooks being mass controlled by OpenAI.

OpenAI also acquired Rockset to power their RAG capabilities. This is actually a very interesting and strategic acquisition.

Rockset was founded by an ex-Facebook team that worked on RocksDB, a database that was built off of Google’s own LevelsDB. The company was valued at almost half a Billion and has done a tonne of work on retrieval and search.

This is what they do.

If you want learn more about the technicals, here is a five-part series exploring about how their database works [Link] and their whitepaper [Link].

Other

• In a weird 🥴 turn of events, both Apple and Microsoft have decided to leave the OpenAI board [Link]. In fact, Apple hadn’t even taken up the position yet, they simply cancelled it. There is a lot of regulatory noise coming for AI and especially OpenAI which are basically the publics perception of an AI company. Big lawsuits have already begun appearing and seems like Apple and Microsoft want to steer clear. It’s funny on Microsoft’s part because they practically own the company. Apple, on the other hand, I have no idea why they even got involved the way they did and marketed it just for hype. I would’ve thought their brand image was worth more than that.

• OpenAI inked 8 partnership deals with media companies in May and June [Link]. One of the biggest ones being with TIME, which gives OAI 100 years of data 😮[Link]. Seems like companies are giving in to the idea of providing their data for training purposes. Only the NYT remains.

• OpenAI is working with Thrive to build a personalised health coach [Link]

• OpenAI’s (Microsoft’s) next compute cluster is going to have 100k GB200 chips, which will make it one of the most powerful 🦾 clusters in the world [Link].

• Japan talks about their new policy for using AI in the military [Link]. With less people joining the military and their aging population, they see AI as a way to utilise personnel “more efficiently”. With drones becoming a lot more common in warfare, and the advancements in robots, this won’t be a personnel issue for long, it will be a technological issue.

As always, Thanks for Reading ❤️

Written by a human named Nofil