- No Longer a Nincompoop
- Posts
- The Cheapest ChatGPT Will Ever Be đ¤
The Cheapest ChatGPT Will Ever Be đ¤
Welcome to edition #39 of the No Longer a Nincompoop with Nofil newsletter.
Hereâs the tea đľ
AI âfriendsâ on the rise đŤ
OpenAI
New free model đ
Hacked đĽď¸
Revenue đľ
AGI Progress Tracker đ
OS advancements đž
A world of virtual lovers
Things are just weird now.
If youâve been reading my newsletter for sometime, you would have seen me write about Character.ai (CAI). This website allows you to chat with fictional characters via text or call, and you can even create your own characters.
CAI was founded by Noam Shazeer, one of the authors of the famous research paper from Google, âAttention is all you needâ. It was in this paper that researchers showcased the potential of the transformer, the very architecture that now powers the most powerful AI models on the planet.
Last week, Shazeer shared a new research blog. In it, they casually detail some of the most advanced technical details regarding LLMs and using them at scale.
The key here is scale.
I mean, CAI was founded in 2021, a mere 3 years ago.
How much scale could they possibly need?
CAI does just under 1/5th of Googleâs entire search volume.
CAI serves 20,000 queries per second while Google does 105,000.
Yes. You read that right. A website that lets you chat with fictional characters does 20% of the global Google search volume. This number is only going to increase as AI gets better and they roll out more features, like voice chat.
Before releasing voice chat, they tested it among 3 million users. They combined for 20 million calls. To put this into perspective, thatâs a whopping 7 calls per user.
When was the last time you called 7 people?
I definitely canât remember the last time I didâŚ
They see ~2 Billion queries a day, the website has 250 Million visitors every month, and the actual platform has 20 Million monthly active users.
If thatâs not crazy enough, the average time spent on the website itself is almost 30 minutesâŚ
If you want to understand just how bad it really is, all you have to do is look at their subreddit. It goes ballistic anytime the site is down.
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/47467532-f7c6-49f2-b17b-a1198f97fd95/image.png?t=1721167641)
This isnât something new. There are hundereds of people talking about their addiction.
And donât think this is just lonely guys. Itâs a 50/50 split between men and women, and some even suggest that there are actually more women than men. There is no way to reliably verify this though.
Itâs no wonder Meta is also trying to create digital clones of famous celebrities.
Itâs also no wonder that literally 3 days after the CAI blog went up, reports came out that Google is also looking into deploying a similar product, although the more likely scenario is that Google simply buys CAI.
Big tech is going to wrangle every cent out of the loneliness epidemic.
OpenAI
New model
OpenAI announced their new free-tier model, GPT-4o mini. It scores 82% on the MMLU Dataset and does quite well on others. This seems like OpenAIâs answer to Claude Haiku and Gemini Flash.
I wouldnât put all my faith in benchmarks though. If youâve actually seen the kinds of questions in some datasets, this wouldnât be all that impressive. I donât take benchmarks into consideration anymore.
The model has a 128K token context window and supports 16k output tokens which is 4x the amount GPT-4o and GPT-4 can output. In an early release on the LLM Leaderboard, it ranked 4th, beating GPT-4 Turbo.
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/787e513e-b3c3-498d-9b02-4bfc333d0886/image.png?t=1721443043)
Source
But itâs the price that really sells this model.
15 cents per 1 Million input tokens.
60 cents per 1 Million output tokens.
To put into perspective how many output tokens this is, you could generate like ~2000+ pages of text for 60 centsâŚ
The bots are going to run wild.
This officially replaces GPT-3.5 in ChatGPT. If you compare the price of this model with models OpenAI released two years ago, there is a 99% reduction in cost.
The cost of intelligence is bee lining to zero.
Now, if you were hoping for the release of the voice capabilities they demonstrated a while back, I have some good news. Theyâve stated that the alpha starts later this month and the full-roll out is coming in about a months time.
Weâll see if that actually happens.
Some people have been testing GPT-4o mini and theyâve found it be unbelievably good, like, better than GPT-4o good. This is suspicous. Upon inspection, it turns the model answers questions and also gives a tonne of extra completely random and useless info to appease the benchmarks [Link]. Iâm always hesistant when I see a small model perform so well, and I suspect mini isnât as good as it may seem.
On another test, GPT-4o mini performed the same as the original GPT-3.5 [Link].
The rather interesting thing about the announcement is that it says youâll be able to provide video as input AND video output. I donât know if I trust video output, but, even video input would be quite impressive.
Hack
OpenAI was hacked April last year and it was never reported to the public or law enforcement. It wasnât a large hack, no customer data or model data was stolen, but comms between employees were comprimised.
There are constant calls for the national risk of China stealing advanced models, techniques and secrets from top American AI labs like OAI and Anthropic.
Two points:
Research is the reason everyone is here in the first place. OpenAI, Google, Anthropic, every AI lab out there, can build their models and products because research was conducted and shared with the wider public.
Researchers collaborated and worked together to synthesise new ideas and work on refining old ones.
The reality of the situation is that most of what you need to build top level AI models is in research that is already out there. Itâs simply a matter of providing the man power and resources to make it work. When it comes to China building their own advanced models, itâs not a matter of if, but when.
China has already shown how capable they are when it comes to building AI models. DeepSeek-Coder-V2 is a Chinese model that ranks 6th on the AI Leaderboard and is a very, very capable open source model.
Will China steal secrets from American AI labs?
Maybe.
As of right now, it seems like they might not even need to. Have you seen their new video generation model Kling? (itâs crazy good).
You see, AI advancement comes with a double edged sword.
On one hand, you want progress. This progress can only come with collaboration. Collaboration can only come with sharing ideas and research.
On the other hand, you donât want adversaries to gain a lead, so you feel compelled to not share secrets. But by not sharing any ideas, you slow down progress.
Here lies the problem.
Itâs very similar to regulation.
Regulation is important and is absolutely necessary.
Failing to regulate AI, even at this stage, would be very dangerous.
Regulating it would be even worse.
So what do you think, what should we do?
Should we regulate now? |
OAI recently bought on the former NSA chief onto their board. Since then, their next frontier AI model went from being just around the corner to 18 months away.
Last week it was announced that OpenAI is blocking access to their models in China. The funny thing is that Microsoft, which provides access to the same models through their cloud platform Azure, is not doing the same.
According to Microsoft, âOpenAI, being an independent company, makes its own decisionsâ. Theyâre also planning to block access in other countries like Russia, Iran and North Korea.
This probably means theyâre going to release something new soon, although, they absolutely donât need to.
According to SimilarWeb, ChatGPT is the 10th most visited site on the planet. Just look at that increase from the GPT-4o launch in May. No matter how good competitors get, OpenAI is chilling.
Also, the fact that Googleâs Gemini and Character.ai are at the same level is hilarious. So much for Googleâs distribution moat.
Revenue
OpenAIâs annualised revenue has doubled in the last six months, increasing to $3.4 Billion.
The biggest surprise to me is how little the API makes. There are tens of thousands of startups and businesses using their API, yet it only accounts for 15% of their revenue.
The sad part for Microsoft is that OpenAI makes more from selling itâs API than Microsoft does selling it on Azure.
A question I would have for someone in sales or marketing:
Is 7.7M paying customers a good ratio for being the 10th most visited site in the world?
Regardless, these are just estimates.
The funny thing is that although this is very impressive, Accenture added $3.6 Billion runrate bookings in the last quarter!
This is what you get when you employ 55,000 âAI practitionersâ. Theyâre aiming for 80k in the next two years.
Will they even need humans to do that kind of work with the type of AI models weâll have in two years?
Guess weâll find out.
Curiously, if AI capabilities keep trending the way they are, it will completely nuke consultancies (I think, could be wrong).
Then again, AI will nuke a lot of industries, not just consulting.
AI Progress Tracker
OpenAI has released a new AI âprogress trackerâ. Simply put, itâs a great marketing tool to make people think they are leading the race to AGI.
Their list goes:
Chatbots - ChatGPT etc
Reasoners - AI that can reason and problem solve like a human
Agents - systems that can take actions
Innovators - AI that can aid in inventions
Organisation - AI that can do the work of an organisation
There arenât any more details.
They say that weâre on level 2. Iâve been building products with LLMs since ChatGPT came out and I see what they mean. Iâm not sure we can classify them at human-level problem solving when it comes to math though.
Yeah, this might be a tokeniser issue, as in how ChatGPT breaks up the words and translates them into machine language, but, why does that matter?
Wrong is wrong. The average person isnât going to know or care about why itâs wrong.
There is speculation that OpenAI is releasing a lot of material like this to make it seem like they are far apart from everyone else. I agree with this. Other models, some even open source, can do much of what ChatGPT can do right now.
The general consensus right now is that Claude 3.5 Sonnet is the best AI to use right now.
The reason why theyâre doing this though, is very interesting.
Some suggest that OpenAI is planning to IPO soon.
How much do you think a share in OpenAI would be?
Side Note: This is Google DeepMindâs Levelâs of AGI Table [Link]
Working with computers, not on them
OpenAI has acquired Multi, a platform that let people remotely control computers and allows for multiplayer use of a computer. Imagine giving the ChatGPT desktop app the ability to use your computer and giving it a task.
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/a945048d-7ded-40ec-861b-664ba180b124/TqaDoMZAAs8hj8dH-ezgif.com-video-to-gif-converter.gif?t=1721264953)
âBook that meeting by emailing this personâ, and it just goes and does it.
As Iâve said before, the way we use computers will fundamentally change. Why use so many different apps when an AI abstraction can use all of them for you, a JARVIS of sorts.
Apple is already laying the groundwork for this on iPhones. I wouldnât be surprised if they were looking at doing it for Macbooks. They probably wouldnât like Macbooks being mass controlled by OpenAI.
OpenAI also acquired Rockset to power their RAG capabilities. This is actually a very interesting and strategic acquisition.
Rockset was founded by an ex-Facebook team that worked on RocksDB, a database that was built off of Googleâs own LevelsDB. The company was valued at almost half a Billion and has done a tonne of work on retrieval and search.
This is what they do.
If you want learn more about the technicals, here is a five-part series exploring about how their database works [Link] and their whitepaper [Link].
Other
In a weird 𼴠turn of events, both Apple and Microsoft have decided to leave the OpenAI board [Link]. In fact, Apple hadnât even taken up the position yet, they simply cancelled it. There is a lot of regulatory noise coming for AI and especially OpenAI which are basically the publics perception of an AI company. Big lawsuits have already begun appearing and seems like Apple and Microsoft want to steer clear. Itâs funny on Microsoftâs part because they practically own the company. Apple, on the other hand, I have no idea why they even got involved the way they did and marketed it just for hype. I wouldâve thought their brand image was worth more than that.
OpenAI inked 8 partnership deals with media companies in May and June [Link]. One of the biggest ones being with TIME, which gives OAI 100 years of data đŽ[Link]. Seems like companies are giving in to the idea of providing their data for training purposes. Only the NYT remains.
OpenAI is working with Thrive to build a personalised health coach [Link]
OpenAIâs (Microsoftâs) next compute cluster is going to have 100k GB200 chips, which will make it one of the most powerful 𦾠clusters in the world [Link].
Japan talks about their new policy for using AI in the military [Link]. With less people joining the military and their aging population, they see AI as a way to utilise personnel âmore efficientlyâ. With drones becoming a lot more common in warfare, and the advancements in robots, this wonât be a personnel issue for long, it will be a technological issue.
How was this edition? |
As always, Thanks for Reading â¤ď¸
Written by a human named Nofil
Reply