Claude 2: Anthropic's chatbot reads 100,000 tokens at a time

What's new

Anthropic released Claude 2 on July 11, 2023. It can be used through an API (a technical access point that companies can use to build their own solutions) and through the beta website claude.ai. At launch, the chatbot was open in the US and the UK, and Anthropic planned to expand to more regions. Denmark and the EU are not mentioned as covered.

The big change is length. Claude 2 can accept up to 100,000 tokens in a single prompt, or roughly 75,000 words, according to Decrypt. A token is a piece of text, often a word or part of one. Responses can also get longer: up to about 4,000 tokens, compared with roughly 512 in Claude 1.3, according to VentureBeat.

What's smart about it

With such a large "workspace," you can give the model an entire report, a contract or a book and ask for a summary or answers to questions. In Decrypt's test, Claude was the only one of the three chatbots that could process a 62,000-word Bible text, and it gave the most thorough explanation in Spanish.

Anthropic also says Claude 2 has improved at coding and math. The company further claims the model is twice as good as Claude 1.3 at giving harmless responses. Anthropic uses methods such as Constitutional AI and red-teaming (deliberately trying to make the model fail) here. Sandy Banerjee, head of GTM at Anthropic, says, according to VentureBeat, that Claude is easy to talk to, clearly explains its thinking and is less likely to give harmful responses.

Cheaper or better?

Better, according to Anthropic's own measurements, which have not been independently verified:

- Multiple choice on the US bar exam (the law licensing exam): 76.5 percent versus 73.0 for Claude 1.3. - Python coding (Codex HumanEval): 71.2 percent versus 56.0. - Math problems (GSM8K): 88.0 percent versus 85.2. - GRE (US graduate admissions test): above the 90th percentile in reading and writing and around the median in quantitative reasoning.

On price, Anthropic says the API price is the same as for Claude 1.3. The actual dollar amount is not given in the sources. According to Decrypt, the chatbot was temporarily free to use in the US and the UK. By comparison, ChatGPT Plus with GPT-4 costs 20 dollars a month.

Against its competitors, Claude 2 has by far the largest context window: 100,000 tokens versus 8,192 for ChatGPT and 8,196 for Google Bard in the free versions at the time. ChatGPT, on the other hand, has the broadest global reach, and only Bard handled tasks involving current data, because it has internet access. VentureBeat calls Claude 2 a direct challenger to OpenAI.

What it's good for

Claude 2 suits work with long documents: summaries, contract review, analysis of large texts and longer writing tasks. According to VentureBeat, Slack, Notion, Midjourney, Zoom, Robin AI, Sourcegraph and Jasper, among others, use the model, for purposes including moderation, customer support and contract analysis.

There are also clear limitations. Anthropic acknowledges that the model can still give inappropriate responses and can be fooled by jailbreaks (tricks that get it to break its own rules). It should not be used for decisions about physical or mental health, and the service was still in beta. Claude can also give wrong answers about current events, and in Decrypt's test, its proposed debt repayment plan was thorough but flawed.

Note that the launch took place in July 2023. Claude 2 is an older model, and availability and prices may have changed since.