Meta announced LIMA, a large-scale language model that demonstrates that pre-trained models can be used to obtain high-quality responses from a small set of prompts.
LIMA (Less Is More for Alignment) is based on LLaMa, a 65 billion parameter model provided for research purposes by a technology company in the first quarter of this year.
Mehta explains that large language models are typically trained in two phases. One is raw text unsupervised pre-training to learn common expressions, and the other is large-scale conditioning and reinforcement learning aimed at improving her AI’s learning. Tailor it to your final task and user preferences.
Using LIMA, Meta aims to demonstrate that it is possible to obtain high-quality results from a small number of symptoms using previously extensively trained models. And to do this, he used his 1,000 carefully selected examples of real-world instructions. 750 of them came from forums like his Stack Exchange and wikiHow, and the remaining 250 were written by the researchers themselves.
To analyze its performance, they compared it to OpenAI’s GPT-4, Anthropic’s Claude, and Google’s Bard in a controlled test of 300 indications. Their results show that LIMA produces the “same or preferred” response in 43, 46, and 58 percent of cases, respectively.
On an absolute scale, as gathered in a study posted on arxiv.org, LIMA responses were “88 percent meeting the immediate requirements and 50 percent saying they thought they were good.” It’s clear,” the researchers said.
Source: Diario.Elmundo
