Rakuten Unveils Open Source Japanese LLMs

E-commerce major Rakuten has released high-performance open-source Japanese large language models to the public. This includes a foundation model Rakuten AI 7B and an instruct model known as Rakuten AI 7B Instruct, both achieving top scores among open Japanese LLMs in LM Evaluation Harness benchmarks.

Additionally, Rakuten has unveiled Rakuten AI 7B Chat, a conversational text generation model based on the instruct model and fine-tuned on chat data.

“At Rakuten, we want to leverage the best tools to solve our customers’ problems. We have a broad portfolio of tools, including proprietary models and our own data science and machine learning models developed over the years. This enables us to provide the most suitable tool for each use case in terms of cost, quality and performance,” says Ting Cai, Chief Data Officer of Rakuten Group.

“With Rakuten AI 7B, we have reached an important milestone in performance and are excited to share our learnings with the open-source community and accelerate the development of Japanese language LLMs.”

Key features

Rakuten AI 7B, a 7 billion-parameter LLM was developed by continuously training Mistral-7B-v0.1, an open LLM from Mistral AI, on Rakuten's GPU cluster. The models are released under the Apache 2.0 license and can be accessed from Rakuten's official Hugging Face repository.

These LLMs boast high performance and efficiency due to training on quality data and utilizing an extended tokenizer optimized for Japanese characters. In LM Evaluation Harness tests, the foundation model achieved an average score of 69.8 points for Japanese and 60.5 points for English, while the instruct model scored 77.3 and 61.3 points, respectively, positioning them as leading open Japanese LLMs.

The models are commercially applicable for various text generation tasks and can serve as a foundation for building new models. Rakuten emphasizes its dedication to AI development through its “AI-nization” initiative, aiming to integrate AI across its operations to drive growth while ensuring it contributes positively to society.