Vicuna: An In-Depth Look
Overview
Vicuna-13B is an open-source chatbot that has been making waves in the world of AI chatbots. It was created by fine-tuning a LLaMA base model using approximately 70K user-shared conversations gathered from ShareGPT.com. This process involved converting HTML back to markdown and filtering out inappropriate or low-quality samples. Additionally, steps were taken to handle multi-turn conversations and long sequences effectively.
The training was carried out with PyTorch FSDP on 8 A100 GPUs in just one day. And for serving the demo, a lightweight distributed serving system was implemented, which can work well with cheaper spot instances from multiple clouds to reduce serving costs.
Core Features
One of the notable features of Vicuna is its ability to generate more detailed and well-structured answers compared to some other models like Alpaca. This was discovered after fine-tuning it with 70K user-shared ChatGPT conversations.
It also has enhanced capabilities in handling long context. The max context length was expanded from 512 in alpaca to 2048, which, although it increased GPU memory requirements, was managed through techniques like gradient checkpointing and flash attention.
Basic Usage
To evaluate Vicuna's performance, a set of 80 diverse questions was created and GPT-4 was utilized to judge the model outputs. By comparing the outputs of different models like LLaMA, Alpaca, ChatGPT, and Vicuna, it was found that GPT-4 prefers Vicuna over state-of-the-art open-source models (LLaMA, Alpaca) in more than 90% of the questions, and it achieves competitive performance against proprietary models (ChatGPT, Bard).
However, it's important to note that evaluating chatbots is not a straightforward task. While Vicuna shows great promise, it still has limitations. For instance, it's not as good at tasks involving reasoning or mathematics, and it may have some issues with accurately identifying itself or ensuring the factual accuracy of its outputs.
In conclusion, Vicuna is an exciting development in the field of open-source chatbots. It offers a lot of potential for users looking for an intelligent chatbot option, but like any AI tool, it has areas that need further improvement and exploration.
