A specialized version of the Llama 2 large language model called Code Llama has been developed specifically for writing and discussing code.
Facebook parent company Meta has introduced an AI-based tool for coding, called Code Llama.
From the Meta AI website, developers can request access to Code Llama. Code Llama is being made available in three sizes with 7B, 13B, and 34B parameters, all of which are free for research and commercial use. 500B code tokens and data pertaining to code are used to train each model. The fill-in-the-middle (FIM) capability, which allows for the insertion of code into already existing code, has been trained into the 7B and 13B base and instruct models. This includes out-of-the-box support for duties like code completion.
The three models address different serving and latency requirements, with the 7B model, for example, served on a single GPU while the 34B model returns the best results and allows for better coding assistance. Meta has fine-tuned two additional variations of the tool: Code Llama – Python was further fine-tuned on 100B tokens of Python code, and Code Llama – Instruct was fine-tuned to understand natural language instructions.