Introducing Falcon2: Subsequent-Gen Language Mannequin by TII

Picture by Creator

The Know-how Innovation Institute (TII) in Abu Dhabi launched its subsequent sequence of Falcon language fashions on Could 14. The brand new fashions match the TII mission as expertise enablers and can be found as open-source fashions on HuggingFace. They launched two variants of the Falcon 2 fashions: Falcon-2-11B and Falcon-2-11B-VLM. The brand new VLM mannequin guarantees distinctive multi-model compatibilities that carry out on par with different open-source and closed-source fashions.

Mannequin Options and Efficiency

The latest Falcon-2 language mannequin has 11 billion parameters and is skilled on 5.5 trillion tokens from the falcon-refinedweb dataset. The newer, extra environment friendly fashions compete properly towards the Meta’s latest Llama3 mannequin with 8 billion parameters. The outcomes are summarized within the beneath desk shared by TII:

Picture by TII

As well as, the Falcon-2 mannequin fares properly towards Google’s Gemma with 7 billion parameters. Gemma-7B outperforms the Falcon-2 common efficiency by solely 0.01. As well as, the mannequin is multi-lingual, skilled on generally used languages inclduing English, French, Spanish and German amongst others.

Nonetheless, the groundbreaking achievement is the discharge of Falcon-2-11B Imaginative and prescient Language Mannequin that provides picture understanding and multi-modularity to the identical language mannequin. The image-to-text dialog functionality with comparable capabilities with latest fashions like Llama3 and Gemma is a major development.

Methods to Use the Fashions for Inference

Let’s get to the coding half so we are able to run the mannequin on our native system and generate responses. First, like every other challenge, allow us to arrange a contemporary atmosphere to keep away from dependency conflicts. Given the mannequin is launched not too long ago, we are going to the necessity the most recent variations of all libraries to keep away from lacking help and pipelines.

Create a brand new Python digital atmosphere and activate it utilizing the beneath instructions:

python -m venv venv
supply venv/bin/activate

Now we’ve a clear atmosphere, we are able to set up our required libraries and dependencies utilizing Python package deal supervisor. For this challenge, we are going to use pictures out there on the web and cargo them in Python. The requests and Pillow library are appropriate for this goal. Furthermore, for loading the mannequin, we are going to you utilize the transformers library that has inner help for HuggingFace mannequin loading and inference. We are going to use bitsandbytes, PyTorch and speed up as a mannequin loading utility and quantization.

To ease up the arrange course of, we are able to create a easy necessities textual content file as follows:

# necessities.txt
speed up  # For distributed loading
bitsandbytes	# For Quantization
torch   # Utilized by HuggingFace
transformers	# To load pipelines and fashions
Pillow  # Primary Loading and Picture Processing
requests	# Downloading picture from URL

We are able to now set up all of the dependencies in a single line utilizing:

pip set up -r necessities.txt

We are able to now begin engaged on our code to make use of the mannequin for inference. Let’s begin by loading the mannequin in our native system. The mannequin is offered on HuggingFace and the entire measurement exceeds 20GB of reminiscence. We can’t load the mannequin in client grade GPUs which normally have round 8-16GB RAM. Therefore, we might want to quantize the mannequin i.e. we are going to load the mannequin in 4-bit floating level numbers as a substitute of the same old 32-bit precision to lower the reminiscence necessities.

The bitsandbytes library offers a straightforward interface for quantization of Massive Language Fashions in HuggingFace. We are able to initalize a quantization configuration that may be handed to the mannequin. HuggingFace internally handles all required operations and units the proper precision and changes for us. The config could be set as follows:

from transformers import BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
  	# Authentic mannequin help BFloat16
    bnb_4bit_compute_dtype=torch.bfloat16,
)

This permits the mannequin to slot in underneath 16GB GPU RAM, making it simpler to load the mannequin with out offloading and distribution. We are able to now load the Falcon-2B-VLM. Being a multi-modal mannequin, we might be dealing with pictures alongside textual prompts. The LLava mannequin and pipelines are designed for this goal as they permit CLIP-based picture embeddings to be projected to language mannequin inputs. The transformers library has built-in Llava mannequin processors and pipelines. We are able to then load the mannequin as beneath:

from transformers import LlavaNextForConditionalGeneration, LlavaNextProcessor
processor = LlavaNextProcessor.from_pretrained(
	"tiiuae/falcon-11B-vlm",
	tokenizer_class="PreTrainedTokenizerFast"
)
mannequin = LlavaNextForConditionalGeneration.from_pretrained(
	"tiiuae/falcon-11B-vlm",
	quantization_config=quantization_config,
	device_map="auto"
)

We go the mannequin url from the HuggingFace mannequin card to the processor and generator. We additionally go the bitsandbytes quantization config to the generative mannequin, so will probably be robotically loaded in 4-bit precision.

We are able to now begin utilizing the mannequin to generate responses! To discover the multi-modal nature of Falcon-11B, we might want to load a picture in Python. For a check pattern, allow us to load this normal picture out there right here. To load a picture from an internet URL, we are able to use the Pillow and requests library as beneath:

from Pillow import Picture
import requests

url = "https://static.theprint.in/wp-content/uploads/2020/07/soccer.jpg"
img = Picture.open(requests.get(url, stream=True).uncooked)

The requests library downloads the picture from the URL, and the Pillow library can learn the picture from bytes to a regular picture format. Now that may have our check picture, we are able to now generate a pattern response from our mannequin.

Let’s arrange a pattern immediate template that the mannequin is delicate to.

instruction = "Write an extended paragraph about this image."
immediate = f"""Person:<picture>n{instruction} Falcon:"""

The immediate template itself is self-explanatory and we have to comply with it for greatest responses from the VLM. We go the immediate and the picture to the Llava picture processor. It internally makes use of CLIP to create a mixed embedding of the picture and the immediate.

inputs = processor(
	immediate,
	pictures=img,
	return_tensors="pt",
	padding=True
).to('cuda:0')

The returned tensor embedding acts as an enter for the generative mannequin. We go the embeddings and the transformer-based Falcon-11B mannequin generates a textual response primarily based on the picture and instruction offered initially.

We are able to generate the response utilizing the beneath code:

output = mannequin.generate(**inputs, max_new_tokens=256)
generated_captions = processor.decode(output[0], skip_special_tokens=True).strip()

There we’ve it! The generated_captions variable is a string that incorporates the generated response from the mannequin.

Outcomes

We examined numerous pictures utilizing the above code and the responses for a few of them are summarized on this picture beneath. We see that the Falcon-2 mannequin has a powerful understanding of the picture and generates legible solutions to indicate its comprehension of the eventualities within the pictures. It might learn textual content and in addition highlights the worldwide data as a complete. To summarize, the mannequin has wonderful capabilities for visible duties, and can be utilized for image-based conversations.

Picture by Creator| Inference pictures from the Web. Sources: Cats Picture, Card Picture, Soccer Picture

License and Compliance

Along with being open-source, the fashions are launched with the Apache2.0 License making them out there for Open Entry. This permits the modification and distribution of the mannequin for private and business makes use of. This implies which you can now use Falcon-2 fashions to supercharge your LLM-based functions and open-source fashions to offer multi-modal capabilities to your customers.

Wrapping Up

General, the brand new Falcon-2 fashions present promising outcomes. However that isn’t all! TII is already engaged on the following iteration to additional push efficiency. They appear to combine the Combination-of-Specialists (MoE) and different machine studying capabilities into their fashions to enhance accuracy and intelligence. If Falcon-2 looks as if an enchancment, be prepared for his or her subsequent announcement.

Kanwal Mehreen Kanwal is a machine studying engineer and a technical author with a profound ardour for information science and the intersection of AI with medication. She co-authored the e-book “Maximizing Productiveness with ChatGPT”. As a Google Technology Scholar 2022 for APAC, she champions variety and educational excellence. She’s additionally acknowledged as a Teradata Range in Tech Scholar, Mitacs Globalink Analysis Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having based FEMCodes to empower ladies in STEM fields.