Ferret-UI-Llama8b

326
68
8.0B
ferret_llama
by
jadechoghari
Image Model
OTHER
8B params
New
326 downloads
Early-stage
Try on Hugging FaceAdd to Compare
Edge AI:
Mobile
Laptop
Server
18GB+ RAM
Mobile
Laptop
Server
Quick Summary

Ferret-UI is the first UI-centric multimodal large language model (MLLM) designed for referring, grounding, and reasoning tasks.

Device Compatibility

Mobile
4-6GB RAM
Laptop
16GB RAM
Server
GPU
Minimum Recommended
8GB+ RAM

Training Data Analysis

🟡 Average (4.8/10)

Researched training datasets used by Ferret-UI-Llama8b with quality assessment

Specialized For

general
science
multilingual
reasoning

Training Datasets (4)

common crawl
🔴 2.5/10
general
science
Key Strengths
  • •Scale and Accessibility: At 9.5+ petabytes, Common Crawl provides unprecedented scale for training d...
  • •Diversity: The dataset captures billions of web pages across multiple domains and content types, ena...
  • •Comprehensive Coverage: Despite limitations, Common Crawl attempts to represent the broader web acro...
Considerations
  • •Biased Coverage: The crawling process prioritizes frequently linked domains, making content from dig...
  • •Large-Scale Problematic Content: Contains significant amounts of hate speech, pornography, violent c...
c4
🔵 6/10
general
multilingual
Key Strengths
  • •Scale and Accessibility: 750GB of publicly available, filtered text
  • •Systematic Filtering: Documented heuristics enable reproducibility
  • •Language Diversity: Despite English-only, captures diverse writing styles
Considerations
  • •English-Only: Limits multilingual applications
  • •Filtering Limitations: Offensive content and low-quality text remain despite filtering
wikipedia
🟡 5/10
science
multilingual
Key Strengths
  • •High-Quality Content: Wikipedia articles are subject to community review, fact-checking, and citatio...
  • •Multilingual Coverage: Available in 300+ languages, enabling training of models that understand and ...
  • •Structured Knowledge: Articles follow consistent formatting with clear sections, allowing models to ...
Considerations
  • •Language Inequality: Low-resource language editions have significantly lower quality, fewer articles...
  • •Biased Coverage: Reflects biases in contributor demographics; topics related to Western culture and ...
arxiv
🟡 5.5/10
science
reasoning
Key Strengths
  • •Scientific Authority: Peer-reviewed content from established repository
  • •Domain-Specific: Specialized vocabulary and concepts
  • •Mathematical Content: Includes complex equations and notation
Considerations
  • •Specialized: Primarily technical and mathematical content
  • •English-Heavy: Predominantly English-language papers

Explore our comprehensive training dataset analysis

View All Datasets

Code Examples

Usage:python
from inference import inference_and_run
image_path = "appstore_reminders.png"
prompt = "Describe the image in details"

# Call the function without a box
inference_text = inference_and_run(image_path, prompt)

print("Inference Text:", inference_text)
Usage:python
from inference import inference_and_run
image_path = "appstore_reminders.png"
prompt = "Describe the image in details"

# Call the function without a box
inference_text = inference_and_run(image_path, prompt)

print("Inference Text:", inference_text)
Usage:python
from inference import inference_and_run
image_path = "appstore_reminders.png"
prompt = "Describe the image in details"

# Call the function without a box
inference_text = inference_and_run(image_path, prompt)

print("Inference Text:", inference_text)
Usage:python
from inference import inference_and_run
image_path = "appstore_reminders.png"
prompt = "Describe the image in details"

# Call the function without a box
inference_text = inference_and_run(image_path, prompt)

print("Inference Text:", inference_text)
Usage:python
from inference import inference_and_run
image_path = "appstore_reminders.png"
prompt = "Describe the image in details"

# Call the function without a box
inference_text = inference_and_run(image_path, prompt)

print("Inference Text:", inference_text)
Usage:python
from inference import inference_and_run
image_path = "appstore_reminders.png"
prompt = "Describe the image in details"

# Call the function without a box
inference_text = inference_and_run(image_path, prompt)

print("Inference Text:", inference_text)
Usage:python
from inference import inference_and_run
image_path = "appstore_reminders.png"
prompt = "Describe the image in details"

# Call the function without a box
inference_text = inference_and_run(image_path, prompt)

print("Inference Text:", inference_text)
Usage:python
from inference import inference_and_run
image_path = "appstore_reminders.png"
prompt = "Describe the image in details"

# Call the function without a box
inference_text = inference_and_run(image_path, prompt)

print("Inference Text:", inference_text)
Call the function without a boxpython
# Task with bounding boxes
image_path = "appstore_reminders.png"
prompt = "What's inside the selected region?"
box = [189, 906, 404, 970]

inference_text = inference_and_run(
    image_path=image_path, 
    prompt=prompt, 
    conv_mode="ferret_llama_3", 
    model_path="jadechoghari/Ferret-UI-Llama8b", 
    box=box
)

print("Inference Text:", inference_text)
Call the function without a boxpython
# Task with bounding boxes
image_path = "appstore_reminders.png"
prompt = "What's inside the selected region?"
box = [189, 906, 404, 970]

inference_text = inference_and_run(
    image_path=image_path, 
    prompt=prompt, 
    conv_mode="ferret_llama_3", 
    model_path="jadechoghari/Ferret-UI-Llama8b", 
    box=box
)

print("Inference Text:", inference_text)
Call the function without a boxpython
# Task with bounding boxes
image_path = "appstore_reminders.png"
prompt = "What's inside the selected region?"
box = [189, 906, 404, 970]

inference_text = inference_and_run(
    image_path=image_path, 
    prompt=prompt, 
    conv_mode="ferret_llama_3", 
    model_path="jadechoghari/Ferret-UI-Llama8b", 
    box=box
)

print("Inference Text:", inference_text)
Call the function without a boxpython
# Task with bounding boxes
image_path = "appstore_reminders.png"
prompt = "What's inside the selected region?"
box = [189, 906, 404, 970]

inference_text = inference_and_run(
    image_path=image_path, 
    prompt=prompt, 
    conv_mode="ferret_llama_3", 
    model_path="jadechoghari/Ferret-UI-Llama8b", 
    box=box
)

print("Inference Text:", inference_text)
Call the function without a boxpython
# Task with bounding boxes
image_path = "appstore_reminders.png"
prompt = "What's inside the selected region?"
box = [189, 906, 404, 970]

inference_text = inference_and_run(
    image_path=image_path, 
    prompt=prompt, 
    conv_mode="ferret_llama_3", 
    model_path="jadechoghari/Ferret-UI-Llama8b", 
    box=box
)

print("Inference Text:", inference_text)
Call the function without a boxpython
# Task with bounding boxes
image_path = "appstore_reminders.png"
prompt = "What's inside the selected region?"
box = [189, 906, 404, 970]

inference_text = inference_and_run(
    image_path=image_path, 
    prompt=prompt, 
    conv_mode="ferret_llama_3", 
    model_path="jadechoghari/Ferret-UI-Llama8b", 
    box=box
)

print("Inference Text:", inference_text)
Call the function without a boxpython
# Task with bounding boxes
image_path = "appstore_reminders.png"
prompt = "What's inside the selected region?"
box = [189, 906, 404, 970]

inference_text = inference_and_run(
    image_path=image_path, 
    prompt=prompt, 
    conv_mode="ferret_llama_3", 
    model_path="jadechoghari/Ferret-UI-Llama8b", 
    box=box
)

print("Inference Text:", inference_text)
Call the function without a boxpython
# Task with bounding boxes
image_path = "appstore_reminders.png"
prompt = "What's inside the selected region?"
box = [189, 906, 404, 970]

inference_text = inference_and_run(
    image_path=image_path, 
    prompt=prompt, 
    conv_mode="ferret_llama_3", 
    model_path="jadechoghari/Ferret-UI-Llama8b", 
    box=box
)

print("Inference Text:", inference_text)
GROUNDING PROMPTSpython
# GROUNDING PROMPTS
GROUNDING_TEMPLATES = [
	'\nProvide the bounding boxes of the mentioned objects.',
 	'\nInclude the coordinates for each mentioned object.',
	'\nLocate the objects with their coordinates.',
	'\nAnswer in [x1, y1, x2, y2] format.',
	'\nMention the objects and their locations using the format [x1, y1, x2, y2].',
	'\nDraw boxes around the mentioned objects.',
	'\nUse boxes to show where each thing is.',
	'\nTell me where the objects are with coordinates.',
	'\nList where each object is with boxes.',
	'\nShow me the regions with boxes.'
]
GROUNDING PROMPTSpython
# GROUNDING PROMPTS
GROUNDING_TEMPLATES = [
	'\nProvide the bounding boxes of the mentioned objects.',
 	'\nInclude the coordinates for each mentioned object.',
	'\nLocate the objects with their coordinates.',
	'\nAnswer in [x1, y1, x2, y2] format.',
	'\nMention the objects and their locations using the format [x1, y1, x2, y2].',
	'\nDraw boxes around the mentioned objects.',
	'\nUse boxes to show where each thing is.',
	'\nTell me where the objects are with coordinates.',
	'\nList where each object is with boxes.',
	'\nShow me the regions with boxes.'
]
GROUNDING PROMPTSpython
# GROUNDING PROMPTS
GROUNDING_TEMPLATES = [
	'\nProvide the bounding boxes of the mentioned objects.',
 	'\nInclude the coordinates for each mentioned object.',
	'\nLocate the objects with their coordinates.',
	'\nAnswer in [x1, y1, x2, y2] format.',
	'\nMention the objects and their locations using the format [x1, y1, x2, y2].',
	'\nDraw boxes around the mentioned objects.',
	'\nUse boxes to show where each thing is.',
	'\nTell me where the objects are with coordinates.',
	'\nList where each object is with boxes.',
	'\nShow me the regions with boxes.'
]
GROUNDING PROMPTSpython
# GROUNDING PROMPTS
GROUNDING_TEMPLATES = [
	'\nProvide the bounding boxes of the mentioned objects.',
 	'\nInclude the coordinates for each mentioned object.',
	'\nLocate the objects with their coordinates.',
	'\nAnswer in [x1, y1, x2, y2] format.',
	'\nMention the objects and their locations using the format [x1, y1, x2, y2].',
	'\nDraw boxes around the mentioned objects.',
	'\nUse boxes to show where each thing is.',
	'\nTell me where the objects are with coordinates.',
	'\nList where each object is with boxes.',
	'\nShow me the regions with boxes.'
]
GROUNDING PROMPTSpython
# GROUNDING PROMPTS
GROUNDING_TEMPLATES = [
	'\nProvide the bounding boxes of the mentioned objects.',
 	'\nInclude the coordinates for each mentioned object.',
	'\nLocate the objects with their coordinates.',
	'\nAnswer in [x1, y1, x2, y2] format.',
	'\nMention the objects and their locations using the format [x1, y1, x2, y2].',
	'\nDraw boxes around the mentioned objects.',
	'\nUse boxes to show where each thing is.',
	'\nTell me where the objects are with coordinates.',
	'\nList where each object is with boxes.',
	'\nShow me the regions with boxes.'
]
GROUNDING PROMPTSpython
# GROUNDING PROMPTS
GROUNDING_TEMPLATES = [
	'\nProvide the bounding boxes of the mentioned objects.',
 	'\nInclude the coordinates for each mentioned object.',
	'\nLocate the objects with their coordinates.',
	'\nAnswer in [x1, y1, x2, y2] format.',
	'\nMention the objects and their locations using the format [x1, y1, x2, y2].',
	'\nDraw boxes around the mentioned objects.',
	'\nUse boxes to show where each thing is.',
	'\nTell me where the objects are with coordinates.',
	'\nList where each object is with boxes.',
	'\nShow me the regions with boxes.'
]
GROUNDING PROMPTSpython
# GROUNDING PROMPTS
GROUNDING_TEMPLATES = [
	'\nProvide the bounding boxes of the mentioned objects.',
 	'\nInclude the coordinates for each mentioned object.',
	'\nLocate the objects with their coordinates.',
	'\nAnswer in [x1, y1, x2, y2] format.',
	'\nMention the objects and their locations using the format [x1, y1, x2, y2].',
	'\nDraw boxes around the mentioned objects.',
	'\nUse boxes to show where each thing is.',
	'\nTell me where the objects are with coordinates.',
	'\nList where each object is with boxes.',
	'\nShow me the regions with boxes.'
]
GROUNDING PROMPTSpython
# GROUNDING PROMPTS
GROUNDING_TEMPLATES = [
	'\nProvide the bounding boxes of the mentioned objects.',
 	'\nInclude the coordinates for each mentioned object.',
	'\nLocate the objects with their coordinates.',
	'\nAnswer in [x1, y1, x2, y2] format.',
	'\nMention the objects and their locations using the format [x1, y1, x2, y2].',
	'\nDraw boxes around the mentioned objects.',
	'\nUse boxes to show where each thing is.',
	'\nTell me where the objects are with coordinates.',
	'\nList where each object is with boxes.',
	'\nShow me the regions with boxes.'
]

Deploy This Model

Production-ready deployment in minutes

Together.ai

Instant API access to this model

Fastest API

Production-ready inference API. Start free, scale to millions.

Try Free API

Replicate

One-click model deployment

Easiest Setup

Run models in the cloud with simple API. No DevOps required.

Deploy Now

Disclosure: We may earn a commission from these partners. This helps keep LLMYourWay free.