Original source of the GGUF file: juanjgit's Orca Mini 3B GGUF Model Card Ensure that the crtansformers Python Library is installed: Here is a self-contained Python script to get you started: