fal/moondream2-docci-instruct

🤗 Hugging Face 来源image-text-to-textapache-2.01.9B 参数3.7 GBsafetensors✓ 1 个校验和今天更新

一条命令提交

在你的模型文件夹旁边运行它。它会制作种子、将文件与 Hugging Face 比对，然后提交。你只需开始做种，并粘贴你账户中的密钥。它只读取你的文件，绝不修改。如果愿意，可以先阅读脚本。

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo fal/moondream2-docci-instruct ./model-folder

Fine tuned version of moondream2 model using gokaygokay/random_instruct_docci dataset. Which gives extremely detailed captions of the images.

pip install transformers timm einops bitsandbytes accelerate flash-attn

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from PIL import Image

DEVICE = "cuda"
DTYPE = (
    torch.float32 if DEVICE == "cpu" else torch.float16
)  # CPU doesn't support float16
revision = "3ec40c7b6b5d87bc0c51edee45e21f5f29b449d8"
tokenizer = AutoTokenizer.from_pretrained(
    "fal-ai/moondream2-docci-instruct",
    trust_remote_code=True,
    revision=revision
)
moondream = AutoModelForCausalLM.from_pretrained(
    "fal-ai/moondream2-docci-instruct",
    trust_remote_code=True,
    torch_dtype=DTYPE,
    device_map={"": DEVICE},
    attn_implementation="flash_attention_2",
    revision=revision
)
moondream.eval()

image_path = "<your_image_path>"
image = Image.open(image_path).convert("RGB")
md_answer = moondream.answer_question(
    moondream.encode_image(image),
    "what is this picture about",
    tokenizer=tokenizer,
)

print(md_answer)