code for i2t inference #10

Activity

1. QuLiao1117 commented on Dec 20, 2024

   

   QuLiao1117

   on Dec 20, 2024

   Contributor

   More actions

   great work! I am wondering can you provide code for i2t inference? There is only code for t2i inference now(scripts/run_t2i.sh)

   The i2t of TokenFlow is built on LLaVA, so you can directly use lmms-eval for inference and evaluation.

   Specifically,

   1. First install the i2t environment by following the installation guide. This will install a "llava" pip package.
   2. Then refer to the LLaVA testing instructions in the lmms-eval repository. Example command:

   python3 -m accelerate.commands.launch \
       --num_processes=8 \
       -m lmms_eval \
       --model llava \
       --model_args pretrained="ByteFlow-AI/Tokenflow-llava-qwen2.5-14B-finetuning" \
       --tasks mme \
       --batch_size 1 \
       --log_samples \
       --log_samples_suffix llava_v1.5_mme \
       --output_path ./logs/
       # you can also change the `--model_args pretrained=` to local path of your model.

   Things might need to be modified:

   • In case lmm-eval can't find the llava package, add a system path in lmms_eval/models/llava.py

   from pathlib import Path
   sys.path.insert(0, str(Path("/path/to/your/TokenFlow/i2t").resolve()))
2. pkulium commented on Dec 20, 2024

   

   pkulium

   on Dec 20, 2024

   Author

   More actions

   Thank you for your reply. Yes, I have used lmms-eval before. I am wondering if there is code available to run inference on a single image of my own?
3. QuLiao1117 commented on Dec 24, 2024

   

   QuLiao1117

   on Dec 24, 2024

   Contributor

   More actions

   Thank you for your reply. Yes, I have used lmms-eval before. I am wondering if there is code available to run inference on a single image of my own?

   This repo does not support this feature currently. PR is welcomed to enable this.

   👍React with 👍1Reacted by pkulium