Repository navigation
The direct application of this method to VAE #15
Description
Great work!
I have a couple of questions regarding the potential extension of your work:
Application to VAE:
Given that continuous modeling often outperforms VQ (Vector Quantization), do you think your approach could be applied to Autoencoder-KL (Variational Autoencoder with Kullback-Leibler divergence)? Specifically, could the continuous latent representations in your framework be adapted to improve the performance or efficiency of VAE models?
Unifying High-Level and Low-Level Representations via Distillation:
Have you considered using distillation techniques to unify high-level and low-level representations within your framework? For instance, could a teacher model with advanced representations guide a student model to learn both high-level semantic features and low-level perceptual details, thereby creating a more cohesive and efficient multimodal system?
I believe addressing these questions could further enhance the versatility and impact of your already groundbreaking work. Thank you for your time, and I look forward to your insights!
Activity
QuLiao1117 commentedon Jan 7, 2025
@lavinal712 Thanks for your attention to TokenFlow.
Regarding application to VAE:
Regarding unification through distillation:
bollossom commentedon Oct 14, 2026
Hi, the continuous-latent and distillation direction discussed here is close to what we explored in HYDRA. We combine a continuous bottleneck with semantic distillation and a flow decoder, and evaluate the tokenizer on both understanding and generation.
I'm one of the authors. Our training and evaluation code, weights, and training logs are available if useful as a related implementation.