Request for Online Image Creation Support for Pivotal Tuning Models

As the title suggests, during the training process of LoRA (as well as other training methods), a technique called pivotal tuning is employed. This approach involves training a LoRA model alongside an embedding model, both of which together constitute the final model. Consequently, both components need to be loaded simultaneously for the method to take effect.

This technique is backed by a paper and its effectiveness has been rigorously demonstrated: https://arxiv.org/abs/2106.05744. Although the paper is based on GANs, we have also conducted migration tests on stable diffusion. This has resulted in the training of several hundred models with high quality and strong generalization capabilities, making the effects outstanding. Here is our account, as civitai's platform only allows the selection of a few specific types during the upload process, we had to choose LoRA for all: https://civitai.com/user/narugo1992/models. All models labeled as LoRA types have been trained using Pivotal Tuning and consist of a .safetensors file and a .pt file. The former needs to be loaded for LoRA, and the latter needs to be loaded for the embedding model. Both components must be used together for the technique to work.

Considering this, we believe that supporting Pivotal Tuning models is crucial for the new feature Online Image Creation. In fact, not only for LoRA, this technique might be used in other contexts as well. The model usually consists of a main model (such as LoRA, Lycoris, CheckPoint, etc.) and one or more embedding models. Therefore, when Online Image Creation loads the model, it should detect and load the main model file (e.g., a .safetensors format file), and the remaining .pt files should be loaded as embeddings, so that the model can function properly.

We hope that the above-mentioned functionality can receive official support.