AI Creates Custom Portraits of Two People from Single Photos

A new AI system can now generate realistic, customized portraits of two people from just a single photo of each person, addressing a long-standing in personalized image generation. This breakthrough, detailed in a recent paper, introduces the PairHuman dataset—the first large-scale collection specifically for dual-person portraits—and a called DHumanDiff that improves facial consistency and scene customization. The technology has practical uses in fields like wedding photography, where couples can preview sample portraits, and healthcare, where personalized images can aid in reminiscence therapy, making it relevant for everyday applications beyond technical research.

The researchers found that their DHumanDiff outperforms existing approaches in generating high-fidelity dual-person portraits, as shown by quantitative metrics in the paper. For example, on an external test set, DHumanDiff achieved a face similarity score of 0.7010, higher than other s like FastComposer (0.5229) and IP-Adapter (0.6839), indicating better preservation of individual identities. also scored 26.2474 in CLIP-T, which measures alignment with text prompts, and 10.8543 in MPS, a metric for human preference, demonstrating its effectiveness in creating visually appealing and semantically accurate images. These confirm that the approach successfully balances facial consistency with customization of scenes, attire, and actions.

To achieve this, the team developed the PairHuman dataset, which contains over 100,000 high-resolution images across four topics: couples, weddings, female friends, and parent-child portraits. Each image includes rich metadata such as detailed captions, human bounding boxes, keypoints, and attribute tags, ensuring diverse visual content for training. The DHumanDiff uses visual disparity-aware conditioning to separate features of the two reference faces, preventing identity confusion, and subject-augmented conditioning to align facial features with text descriptions. Additionally, a cascaded inference mechanism adjusts the focus between facial details and scene elements during generation, allowing for flexible image layouts without requiring fine-tuning for each new subject.

The data from the paper shows that models trained on the PairHuman dataset, including DHumanDiff, produce better than those trained on existing datasets like FFHQ-wild. In experiments, PairHuman-trained s improved CLIP-T scores from around 17.1 to 26.5, indicating stronger textual consistency, and CLIP-I scores from 0.78 to 0.84, reflecting better image authenticity. Qualitative examples in Figure 8 of the paper illustrate that these s generate specific details like "white bridal veils" and complex poses more accurately. The DHumanDiff also requires only 32 GPU hours for training, compared to 2,096 hours for FastComposer, making it more efficient while maintaining high performance across metrics like face similarity and user preference scores.

Of this work are significant for real-world applications, as it enables personalized dual-person portrait generation without the need for extensive fine-tuning or multiple reference images. In wedding photography, couples can use this technology to visualize and select poses and settings beforehand, enhancing planning efficiency. In psychology and social sciences, customizable portraits can serve as stimuli for studying interpersonal dynamics, while in human-computer interaction, they can create engaging content for virtual environments. The dataset's focus on high photographic standards, with images avoiding issues like truncated faces or cluttered backgrounds, ensures that generated portraits meet aesthetic expectations, broadening its utility in creative and therapeutic contexts.

However, the paper notes several limitations that future work must address. The PairHuman dataset currently has a demographic bias, with 201,922 Asian subjects and an average age of 28.3 years, which may limit its applicability across diverse cultural and age groups. Additionally, DHumanDiff shows sensitivity to lighting conditions in reference images, as highlighted in Appendix A.2, where unnatural lighting led to artifacts in generated portraits. also only supports dual-person scenarios due to its paired cross-attention mechanisms, and expanding to multi-person generation would require scalable tokenization strategies. Despite these s, the researchers suggest that incorporating more diverse data and 3D-aware modeling could enhance robustness and inclusivity in future iterations.