1. Workflow Overview

Purpose:
This workflow is designed for product-background blending, featuring:

• Precise product segmentation (e.g., perfume bottle) via SAM + GroundingDINO

• Seamless fusion with BrushNet-controlled generation (e.g., forest background)

• Supports resolution scaling, prompt optimization, and mask inversion

Core Models:

• Stable Diffusion XL (SDXL): Base model (LEOSAM HelloWorld)

• BrushNet: Local control model for style consistency

• Segment Anything (SAM): Auto-masking products

• GroundingDINO: Object detection via text prompts (e.g., "bottle")

2. Node Breakdown

Key Nodes:

• BrushNetLoader: Loads BrushNet model (segmentation_mask_brushnet_ckpt_sdxl_v1.safetensors)

• GroundingDinoSAMSegment: Generates masks using SAM + GroundingDINO

• InvertMask: Inverts masks to protect product areas

• WD14Tagger: Auto-generates background tags (e.g., "sunshine, petals")

Installation:

• BrushNet: Manual install via GitHub (download .safetensors)

• SAM/GroundingDINO: Install Impact Pack and Segment Anything via ComfyUI Manager

• WD14Tagger: Use comfyui-wd14-tagger plugin

Dependencies:

• BrushNet model must be placed in ComfyUI/models/brushnet

• SAM model (sam_vit_h_4b8939.pth) required in ComfyUI/models/sam

3. Workflow Structure

Group Logic:

1. Segmentation Group:

   • Input: Product image + text prompt (e.g., "bottle")

   • Output: Product mask (white=protected)

   • Key Nodes: LoadImage → ImageScale → GroundingDinoSAMSegment

2. Background Group:

   • Input: Background image + WD14Tagger labels

   • Output: Background prompts (e.g., "forest scenery")

   • Key Nodes: LoadImage → WD14Tagger → CLIPTextEncode

3. BrushNet Blending Group:

   • Input: Product mask, background prompts, BrushNet model

   • Output: Blended image (product + background)

   • Key Nodes: BrushNet → KSampler → VAEDecode

4. Inputs & Outputs

Inputs:

• Required:

  • Product image (PNG/JPG)

  • Background image (PNG/JPG)

  • Object prompt (e.g., "bottle")

• Optional:

  • Mask threshold (default: 0.32)

  • Sampling steps (default: 25)

Outputs:

• Format: PNG image

• Content: Natural product-background fusion (e.g., bottle in a forest)

5. Notes

⚠️ Warnings:

• Low VRAM: Reduce resolution (<768x1024) or use --medvram

• Poor masking: Adjust GroundingDINO threshold (0.2~0.5)

• BrushNet errors: Verify model is float16 format

💡 Tips:

• Use high-res backgrounds (≥1024px) for sharp outputs

• Try dpmpp_2s_ancestral sampler in KSampler for details