Case Study: How Moment Factory Reimagined 3D Projection Mapping with ComfyUI

What enables generative AI to operate effectively in architectural contexts? Moment Factory leveraged ComfyUI to revolutionize their strategy for initial idea generation, visual style refinement, and experimentation in projection mapping for buildings.

Prior to adopting ComfyUI, this stage took longer, involved vaguer concepts, and posed higher challenges. With ComfyUI, the process accelerated, becoming grounded in physical space early on.

Period Before Implementation: Limited Iteration Speed, Conceptual Choices, Delayed Issues

Traditional initial concept and style development relied on still sketches, reference collections, mood boards, and theoretical discussions about creative goals. For projecting images onto structures, this presents difficulties. Actual performance rarely becomes clear until scaling up for testing. Seam alignment, pixel resolution, positional shifts, and composition problems typically emerge later in development phases, making changes costly. Customarily, this results in fewer options being seriously considered, extended feedback cycles, creative choices made without tangible spatial evidence, and setbacks deferred to production stages.

Shifts Introduced Through Adoption

Moment Factory developed a tailored ComfyUI pipeline to enhance and expedite extensive areas of preparatory sketching, look-specific inquiry, and initial creative steps. They transformed how selections were determined rather than merely producing images.

1. Repetition Eliminated as a Constraint

ComfyUI transformed the refinement cycle into a quicker, more targeted, and deliberate activity. Anchored to practical production metrics, they investigated more than 20 central creative pathways, with 20 to 40 versions each, covering ultra-real styles up to etched illustrations. The team employed parameter adjustments and batch handling for rapid movement while actively exploring system capacities to understand limitations. This range of investigation became feasible specifically due to ComfyUI.

2. Conception Reduced from Multiple Days to Hours

Key acceleration occurred in preparatory phases. Activities previously requiring days of exchanges between developers and designers finalized in hours. Intentionally low-resolution outputs around 2K enabled prompt assessments and instant alterations at locations, later verified in media sequences rapidly. This coarse-resolution phase focused on confirming design choices rather than finish quality, compressing timelines drastically.

3. Spatial Reliability Occurred Initially

Success hinged on imposing spatial constraints from the outset. The workflow utilized architectural surface blueprints so output shapes were inherently structured from initiation. Multi-template capabilities ran concurrently, including flat surfaces and panoramic layouts. ControlNet introduced structural data directly into diffusion, ensuring dimensional logic stayed integral throughout. This embedded spatial integrity during early design, simplifying integration and converting ideas into collaborative reference points.

4. Approval Signified Continuation Instead of Rebuilding

Once a design approach was validated, progress didn't restart. They could refine specific regions via inpainting, retain compositions, and amplify resolved outputs to 18K resolution in approximately twenty minutes. This transformed rate of motion from concept to deployment-stage material. Under earlier conditions, approval often necessitated recreating work; now, it drove progress.

5. Reduced Team Numbers, Enhanced Cooperation

Once stabilized, one primary artist managed ComfyUI, supported by two contributors guiding artistic direction, prompt adjustments, selection, and alignment dialogues. Defining fresh collaborative methods ensured creative focus remained central, turning ComfyUI into a shared exploration apparatus instead of a solitary arrangement.

6. The Definitive Validation Moment

Initial assessments within innovation groups indicated a leap; flexibility exceeded rigid tool constraints. A pivotal demonstration at 25 Broadway surfaced when a surface template switch reran the pipeline without altering base elements. Layout continuity and spatial logic preserved integrity. Content integrated seamlessly into media servers unnoticed. Silence ensued; it became perceived as indispensable rather than speculative, shifting inquiries from possibility to practical application. This revealed ComfyUI's role beyond research: transforming creative thinking and workflow evolution across operations.

ComfyUI's Critical Nature for Building Scales

Investigations into diffusion workflows at large architectural scales occurred for years. The goal centered on applying generative systems as structured spatial components under demanding settings. Such dimensions necessitated precision in spatial conditioning controls, UV layout incorporation, depth constraint injection, rapid template shifts without disrupting arrangements, progressive enhancement without full redrafts, and evolving pipelines responding to constraints. Node-based configuration allowed modifying workflow logic itself as projects advanced, making architectural-scale generative operations reliably feasible.

Key Insights

ComfyUI didn't influence creative decisions. Human vision persisted within architectural and production limitations. Its value centered on structural adaptability: reshaping workflows as projects evolved, integrating spatial inputs directly, swapping blueprints without disruption, allowing refinements sans rebuilds. Generative systems became controllable mediums rather than black boxes, embedding spatial logic early and scaling dependably. This improved speed while enabling early validation against spatial parameters, minimizing uncertainty during transitions to production. Ultimately, ComfyUI made architectural-scale generative workflows structurally sustainable under production pressures.

Key Contributors:
Guillaume Borgomano | Principal Multimedia Coordinator & Innovation Creative Head
Conner Tozier | Senior Motion Expert & Generative AI Principal

Further details: https://momentfactory.com/blogs/lab/exploration-at-25-broadway