Skip to main content
API Backends for SwarmUI
Tool

API Backends for SwarmUI

Hartsy 1.3
comments, jump to the thread 58 views

Width

Text size

Line spacing

Supported platforms

  • Windows
  • Linux
  • macOS

Installation

git clone https://github.com/HartsyAI/SwarmUI-API-Backends.git

Features

  • Six API providers in one backend
  • Provider parameters that show and hide per model
  • Video generation through Fal.ai
  • Utility models for background removal, upscaling and face restoration
  • Per-user API key management
  • Custom base URL for enterprise deployments
  • Permission system integration

System requirements

Billing
Charged by each provider per request
SwarmUI
Installed and running
API keys
One per provider you enable

Links

License

License
MIT

The APIBackends extension registers API hosted models into SwarmUI's normal model list, so they can be selected and used from the Generate tab exactly like local models. There is no separate tab, no separate queue and no separate history. You pick API Models/OpenAI/dall-e-3 the same way you pick a checkpoint on your own drive.

API usage incurs costs from the respective providers. Understand the pricing before you enable one, and keep your keys to yourself.

Six providers, one backend

  • OpenAI: DALL-E 2, DALL-E 3, GPT Image 1, GPT Image 1.5, GPT Image 2, Sora
  • Ideogram: V1, V2, V2 Turbo, V3
  • Black Forest Labs: FLUX Pro, Ultra, Dev, Kontext Pro and Max, FLUX 2 Pro and Max
  • Grok: Grok 2 Image
  • Google: Imagen 3.0, Gemini 2.0 Flash, Gemini 2.5 Flash, Gemini 3.1 Flash, Gemini 3 Pro
  • Fal.ai: 130 plus image, video and utility models behind a single API key

Each provider is a toggle on one backend card, so enabling a second one is a checkbox rather than another backend to configure.

The Fal.ai catalog

Fal.ai acts as a unified gateway to models from many providers, so one key covers all of them.

Image: FLUX (dev, schnell, pro, ultra, kontext, flex, lora, general), Recraft V3, Ideogram V2 through V3, Stability AI (SD 3.5, SDXL Lightning, Stable Cascade), Grok, Google (Nano Banana Pro, Imagen 3, Gemini Flash), Kling, Qwen, Bria, ByteDance Seedream, Reve, HiDream, OmniGen, AuraFlow, Lumina, Sana, Playground, Kolors, MiniMax, Step1X, Hunyuan, UNO and Instant Character.

Video: ByteDance Seedance 2.0 and 1.0 in text to video, image to video and reference to video, Sora 2, Veo 3.1, Kling, Grok Video, MiniMax Hailuo, PixVerse V5, Wan 2.2, LTX, Vidu Q3, Hunyuan, Mochi, Luma Ray 2, Pika V2.2, Kandinsky 5 Pro, Magi, CogVideoX, SkyReels and Decart Lucy.

Utility: background removal (rembg, Bria RMBG 2.0, BEN V2), upscaling (Clarity, Topaz, Creative Upscaler, ESRGAN, Topaz Video), face restoration (CodeFormer) and video background removal.

Installing

Install it from Server > Extensions, find "SwarmUI-API-Backends" in the list, press Install and restart when prompted.

To install by hand, close SwarmUI, clone the repository into SwarmUI/src/Extensions, then run update-windows.bat or update-linuxmac.sh from the SwarmUI root to rebuild.

Adding your keys and the backend

  1. Open the User tab and paste your key for each provider you plan to use, then save.
  2. Open Server > Backends, press Add New Backend and pick 3rd Party Paid API Backends.
  3. Toggle on the providers you want, save, and restart the backend.
  4. Open Generate. API models appear under API Models/<Provider>/..., for example API Models/OpenAI/dall-e-3 or API Models/Fal/Sora/sora-2-t2v.

Keys live in SwarmUI user data, per user, and are never hardcoded. The backend card itself only carries the provider toggles and an optional base URL override.

Parameters follow the model

Parameters are shown and hidden automatically based on the model you select, so the sidebar only ever offers what the current model can actually use.

  • OpenAI: image size, quality, style, background, output format, moderation, output compression. GPT Image 2 goes up to 2K but does not support transparent backgrounds.
  • Ideogram: aspect ratio, style type, magic prompt, rendering speed, remix weight, color theme, negative prompt, image prompt.
  • Black Forest Labs: width, height, guidance, steps, prompt enhancement, raw mode, image prompt, safety filter level, output format.
  • Grok: aspect ratio and output resolution.
  • Google: aspect ratio, person generation, image size, image resolution.
  • Fal.ai video: each family carries its own duration, aspect ratio, resolution, generate audio and negative prompt set.

Video outputs are saved as .mp4. Images follow your normal image format settings.

How it is put together

The extension uses a data driven factory pattern so providers stay modular.

  1. Each provider class implements IProviderSource and declares its models.
  2. ProviderDefinitions aggregates every provider source into a single list.
  3. APIProviderInit and ModelFactory convert those into SwarmUI T2IModel objects.
  4. DynamicAPIBackend.Init() registers models only for the providers you enabled.
  5. At generation time DynamicAPIBackend routes to the right provider through RequestBuilder.

No new HTTP routes are added. The backend registers an extra model provider via ModelsAPI.ExtraModelProviders["dynamic_api_backends"], which makes API models appear as remote models in the normal browser, and its parameters come back through ListT2IParams. Generation runs through Swarm's own GenerateText2Image and GenerateText2ImageWS.

Troubleshooting

No API models in the model list

Make sure at least one provider is enabled in the backend settings, that you pressed Save, and that you restarted the backend. Check the SwarmUI server logs for initialization errors.

API key not found

Open the User tab, add the key for the provider you are trying to use, and press Save after pasting it.

The API request failed with a 4xx or 5xx

Verify the key is valid and has balance. The server logs carry the full error response from the provider. Some providers rate limit, so wait and retry.

Content policy violation

Some providers, ByteDance Seedance among them, run strict server side filters. A video can be fully generated before being rejected, which still incurs cost. Adjust the prompt to avoid triggering the filter.

Video finishes but no output appears

Check the server logs, and confirm you are on a video model. Their ids end in -t2v, -i2v or -ref2v.

Parameters are not showing or hiding correctly

Hard refresh the browser with Ctrl+Shift+R, or switch to a different model and back.

Changelog

  • 1.3: ByteDance Seedance video models, toggle switch UI for backend settings, an expanded utility model catalog and content policy troubleshooting.
  • 1.2: Fal.ai video generation across 120 plus models, automatic parameter visibility per model type, video output as mp4.
  • 1.1: modular provider and model factory architecture, unified API Models/<Provider>/... naming.

Comments

Your avatar