Custom domain setup fails with "Not authorized" on self-hosted instances

[scorpion7slayer]

Custom domain setup fails with "Not authorized" on self-hosted instances

Description

When trying to set up a custom domain for an organization on a self-hosted Cap instance, the operation fails with a 500 Internal Server Error and the message "Not authorized".

The custom domain dialog works fine (UI loads, input is enabled, the "Next" button is clickable), but clicking "Next" triggers a Server Action (POST /dashboard/settings/organization) that returns a 500 error.

Steps to reproduce

1. Deploy Cap using the official docker-compose.coolify.yml or docker-compose.yml with the pre-built image ghcr.io/capsoftware/cap-web:latest
2. Log in and go to Organization Settings
3. Click Setup on the Custom Domain section
4. Enter a valid domain (e.g. share.example.com)
5. Click Next

Expected behavior

The domain should be saved and the setup should proceed to the verification step.

Actual behavior

The request fails with:

POST https://cap.example.com/dashboard/settings/organization 500 (Internal Server Error)

Server logs show:

⨯ Error: Not authorized

Root cause analysis

After investigating the source code, the issue appears to be related to two things:

1. NEXT_PUBLIC_IS_CAP is not set in the pre-built Docker image. The proxy middleware in apps/web/proxy.ts activates restrictive logic when buildEnv.NEXT_PUBLIC_IS_CAP !== "true". While /dashboard is in the allowlist, the Server Action context may still be affected by the middleware behavior on self-hosted builds.

2. The getCurrentUser() call in apps/web/actions/organization/update-domain.ts returns null for the Server Action, even though the same session works fine for page rendering. This suggests the session is not being properly passed or validated in the Server Action context on self-hosted builds.

Environment

• Cap version: ghcr.io/capsoftware/cap-web:latest (as of July 2026)
• Deployment: Docker Compose on VPS (Dokploy)
• NEXT_PUBLIC_IS_CAP: not set (default in pre-built image)
• WEB_URL and NEXTAUTH_URL: correctly configured with production domain
• NEXTAUTH_SECRET: properly generated with openssl rand -hex 32

Suggested fix

Either:

• Set NEXT_PUBLIC_IS_CAP=true in the pre-built Docker image so self-hosted instances behave like cap.so
• Or ensure Server Actions properly pass the session on self-hosted builds where NEXT_PUBLIC_IS_CAP is unset
• Or document that custom domains require building from source with NEXT_PUBLIC_IS_CAP=true

Related

• fix(web): exclude .well-known from proxy matcher so self-hosted workflows run #1884 (similar proxy middleware issue affecting workflows on self-hosted)

[linear]

CAP-766

[tembo]

Thanks for the detailed analysis. A few things from the codebase that are relevant here:

PR #1884 addresses a related proxy issue - That PR is still open and fixes the proxy matcher to exclude .well-known paths that were breaking workflows on self-hosted builds. It's worth tracking since it touches the same apps/web/proxy.ts file and the same NEXT_PUBLIC_IS_CAP conditional logic you identified.

The domain setup has a deeper issue for self-hosted - Looking at apps/web/actions/organization/domain-utils.ts, all the domain operations (addDomain, checkDomainStatus, verifyDomain) call the Vercel API directly:

const response = await fetch(
  `https://api.vercel.com/v9/projects/${process.env.VERCEL_PROJECT_ID}/domains?teamId=${process.env.VERCEL_TEAM_ID}`,
  {
    headers: {
      Authorization: `Bearer ${process.env.VERCEL_AUTH_TOKEN}`,
    },
    // ...
  }
);

Even if the auth issue is resolved, custom domains on self-hosted instances would need these Vercel credentials or an alternative implementation that doesn't depend on Vercel's domain management API.

The userIsPro check is also a blocker - In apps/web/actions/organization/update-domain.ts:23-25, there's a subscription check that will fail on self-hosted without Stripe configuration. This is the same class of issue as #1405 where organization invites require a subscription even on self-hosted.

For the immediate auth issue - The getCurrentUser() function in packages/database/auth/session.ts uses getServerSession(authOptions()). If Server Actions aren't receiving the session cookies properly, it could be related to how the proxy middleware handles POST requests differently from page renders when NEXT_PUBLIC_IS_CAP is unset.

Setting NEXT_PUBLIC_IS_CAP=true in the Docker build (or as a runtime env var) would bypass the restrictive proxy logic, but would also expose marketing/landing pages that are intended only for cap.so.