Fix OpenAI API Key Configuration in .env File

[actuallyrizzn]

Issue: OpenAI API Key Configuration in .env File

Problem

When setting up the Letta server via the installer, the .env file may be created with quoted values for API keys. This causes the server to fail when attempting to use OpenAI API keys, resulting in 500 errors when creating agents.

Root Causes

1. Quoted Environment Variables

   • Docker's --env-file flag includes quotes as part of the environment variable value
   • If .env contains: OPENAI_API_KEY="sk-proj-..."
   • Docker sets: OPENAI_API_KEY="sk-proj-..." (with quotes in the value)
   • This causes OpenAI API to reject requests with "Invalid authorization header"

2. Missing LETTA_ENCRYPTION_KEY

   • Letta's Secret class requires LETTA_ENCRYPTION_KEY to be set, even if encryption is not actively used
   • Without it, the Secret class may incorrectly identify API keys as encrypted and fail to decrypt them
   • This results in None being returned instead of the actual API key

Required Fixes

1. Ensure .env file format does not include quotes around values:

   # ✅ Correct
   OPENAI_API_KEY=sk-proj-abc123...
   
   # ❌ Incorrect
   OPENAI_API_KEY="sk-proj-abc123..."

2. Ensure LETTA_ENCRYPTION_KEY is set in .env:

   LETTA_ENCRYPTION_KEY=dummy-encryption-key-32-chars-long!!

   Note: This is required even if encryption is not being used.

Impact

• Severity: High
• User Impact: Server cannot authenticate with OpenAI API, agent creation fails
• Workaround: Manual editing of .env file after installation

References

See detailed incident report for full technical details and investigation.

[actuallyrizzn]

Incident Report: OpenAI API Key Not Being Loaded by Letta Server

Date: November 19, 2025
Severity: High
Status: Resolved
Affected System: Letta Server (Self-hosted Docker container)

---

Executive Summary

The Letta server running in a Docker container was unable to use the OpenAI API key configured in the .env file, resulting in 500 Internal Server Error responses when attempting to create agents. The issue was caused by two separate problems:

1. Quoted API key in .env file - Docker's --env-file was including literal quotes as part of the environment variable value
2. Secret encryption false positive - Letta's Secret class incorrectly identified the API key as encrypted, causing decryption to fail and return None

Both issues have been resolved, and the server is now functioning correctly.

---

Problem Description

Symptoms

• Letta server running in Docker container (screen session)
• Server unable to authenticate with OpenAI API
• HTTP 500 Internal Server Error when creating agents
• Error logs showing: 401 Unauthorized from OpenAI API
• Error message: Invalid authorization header

Impact

• User Impact: Unable to create new agents or use OpenAI-powered features
• System Impact: All agent creation requests failing
• Duration: Unknown (discovered during troubleshooting session)

---

Investigation Timeline

Initial Investigation

1. Environment Variable Check

   • Verified .env file exists at /root/.letta/.env
   • Confirmed OPENAI_API_KEY is set in the file
   • Checked Docker container environment variables
   • Finding: API key was present but included literal quotes: "sk-proj-..."

2. Docker Container Inspection

   docker exec <container> printenv OPENAI_API_KEY
   # Output: "sk-proj-..." (with quotes)

   • Finding: Environment variable contained quotes as part of the value

3. API Key Validation

   • Tested API key directly with OpenAI Python client
   • Result: Key works correctly when used directly
   • Conclusion: Key itself is valid; issue is with how it's being passed to OpenAI

Root Cause Analysis

Issue #1: Quoted Environment Variable

Location: /root/.letta/.env

Problem:

OPENAI_API_KEY="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"

Docker's --env-file flag does not strip quotes from values. When the .env file contains:

OPENAI_API_KEY="value"

Docker includes the quotes as part of the environment variable value, resulting in:

OPENAI_API_KEY="sk-proj-..."

When this value is used in the Authorization header as Bearer "sk-proj-...", OpenAI rejects it with "Invalid authorization header".

Fix Applied:
Removed quotes from the .env file:

sed -i 's/^OPENAI_API_KEY="\(.*\)"$/OPENAI_API_KEY=\1/' /root/.letta/.env

Result: Environment variable now contains the key without quotes, but issue persisted.

Issue #2: Secret Encryption False Positive

Location: Letta's Secret class (/app/letta/schemas/secret.py)

Problem:
After fixing the quotes issue, the API key was still not working. Investigation revealed:

1. Letta uses a Secret class to handle API keys
2. The Secret.from_db() method calls Secret.from_plaintext()
3. from_plaintext() checks if the value appears encrypted using CryptoUtils.is_encrypted()
4. The OpenAI API key (164 characters, base64-decodable) matched the encryption format pattern
5. is_encrypted() returned True (false positive)
6. get_plaintext() attempted to decrypt the value
7. Without LETTA_ENCRYPTION_KEY set, decryption failed
8. Method returned None instead of the plaintext key

Code Flow:

# Provider creation
provider = OpenAIProvider(
    api_key=model_settings.openai_api_key,  # Key is set correctly
    ...
)

# When making API call
secret = provider.get_api_key_secret()  # Returns Secret object
api_key = secret.get_plaintext()  # Returns None!

Root Cause:
The CryptoUtils.is_encrypted() method uses a heuristic check:

def is_encrypted(cls, value: str) -> bool:
    try:
        decoded = base64.b64decode(value)
        # Check if length matches encryption format
        return len(decoded) >= SALT_SIZE + IV_SIZE + TAG_SIZE + 1
    except Exception:
        return False

OpenAI API keys are:

• Base64-decodable (or appear to be)
• Long enough to match the minimum encrypted size (164 characters)
• Therefore incorrectly identified as encrypted

Fix Applied:
Added LETTA_ENCRYPTION_KEY to the .env file:

LETTA_ENCRYPTION_KEY=dummy-encryption-key-32-chars-long!!

Why This Works:

• Setting LETTA_ENCRYPTION_KEY makes CryptoUtils.is_encryption_available() return True
• This prevents the _plaintext_cache from being cleared in get_plaintext()
• The cache set during from_plaintext() is preserved and returned correctly

---

Resolution Steps

Step 1: Remove Quotes from .env File

# Edit /root/.letta/.env
sed -i 's/^OPENAI_API_KEY="\(.*\)"$/OPENAI_API_KEY=\1/' /root/.letta/.env

# Verify
cat /root/.letta/.env | grep OPENAI_API_KEY
# Output: OPENAI_API_KEY=sk-proj-... (no quotes)

Step 2: Add LETTA_ENCRYPTION_KEY

# Add to /root/.letta/.env
echo "" >> /root/.letta/.env
echo "# Encryption key for Letta secrets (required even if not using encryption)" >> /root/.letta/.env
echo "LETTA_ENCRYPTION_KEY=dummy-encryption-key-32-chars-long!!" >> /root/.letta/.env

Step 3: Restart Letta Server

# Stop existing container
docker stop $(docker ps -q --filter ancestor=letta/letta:latest)

# Restart using startup script
bash /root/launch_letta.sh

Step 4: Verification

# Test Secret.get_plaintext() returns correct value
docker exec <container> python3 -c "
from letta.schemas.providers.openai import OpenAIProvider
from letta.settings import model_settings
from letta.schemas.secret import CryptoUtils

print('Encryption available:', CryptoUtils.is_encryption_available())
provider = OpenAIProvider(name='openai', api_key=model_settings.openai_api_key, base_url='https://api.openai.com/v1')
secret = provider.get_api_key_secret()
plaintext = secret.get_plaintext()
print('Plaintext from secret:', repr(plaintext[:30]) if plaintext else 'None')
"
# Expected output: Plaintext from secret: 'sk-proj-M_WPWcmsG7t-BI_W6qpymR'

---

Technical Details

Environment Configuration

• Server: Self-hosted Letta server in Docker
• Container: letta/letta:latest
• Port: 8284 (host) -> 8283 (container)
• Environment File: /root/.letta/.env
• Startup Script: /root/launch_letta.sh

Files Modified

1. /root/.letta/.env
   • Removed quotes from OPENAI_API_KEY
   • Added LETTA_ENCRYPTION_KEY

Code Locations

• Secret Class: /app/letta/schemas/secret.py
• Crypto Utils: /app/letta/helpers/crypto_utils.py
• Provider Class: /app/letta/schemas/providers/base.py
• OpenAI Provider: /app/letta/schemas/providers/openai.py

Key Functions Involved

• Secret.from_db() - Creates Secret from database values
• Secret.from_plaintext() - Creates Secret from plaintext value
• Secret.get_plaintext() - Retrieves decrypted plaintext
• CryptoUtils.is_encrypted() - Heuristic check for encrypted values
• CryptoUtils.is_encryption_available() - Checks if encryption key is configured

---

Prevention & Recommendations

Immediate Actions

1. ✅ Documented .env file format requirements

   • Do not use quotes around values in .env files when using Docker's --env-file
   • Docker includes quotes as part of the value

2. ✅ Added LETTA_ENCRYPTION_KEY to .env

   • Required for Secret class to function correctly
   • Even if encryption is not actively used

Long-term Recommendations

1. Improve Secret Class Logic

   • Consider improving is_encrypted() heuristic to reduce false positives
   • Add explicit flag to indicate whether a value is actually encrypted
   • Consider storing encryption metadata separately

2. Environment Variable Validation

   • Add startup validation to check API keys are properly formatted
   • Log warnings when Secret.get_plaintext() returns None
   • Provide clearer error messages when API authentication fails

3. Documentation Updates

   • Document that LETTA_ENCRYPTION_KEY is required for self-hosted deployments
   • Add examples of correct .env file format
   • Document the Secret class behavior and requirements

4. Testing

   • Add integration tests for API key loading
   • Test with various API key formats
   • Test Secret class with and without encryption key

Configuration Best Practices

Correct .env Format:

# ✅ Correct - No quotes
OPENAI_API_KEY=sk-proj-abc123...
LETTA_ENCRYPTION_KEY=dummy-encryption-key-32-chars-long!!
LETTA_SERVER_PASSWORD=go0dp4ssw0rd

# ❌ Incorrect - Quotes included in value
OPENAI_API_KEY="sk-proj-abc123..."

Docker Command:

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8284:8283 \
  --env-file ~/.letta/.env \
  -e SECURE=true \
  -e LETTA_SERVER_PASSWORD="go0dp4ssw0rd" \
  --add-host=host.docker.internal:host-gateway \
  letta/letta:latest

---

Lessons Learned

1. Docker --env-file Behavior

   • Docker's --env-file does not strip quotes from values
   • This is different from shell variable expansion
   • Always verify environment variables in the container

2. Heuristic Checks Can Fail

   • The is_encrypted() check is a heuristic, not definitive
   • Long, base64-decodable strings can trigger false positives
   • Need explicit metadata or better heuristics

3. Silent Failures

   • Secret.get_plaintext() returning None was a silent failure
   • No clear error message indicated the root cause
   • Better error handling and logging needed

4. Configuration Requirements

   • LETTA_ENCRYPTION_KEY is required even if not using encryption
   • This should be documented and validated at startup

---

Verification

Test Results

• ✅ API key correctly loaded in container environment
• ✅ Secret.get_plaintext() returns correct value
• ✅ Agent creation succeeds
• ✅ OpenAI API calls work correctly
• ✅ Streaming responses functional

Monitoring

• Monitor server logs for any Secret-related warnings
• Watch for 401/403 errors from OpenAI API
• Verify agent creation success rate

---

References

• Letta Server Documentation
• Docker --env-file documentation
• Letta Secret class implementation
• OpenAI API authentication requirements

---

Incident Timeline

Time	Event
T+0	Issue reported: Server not picking up OpenAI API key
T+5	Initial investigation: Found quotes in .env file
T+10	Fixed quotes issue, restarted server
T+15	Issue persisted, deeper investigation
T+30	Discovered Secret class false positive
T+35	Added LETTA_ENCRYPTION_KEY to .env
T+40	Restarted server, verified fix
T+45	Incident resolved

---

Report Prepared By: AI Assistant
Reviewed By: [Pending]
Date: November 19, 2025

[actuallyrizzn]

(Also, yes, I'm aware my real key is listed here. It has been disabled in my API panel, but I wanted to leave it in the report in case there were issues with replicability).

[actuallyrizzn]

PR Summary

A PR has been opened (#3070) with a surgical fix for this issue.

The Fix:

• Added a check in Secret.get_plaintext() to respect the was_encrypted flag before trusting the is_encrypted() heuristic
• When was_encrypted=False, the method now returns the cached plaintext value immediately, preventing false positives from clearing the cache
• Only 4 lines added to letta/schemas/secret.py

Testing:

• Added 3 focused unit tests that specifically target this scenario:
  • Test with real OpenAI API key that triggers false positive
  • Test that was_encrypted=False prevents cache clearing
  • Test that existing behavior for actually encrypted values is preserved

The fix is minimal, focused, and fully tested. Ready for review.