Conversation
…ing docs Add mixed_worker.py demonstrating numpy used by both GPU and CPU endpoints -- the key scenario where the dependency blacklist fix matters. Update README with GPU vs CPU packaging section explaining base image differences, build exclusions, and the runtime safety net.
There was a problem hiding this comment.
Pull request overview
Adds a new “mixed dependencies” worker example and expands the dependency-management documentation to explain how Flash packages dependencies differently for GPU vs CPU runtimes.
Changes:
- Add
mixed_worker.pyshowing a shared Python dependency (numpy) used by both GPU and CPU endpoints. - Expand
04_dependencies/README.mdwith GPU vs CPU packaging guidance, build exclusions, and runtime behavior notes. - Update README code snippets to use
GpuType.NVIDIA_GEFORCE_RTX_4090instead of staleGpuGroupexamples.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| 01_getting_started/04_dependencies/mixed_worker.py | New example worker demonstrating shared dependency usage across GPU/CPU endpoints. |
| 01_getting_started/04_dependencies/README.md | Documentation updates for packaging behavior, exclusions, and updated GPU type references. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
- Clarify GPU worker docstring: numpy computations are CPU-bound despite GPU instance - Add input validation for size (clamped 1-10000) and values (list, max 100k elements) - Add note about GpuGroup vs GpuType enum inconsistency in README
Replace untyped `request: dict` with `numbers: list[float]` to fix division by zero on empty input and provide proper Swagger examples. Module-level Pydantic models cannot be used in LB endpoints because function bodies are serialized to remote workers.
- Use typed param (numbers: list[float]) instead of untyped dict for proper Swagger examples and input validation - Remove broad except Exception that swallowed ImportError as misleading device info; torch should be available on GPU worker images - Add dependencies=["torch"] for environments where torch needs explicit installation
runpod-Henrik
left a comment
runpod-Henrik
left a comment
There was a problem hiding this comment.
What changed: New mixed_worker.py demonstrating numpy on both a GPU and a CPU endpoint in the same file. Updated README documenting GPU vs CPU packaging differences and build exclusions. gpu_lb.py cleanup: typed parameters, torch declared as a dependency, defensive error-swallowing removed.
What was tested:
mixed_worker.pyend-to-end — both GPU (matrix multiply) and CPU (statistics) endpoints executed on RunPod infrastructure and completed successfully with numpy available in both runtime environmentsflash runat repo root — both new endpoints discovered and routable alongside all existing workers, no collisions
What works: numpy available on both GPU and CPU endpoints. Both workers discoverable and routable.
Verdict: Pass.
Summary
mixed_worker.pydemonstrating numpy used by both GPU and CPU endpoints04_dependencies/README.mdwith GPU vs CPU packaging section, build exclusions, runtime safety net docsGpuGroupreferences toGpuType.NVIDIA_GEFORCE_RTX_4090Companion PRs:
Test plan
make quality-checkpassespython mixed_worker.pyruns locally without errorsflash runin04_dependencies/discovers all three workers