-
Notifications
You must be signed in to change notification settings - Fork 487
Romain/test-ok #3717
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Romain/test-ok #3717
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| import os | ||
|
|
||
| from fastapi import FastAPI | ||
| from huggingface_hub import InferenceClient | ||
| from langchain_core.messages import HumanMessage | ||
| from langchain_openai import ChatOpenAI | ||
| from openai import OpenAI | ||
|
|
||
| app = FastAPI() | ||
|
|
||
|
|
||
| @app.put("/prompt/{user_id}/{user_name}") | ||
| def prompt(user_id: int, user_name: str): | ||
| safe_user_chat = f"ints are safe {user_id}" | ||
| user_chat = f"ints are safe {user_name}" | ||
|
|
||
| client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY")) | ||
| client.chat.completions.create( | ||
| # proruleid: prompt-injection-fastapi | ||
| messages=[ | ||
| {"role": "system", "content": "You are a helpful assistant."}, | ||
| {"role": "user", "content": user_chat}, | ||
| ], | ||
| temperature=0, | ||
| ) | ||
|
|
||
| client.chat.completions.create( | ||
| messages=[ | ||
| {"role": "system", "content": "You are a helpful assistant."}, | ||
| {"role": "user", "content": safe_user_chat}, | ||
| ], | ||
| temperature=0, | ||
| ) | ||
|
|
||
| huggingface = InferenceClient() | ||
| # proruleid: prompt-injection-fastapi | ||
| res = huggingface.text_generation(user_chat, stream=True, details=True) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Semgrep identified an issue in your code: Dataflow graphflowchart LR
classDef invis fill:white, stroke: none
classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none
subgraph File0["<b>python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py</b>"]
direction LR
%% Source
subgraph Source
direction LR
v0["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L13 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 13] user_name</a>"]
end
%% Intermediate
subgraph Traces0[Traces]
direction TB
v2["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L13 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 13] user_name</a>"]
v3["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L15 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 15] user_chat</a>"]
end
v2 --> v3
%% Sink
subgraph Sink
direction LR
v1["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L37 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 37] user_chat</a>"]
end
end
%% Class Assignment
Source:::invis
Sink:::invis
Traces0:::invis
File0:::invis
%% Connections
Source --> Traces0
Traces0 --> Sink
To resolve this comment: ✨ Commit Assistant Fix Suggestion
Prompt injection is possible when user-controlled input is included in the prompt for an LLM without validation, escaping, or clear segmentation, allowing users to "break out" of the intended structure. Input validation reduces the risk of unexpected prompt alteration. 💬 Ignore this findingReply with Semgrep commands to ignore this finding.
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by prompt-injection-fastapi. You can view more details about this finding in the Semgrep AppSec Platform. |
||
|
|
||
| huggingface = InferenceClient() | ||
| # proruleid: prompt-injection-fastapi | ||
| res = huggingface.text_generation(user_chat, stream=True, details=True) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Semgrep identified an issue in your code: Dataflow graphflowchart LR
classDef invis fill:white, stroke: none
classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none
subgraph File0["<b>python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py</b>"]
direction LR
%% Source
subgraph Source
direction LR
v0["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L13 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 13] user_name</a>"]
end
%% Intermediate
subgraph Traces0[Traces]
direction TB
v2["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L13 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 13] user_name</a>"]
v3["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L15 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 15] user_chat</a>"]
end
v2 --> v3
%% Sink
subgraph Sink
direction LR
v1["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L41 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 41] user_chat</a>"]
end
end
%% Class Assignment
Source:::invis
Sink:::invis
Traces0:::invis
File0:::invis
%% Connections
Source --> Traces0
Traces0 --> Sink
To resolve this comment: ✨ Commit Assistant Fix Suggestion
Using strong input validation and separating user input contextually in prompts helps prevent attackers from injecting harmful instructions into LLM queries. 💬 Ignore this findingReply with Semgrep commands to ignore this finding.
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by prompt-injection-fastapi. You can view more details about this finding in the Semgrep AppSec Platform. |
||
|
|
||
| chat = ChatOpenAI(model="gpt-3.5-turbo-1106", temperature=0.2) | ||
| # proruleid: prompt-injection-fastapi | ||
| chat.invoke([HumanMessage(content=user_chat)]) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Semgrep identified an issue in your code: Dataflow graphflowchart LR
classDef invis fill:white, stroke: none
classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none
subgraph File0["<b>python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py</b>"]
direction LR
%% Source
subgraph Source
direction LR
v0["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L13 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 13] user_name</a>"]
end
%% Intermediate
subgraph Traces0[Traces]
direction TB
v2["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L13 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 13] user_name</a>"]
v3["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L15 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 15] user_chat</a>"]
end
v2 --> v3
%% Sink
subgraph Sink
direction LR
v1["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L45 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 45] user_chat</a>"]
end
end
%% Class Assignment
Source:::invis
Sink:::invis
Traces0:::invis
File0:::invis
%% Connections
Source --> Traces0
Traces0 --> Sink
To resolve this comment: ✨ Commit Assistant Fix Suggestion
Alternatively, if you want to reject invalid usernames altogether, raise an error if the input doesn't match your allowed pattern. Input sanitization reduces the risk of prompt injection by removing unexpected control characters or instructions that a malicious user could provide. 💬 Ignore this findingReply with Semgrep commands to ignore this finding.
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by prompt-injection-fastapi. You can view more details about this finding in the Semgrep AppSec Platform. |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Semgrep identified an issue in your code:
A prompt is created and user-controlled data reaches that prompt. This can lead to prompt injection. Make sure the user inputs are properly segmented from the system's in your prompts.
Dataflow graph
flowchart LR classDef invis fill:white, stroke: none classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none subgraph File0["<b>python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py</b>"] direction LR %% Source subgraph Source direction LR v0["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L13 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 13] user_name</a>"] end %% Intermediate subgraph Traces0[Traces] direction TB v2["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L13 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 13] user_name</a>"] v3["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L15 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 15] user_chat</a>"] end v2 --> v3 %% Sink subgraph Sink direction LR v1["<a href=https://github.com/semgrep/semgrep-rules/blob/c2f7953f3ffe21cf3532da7aa3dc6052d919b2cf/python/fastapi/ai/prompt-injection-fastapi/prompt-injection-fastapi.py#L20 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 20] [<br> {"role": "system", "content": "You are a helpful assistant."},<br> {"role": "user", "content": user_chat},<br> ]</a>"] end end %% Class Assignment Source:::invis Sink:::invis Traces0:::invis File0:::invis %% Connections Source --> Traces0 Traces0 --> SinkTo resolve this comment:
✨ Commit Assistant Fix Suggestion
user_chat = f"ints are safe {user_name}"with code that validates or sanitizesuser_name.user_nameto be a plain name, restrict to allowed characters using a regex or manual check. Example:import reand useif not re.match(r"^[a-zA-Z0-9_ -]{1,32}$", user_name): raise ValueError("Invalid user name").user_name = user_name.replace("{", "").replace("}", "").user_chat = f"ints are safe {user_name}".user_chatfor all calls instead of the raw one. For example, in your OpenAI and HuggingFace requests, replace the user message content parameter with the sanitized version."\nSystem: ..."or similar) by keeping formatting simple and validated.Only allow trusted or validated input to reach the LLM prompt, since prompt injection can result in loss of control over the model's outputs or leakage of system information.
💬 Ignore this finding
Reply with Semgrep commands to ignore this finding.
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasonsAlternatively, triage in Semgrep AppSec Platform to ignore the finding created by prompt-injection-fastapi.
You can view more details about this finding in the Semgrep AppSec Platform.