feat: 2.6.0 inference

README.md

@@ -6,7 +6,7 @@ Includes `synapsectl` command line utility:
 
     % synapsectl --help
     usage: synapsectl [-h] [--uri URI] [--version] [--verbose]
-                    {discover,info,query,start,stop,configure,logs,read,plot,file,taps,deploy,build,settings} ...
+                    {discover,info,query,start,stop,configure,logs,read,plot,file,taps,deploy,build,settings,deploy-model} ...
 
     Synapse Device Manager
 
@@ -17,7 +17,7 @@ Includes `synapsectl` command line utility:
     --verbose, -v         Enable verbose output
 
     Commands:
-    {discover,info,query,start,stop,configure,logs,read,plot,file,taps,deploy,build,settings}
+    {discover,info,query,start,stop,configure,logs,read,plot,file,taps,deploy,build,settings,deploy-model}
         discover            Discover Synapse devices on the network
         info                Get device information
         query               Execute a query on the device
@@ -32,6 +32,7 @@ Includes `synapsectl` command line utility:
         deploy              Deploy an application to a Synapse device
         build               Cross-compile and package an application into a .deb without deploying
         settings            Manage the persistent device settings
+        deploy-model        Deploy a machine learning model to a Synapse device
 
 As well as the base for a device implementation (`synapse/server`),
 
@@ -194,3 +195,123 @@ After recording data to a file, you can generate plots to visualize your data. U
 ```
 synapsectl plot --dir <path to directory containing .dat and .json>
 ```
+
+## Model Deployment
+
+Deploy machine learning models to Synapse devices.
+
+### Prerequisites
+
+1. **Docker** — required for model conversion
+2. **QAIRT SDK v2.34** (Qualcomm AI Runtime) — required for model conversion for DSP runtime, not required for CPU runtime
+
+### Quick Start — Deploy a Float Model (CPU)
+
+The simplest path — no calibration data needed, runs on CPU via onnxruntime:
+
+```bash
+synapsectl deploy-model model.onnx \
+  --name my_model \
+  -u <device-ip>
+```
+
+#### Installing the QAIRT SDK
+
+1. Create a free account at [softwarecenter.qualcomm.com](https://softwarecenter.qualcomm.com/) (no paid license required)
+2. Download **Qualcomm Software Center** (Linux `.deb`) and **Qualcomm AI Runtime v2.34** (Linux `.qik`) from the website
+3. Install both:
+   ```bash
+   # Install Qualcomm Software Center (includes the qik package manager)
+   sudo dpkg -i QualcommSoftwareCenter*.deb
+
+   # Install the QAIRT SDK
+   sudo /opt/qcom/softwarecenter/bin/qik/qik INSTALL "/path/to/Qualcomm_AI_Runtime_SDK.2.34.0.250424.Linux-AnyCPU.qik"
+   ```
+4. The SDK installs to `/opt/qcom/aistack/qairt/2.34.0.250424`. You'll pass this path as `--snpe-root` when deploying models.
+
+
+### Deploy a Quantized Model (DSP)
+
+For production performance, quantize the model to INT8 for DSP inference. This requires **calibration data** — a small set of example inputs that represent what the model will see in real use.
+
+#### What is calibration data and why do I need it?
+
+Quantization converts your model from 32-bit floats to 8-bit integers, making it ~4x smaller and much faster on the DSP. But to do this well, the quantizer needs to see what typical input values look like so it can choose the right scale for each layer. Bad calibration data leads to accuracy loss.
+
+**Good calibration data** is a handful of real (or realistic) inputs from your application. For example:
+- If your model processes neural signals, use 10-50 snippets of actual recorded neural data
+- If your model processes audio, use 10-50 clips of real audio
+- If you don't have real data yet, synthetic data that matches the expected distribution is acceptable
+
+Ideally, use at least **1000 representative samples** for best accuracy. Fewer samples (50-100) can work for initial testing, but more data gives the quantizer a better picture of your model's value ranges.
+
+#### Step 1: Create calibration `.raw` files
+
+Each `.raw` file is a flat binary dump of float32 values matching your model's input shape. Create them with numpy:
+
+```python
+import numpy as np
+
+# Example: model expects input shape [1, 1920]
+# Load your real data here — these should be actual inputs, not random noise
+for i, sample_data in enumerate(my_real_samples[:20]):
+    # sample_data should be a numpy array with shape matching your model input
+    sample_data.astype(np.float32).tofile(f"sample_{i:03d}.raw")
+```
+
+If you don't have real data yet, you can use synthetic data to get started (accuracy may be lower):
+
+```python
+import numpy as np
+
+for i in range(20):
+    sample = np.random.randn(1, 1920).astype(np.float32)
+    sample.tofile(f"sample_{i:03d}.raw")
+```
+
+#### Step 2: Create an input list file
+
+Create a text file called `input_list.txt` listing your `.raw` files, one per line. Put this file in the same directory as the `.raw` files.
+
+```
+sample_000.raw
+sample_001.raw
+sample_002.raw
+sample_003.raw
+sample_004.raw
+sample_005.raw
+sample_006.raw
+sample_007.raw
+sample_008.raw
+sample_009.raw
+```
+
+#### Step 3: Deploy with quantization
+
+```bash
+synapsectl deploy-model model.onnx \
+  --name my_model \
+  --quantize --input-list input_list.txt \
+  --snpe-root /opt/qcom/aistack/qairt/2.34.0.250424 \
+  -u <device-ip>
+```
+
+### Use in Your C++ App
+
+```cpp
+#include <synapse-app-sdk/inference/model.hpp>
+
+// Loads models/<name>.dlc from the device model directory
+auto model = synapse::create_model("my_model");
+
+if (model && model->is_ready()) {
+    auto result = model->infer(input_data);
+    // result.success, result.outputs, result.inference_time_us
+}
+```
+
+The runtime is selected automatically: quantized models run on the DSP, float models run on CPU. You can also specify a runtime explicitly:
+
+```cpp
+auto model = synapse::create_model("my_model", synapse::InferenceRuntime::kDsp);
+```

setup.py

@@ -6,11 +6,17 @@
 
 setup(
     name="science-synapse",
-    version="2.5.0",
+    version="2.6.0a2",
     description="Client library and CLI for the Synapse API",
     author="Science Team",
     author_email="team@science.xyz",
     packages=find_packages(include=["synapse", "synapse.*"]),
+    package_data={
+        "synapse": [
+            "utils/model_converter/docker/Dockerfile",
+            "utils/model_converter/docker/convert.py",
+        ],
+    },
     long_description=long_description,
     long_description_content_type="text/markdown",
     python_requires=">=3.9",
@@ -32,6 +38,14 @@
         "scipy",
         "h5py",
     ],
+    extras_require={
+        "model-convert": [
+            "onnx>=1.12.0,<1.16.0",
+            "torch",
+            "packaging",
+            "protobuf",
+        ],
+    },
     entry_points={
         "console_scripts": [
             "synapsectl = synapse.cli:main",

synapse/cli/__main__.py

@@ -10,6 +10,7 @@
 
 from synapse.cli import (
     apps,
+    deploy_model,
     discover,
     files,
     offline_plot,
@@ -77,6 +78,7 @@ def main():
     taps.add_commands(subparsers)
     apps.add_commands(subparsers)
     settings.add_commands(subparsers)
+    deploy_model.add_commands(subparsers)
     args = parser.parse_args()
 
     # If we need to setup the device URI, do that now

synapse/cli/build.py

@@ -340,6 +340,11 @@ def build_deb_package(app_dir: str, app_name: str, version: str = "0.1.0") -> bo
         lib_dst_dir = os.path.join(staging_dir, "opt", "scifi", "lib")
         os.makedirs(lib_dst_dir, exist_ok=True)
 
+        # QNN libraries are dlopen'd from /usr/lib/ (hardcoded paths in SDK),
+        # so they must be staged there — not in /opt/scifi/lib/
+        qnn_dst_dir = os.path.join(staging_dir, "usr", "lib")
+        os.makedirs(qnn_dst_dir, exist_ok=True)
+
         try:
             arch_suffix = detect_arch()
             image_tag = f"{app_name}:latest-{arch_suffix}"
@@ -349,6 +354,33 @@ def build_deb_package(app_dir: str, app_name: str, version: str = "0.1.0") -> bo
                 f"[yellow]Extracting SDK libraries from Docker image [bold]{image_tag}[/bold]...[/yellow]"
             )
 
+            # Skip vcpkg .so files already shipped by scifi-headstage-shared-libraries
+            # to avoid dpkg file-overwrite conflicts when the app deb is installed.
+            extract_script = r"""
+set -e
+filter=/tmp/scifi_shared_libs.txt
+: > "$filter"
+if dpkg-query -W -f='${Status}' scifi-headstage-shared-libraries 2>/dev/null | grep -q "install ok installed"; then
+    dpkg-query -L scifi-headstage-shared-libraries | grep -oE '[^/]+\.so[^/]*$' | sort -u > "$filter" || true
+else
+    echo "WARNING: scifi-headstage-shared-libraries not installed in build image; packaging all vcpkg .so files (may conflict on device)" >&2
+fi
+
+find /usr/lib -maxdepth 1 -name 'libsynapse*.so*' -exec cp -a {} /out/ \;
+
+vcpkg_lib="${VCPKG_ROOT}/build/host/vcpkg_installed/arm64-linux-dynamic-release/lib"
+if [ -d "$vcpkg_lib" ]; then
+    find "$vcpkg_lib" -maxdepth 1 -name '*.so*' -print0 | while IFS= read -r -d '' f; do
+        base=$(basename "$f")
+        if [ -s "$filter" ] && grep -qxF "$base" "$filter"; then
+            echo "Skipping $base (already shipped by scifi-headstage-shared-libraries)" >&2
+        else
+            cp -a "$f" /out/
+        fi
+    done
+fi
+""".strip()
+
             docker_cmd = [
                 "docker",
                 "run",
@@ -360,11 +392,29 @@ def build_deb_package(app_dir: str, app_name: str, version: str = "0.1.0") -> bo
                 image_tag,
                 "/bin/bash",
                 "-c",
-                "find /usr/lib -name 'libsynapse*.so*' -exec cp -a {} /out/ \\;",
+                extract_script,
             ]
 
             subprocess.run(docker_cmd, check=True)
 
+            # Extract QNN runtime libraries → /usr/lib/ (dlopen'd by absolute path)
+            docker_cmd_qnn = [
+                "docker",
+                "run",
+                "--rm",
+                "--platform",
+                platform_opt,
+                "-v",
+                f"{qnn_dst_dir}:/out",
+                image_tag,
+                "/bin/bash",
+                "-c",
+                "find /usr/lib -maxdepth 1 -name 'libQnn*.so' -exec cp -a {} /out/ \\; && "
+                "if [ -d /usr/lib/rfsa ]; then cp -a /usr/lib/rfsa /out/; fi",
+            ]
+
+            subprocess.run(docker_cmd_qnn, check=True)
+
         except subprocess.CalledProcessError as exc:
             console.print(
                 f"[bold red]Error:[/bold red] Failed to copy SDK libraries from Docker image: {exc}"