Skip to content

Feature/integrations pt1 integrations#193

Open
PaperMtn wants to merge 63 commits intogoogle:mainfrom
PaperMtn:feature/integrations-pt1-integrations
Open

Feature/integrations pt1 integrations#193
PaperMtn wants to merge 63 commits intogoogle:mainfrom
PaperMtn:feature/integrations-pt1-integrations

Conversation

@PaperMtn
Copy link
Copy Markdown
Contributor

Implement All Integration Management Functions - Pt 1: Integration functions & Helpers

Summary

Part of a larger PR to implement all SOAR integration management capabilities to the SecOps SDK. This PR includes:

  • Integration functions
  • Helper functions
  • Models needed for future functions

New SDK Modules (src/secops/chronicle/integration/)

Module Description
integrations.py List, get, create, delete, download, export, diff, transition (staging/production), and manage integrations
integration_instances.py CRUD, test execution, affected items, and default instance retrieval for integration instances
marketplace_integrations.py List, get, diff, install, and uninstall marketplace integrations

New Utility Helpers (src/secops/chronicle/utils/)

New helpers added for consistency and deduplication:

  • Unit tests added for each

format_utils.py

  • format_resource_id() — Extracts the final ID segment from a full Chronicle resource path (e.g. projects/.../instances/.../abc123abc123). Safely passes through plain IDs.
    • Allows users to pass either the full resource name or just the ID to functions
  • parse_json_list() — Accepts either a list[dict] or a JSON string and returns a parsed list. Used for parameter inputs (e.g. --parameters '[{"name":"p1"}]').
  • build_patch_body() — Builds a PATCH request body and updateMask query param from a list of (api_key, mask_key, value) tuples. Auto-generates the update mask from non-None fields, or accepts an explicit override.

request_utils.py

  • chronicle_request_bytes() — New helper for endpoints that return raw bytes (used by download_integration and download_integration_dependency).
    • Allows expansion into API functions that download objects

New Data Models (src/secops/chronicle/models.py)

Enums:

  • PythonVersionV2_7, V3_7, V3_11
  • DiffTypeCOMMERCIAL, PRODUCTION, STAGING
  • TargetModePRODUCTION, STAGING
  • IntegrationTypeRESPONSE, EXTENSION
  • IntegrationParamTypeBOOLEAN, INT, STRING, PASSWORD, IP, URL, DOMAIN, EMAIL, VALUES_LIST, etc.
  • ActionParamTypeSTRING, BOOLEAN, WFS_REPOSITORY, USER_REPOSITORY, STAGES_REPOSITORY
  • ActionTypePING, CONNECTOR, JOB, etc.
  • ParamType — For connector parameters
  • ConnectorParamModeREQUIRED, OPTIONAL, etc.
  • ConnectorRuleType — Rule types for connector definitions
  • ScheduleTypeONE_TIME, DAILY, WEEKLY, MONTHLY, ADVANCED
  • DayOfWeekMONDAY through SUNDAY
  • IntegrationParameterType — For integration instance parameters
  • ConnectorConnectivityStatusCONNECTED, DISCONNECTED, etc.
  • TransformerType — Transformer definition types
  • LogicalOperatorType — Logical operator definition types

Dataclasses:

  • IntegrationParam — Integration-level parameter definition
  • ActionParameter — Action parameter with type and validation
  • ConnectorParameter — Connector parameter with mode and rules
  • ConnectorRule — Connector rule definition
  • IntegrationJobInstanceParameter — Job instance parameter
  • Date, TimeOfDay — Date/time components for scheduling
  • OneTimeScheduleDetails, DailyScheduleDetails, WeeklyScheduleDetails, MonthlyScheduleDetails — Schedule configuration
  • AdvancedConfig — Advanced scheduling (cron-like)
  • JobParameter — Job definition parameter
  • IntegrationInstanceParameter — Integration instance parameter
  • ConnectorInstanceParameter — Connector instance parameter
  • TransformerDefinitionParameter — Transformer parameter
  • IntegrationLogicalOperatorParameter — Logical operator parameter

CLI Commands (src/secops/cli/commands/integration/)

Added a complete secops integration CLI command tree with 21 subcommand modules, all registered via integration_client.py:

CLI Subcommand Operations
secops integration integrations list, get, create, delete, download, download-dependency, export-items, get-affected-items, get-agent, get-diff, get-dependencies, get-restricted-agents, transition, update, update-custom
secops integration instances list, get, create, update, delete, test, get-affected-items, get-default
secops integration marketplace list, get, diff, install, uninstall

Tests

Unit Tests - SDK Modules (tests/chronicle/integration/)

  • test_integrations.py
  • test_integration_instances.py
  • test_marketplace_integrations.py

Unit Tests - Utility Helpers (tests/chronicle/utils/)

  • test_format_utils.py -Tests for format_resource_id, parse_json_list, build_patch_body
  • test_request_utils.py - Tests for chronicle_request_bytes and updated request helpers

Documentation Updates

README.md

SDK usage examples for every new module, following the established format:

  • Integration management (list, get, create, delete, download, export, diff, transition)
  • Integration Instances (CRUD, test, affected items, default instance)
  • Marketplace Integrations (list, get, diff, install, uninstall)

CLI.md

CLI usage documentation with bash examples for all integration subcommands, including workflow examples for revision-based safe updates.

api_module_mapping.md

  • Updated endpoint count: v1: 17, v1beta: 88, v1alpha: 203
  • Added CLI command column entries for all implemented integration endpoints across v1beta and v1alpha
  • Added all new endpoint-to-module mappings

ChronicleClient Updates

  • client.py - All new integration module functions added as ChronicleClient methods with full docstrings
  • __init__.py - All new functions exported in __all__ and publicly importable from secops.chronicle

API Version Support

All integration modules support both v1beta (default) and v1alpha via the api_version parameter

@PaperMtn
Copy link
Copy Markdown
Contributor Author

It's a big design decision, but I was thinking something similar when I was putting this together. The more that gets added, the bigger ChronicleClient is going to get. There are going to be other design decisions made at some point I guess as the project grows, such as can the documentation live in the README

Fow now, there will be other implications with helpers, documentation etc. as well

Do you want me to put something together and refactor these functions into it to see what it looks like?

@mihirvala08
Copy link
Copy Markdown
Collaborator

It's a big design decision, but I was thinking something similar when I was putting this together. The more that gets added, the bigger ChronicleClient is going to get. There are going to be other design decisions made at some point I guess as the project grows, such as can the documentation live in the README

Fow now, there will be other implications with helpers, documentation etc. as well

Do you want me to put something together and refactor these functions into it to see what it looks like?

Another approach I see to reduce burden from ChronicleClient is to have SOAR namespace. And SOAR service (namespace) will have all soar methods. below is example:

class SoarService:
     """Namespace for all SOAR-related operations in Google SecOps."""

     def __init__(self, client: "ChronicleClient"):
         self._client = client
         
     def list_cases(
         self,
         page_size: Optional[int] = None,
         page_token: Optional[str] = None,
         **kwargs
     ) -> Dict[str, Any]:
         """List SOAR cases."""
         # Delegate to the underlying implementation, passing the parent client
         return _list_cases(self._client, page_size=page_size, page_token=page_token, **kwargs)
from secops.chronicle.soar import SoarService

class ChronicleClient:
    def __init__(self, project_id: str, customer_id: str, ...):
        # ... existing auth and URL setup ...
        self._session = session
        self.base_url = ...

        # Attach the SOAR namespace
        self.soar = SoarService(self)

And user can access soar method as : chronicle_client.soar.list_integration(args)

@PaperMtn
Copy link
Copy Markdown
Contributor Author

I agree that this is probably the best appraoch, and avoids having users create a new client.

I've refactored this PR into a SOARService that is a namespace feature and has the Integration functions under it.

@mihirvala08
Copy link
Copy Markdown
Collaborator

@PaperMtn, would it be possible to create FR/Issue with endpoints that are implemented for these PRs (all parts)?

It would help to have centralized discussion on common approach and reviews.

Thanks!

@PaperMtn
Copy link
Copy Markdown
Contributor Author

PaperMtn commented Apr 2, 2026

I've created a FR: [#207]

PaperMtn added 12 commits April 3, 2026 07:08
* main:
  test: expand auth error detection in case CLI integration tests
# By Mihir Vala (13) and others
# Via Mihir Vala (5) and GitHub (4)
* main: (21 commits)
  chore:minor error log formatting
  chore: fixed feed integration tests
  chore: bump version and added changelog
  chore: minor refactoring and improvements
  chore: fixed list methods for integration tests.
  feat: implement remove_none_values helper
  feat: add remove_none_values helper
  chore: refactor to use formatting helpers and param building pattern
  chore: update test cases for format_id helper usage
  chore: fixed unit tests for new request utils
  refactor: remove explicit api_version parameters from chronicle request calls and added as_list.
  feat: refactor remaining modules to use request helpers.
  chore: minor changelog refactor
  chore: bump version to 0.38.0
  chore: bump version to 0.36.0 and update changelog. Added local config doc.
  chore: add scope parameter to config load/save and improve argument handling. added view local config support.
  Upgrade GitHub Actions for Node 24 compatibility
  Upgrade GitHub Actions to latest versions
  chore: linting fix
  feat(cli): add SECOPS_LOCAL_CONFIG_DIR env var support
  ...

# Conflicts:
#	src/secops/chronicle/utils/format_utils.py
#	tests/chronicle/utils/test_format_utils.py
@PaperMtn PaperMtn requested a review from mihirvala08 April 3, 2026 19:17
# By Mihir Vala (9) and Isha Shree (2)
# Via Mihir Vala (2) and GitHub (1)
* main:
  chore: fix unit tests
  chore: minor refactoring and formatting
  chore: added docs in README and CLI. Added changelog. Updated project version.
  chore: added client integration tests
  chore: fixed unit tests
  chore: case integration tests fix
  chore: fixed unit tests
  chore: refactoring and improvements
  chore: fixed unnessary changes
  Addressing comments
  Adding new APIs in cli

# Conflicts:
#	README.md
#	api_module_mapping.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants