Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

blank_issues_enabled: false
Expand Down
2 changes: 1 addition & 1 deletion .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

# To get started with Dependabot version updates, you'll need to specify which
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/autoupdate.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

name: autoupdate
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/codacy.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

# This workflow uses actions that are not certified by GitHub.
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/dependabot.autoapprove.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

name: Auto approve
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/dependabot.automerge.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

name: automerge
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/format.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

name: Google Java Format
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pr-title-checker.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

name: "PR Title Checker"
Expand Down
2 changes: 1 addition & 1 deletion .gitlab-ci.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

# default image used by all pipelines if not overridden per job
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

repos:
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,9 @@ vdk --help</code></pre>
<a href="#contributing"><img src="https://img.shields.io/badge/Support%20and%20Contributing-blue" alt="Support and Contributing"></a>
</p>

# Notebooks
- [Processing data using SQL and local database](https://colab.research.google.com/github/vmware/versatile-data-kit/blob/main/examples/notebook-examples/sql-and-local-database.ipynb)

# Support and Contributing
For Support, you can join our Slack channel, create an [issue](https://github.com/vmware/versatile-data-kit/issues) or [pull request](https://github.com/vmware/versatile-data-kit/pulls) on GitHub to submit suggestions or changes. <br>
If you are interested in contributing as a developer, visit the [contributing](https://github.com/vmware/versatile-data-kit/blob/main/CONTRIBUTING.md) page.
Expand Down
2 changes: 1 addition & 1 deletion cicd/.gitlab-ci-lib.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

# This is a shared library with common util gitlab "jobs" that can be reused across any project
Expand Down
2 changes: 1 addition & 1 deletion cicd/build.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash -e
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
Expand Down
2 changes: 1 addition & 1 deletion cicd/docker_parse.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash

# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

# Parses a Docker image name and extracts information such as the registry, username, repository, and tag.
Expand Down
2 changes: 1 addition & 1 deletion cicd/docker_push_vdk.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env bash

# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

# This Bash script provides a utility to push a containers image to multiple container registries.
Expand Down
2 changes: 1 addition & 1 deletion cicd/notify.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash -e
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0

curl -X POST -H "Content-type: application/json" --data "{\"text\":\"$1\"}" $2
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from vdk.api.job_input import IJobInput

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import json
import pathlib
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import json
import pathlib
Expand Down
2 changes: 1 addition & 1 deletion examples/airflow-example/dags/airflow_example_dag.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from datetime import datetime

Expand Down
2 changes: 1 addition & 1 deletion examples/chunker/00_properties.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
# Copyright 2021-2024 VMware, Inc.
# SPDX-License-Identifier: Apache-2.0
Expand Down
2 changes: 1 addition & 1 deletion examples/chunker/10_chunk_data.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
# Copyright 2021-2024 VMware, Inc.
# SPDX-License-Identifier: Apache-2.0
Expand Down
2 changes: 1 addition & 1 deletion examples/chunker/config.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
# Copyright 2021-2024 VMware, Inc.
# SPDX-License-Identifier: Apache-2.0
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import logging
import os
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import logging
import os
Expand Down
2 changes: 1 addition & 1 deletion examples/confluence-reader/confluence_document.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0


Expand Down
2 changes: 1 addition & 1 deletion examples/confluence-reader/fetch_confluence_space.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import logging
import os
Expand Down
2 changes: 1 addition & 1 deletion examples/dag-example/example-dag/example_dag.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from vdk.plugin.dag.dag_runner import DagInput

Expand Down
2 changes: 1 addition & 1 deletion examples/dag-example/ingest-job1/10_insert_data.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import json
import pathlib
Expand Down
2 changes: 1 addition & 1 deletion examples/dag-example/ingest-job2/10_insert_data.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import json
import pathlib
Expand Down
2 changes: 1 addition & 1 deletion examples/dag-example/read-data-job/10_read.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from vdk.api.job_input import IJobInput

Expand Down
2 changes: 1 addition & 1 deletion examples/dag-with-args-example/dag-job/dag_job.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from vdk.plugin.dag.dag_runner import DagInput

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import json
import pathlib
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import json
import pathlib
Expand Down
2 changes: 1 addition & 1 deletion examples/dag-with-args-example/read-job-canada/10_read.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from vdk.api.job_input import IJobInput

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from vdk.api.job_input import IJobInput

Expand Down
2 changes: 1 addition & 1 deletion examples/dag-with-args-example/read-job-usa/10_read.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from vdk.api.job_input import IJobInput

Expand Down
2 changes: 1 addition & 1 deletion examples/energy/dashboard.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import math

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import inspect
import logging
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import inspect
import logging
Expand Down
2 changes: 1 addition & 1 deletion examples/gdp-execution-id-example/20_python_step.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import logging

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from vdk.api.job_input import IJobInput

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import logging

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import pathlib

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import json
import logging
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import hashlib
from typing import Any
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import logging
from typing import Any
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
from vdk.api.job_input import IJobInput

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import os
from unittest import mock
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import sqlite3

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import requests

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import csv

Expand Down
2 changes: 1 addition & 1 deletion examples/life-expectancy/dashboard.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import math

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import inspect
import logging
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import inspect
import logging
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import inspect
import logging
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import inspect
import logging
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import inspect
import logging
Expand Down
2 changes: 1 addition & 1 deletion examples/online-exhibition/view.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import math

Expand Down
2 changes: 1 addition & 1 deletion examples/pgvector-embedder/00_properties.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import os.path
import pathlib
Expand Down
2 changes: 1 addition & 1 deletion examples/pgvector-embedder/01_setup_huggingface.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
# Copyright 2021-2024 VMware, Inc.
# SPDX-License-Identifier: Apache-2.0
Expand Down
2 changes: 1 addition & 1 deletion examples/pgvector-embedder/20_embed_data.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
# Copyright 2021-2024 VMware, Inc.
# SPDX-License-Identifier: Apache-2.0
Expand Down
2 changes: 1 addition & 1 deletion examples/pgvector-embedder/40_ingest_embeddings.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
import logging
import pickle
Expand Down
2 changes: 1 addition & 1 deletion examples/pgvector-embedder/common/database_storage.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
# Copyright 2021-2024 VMware, Inc.
# SPDX-License-Identifier: Apache-2.0
Expand Down
2 changes: 1 addition & 1 deletion examples/pgvector-embedder/common/file_storage.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023-2025 Broadcom
# Copyright 2023-2026 Broadcom
# SPDX-License-Identifier: Apache-2.0
# Copyright 2021-2024 VMware, Inc.
# SPDX-License-Identifier: Apache-2.0
Expand Down
Loading
Loading