Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
52a4f56
SC-5131:Update semgrep.yml (#159)
Revanthathreya Feb 24, 2025
60c46c1
SC-5790:Create Gitleaks.yml
Revanthathreya Jun 10, 2025
ed57ecb
SC-5790:Update and rename customRule.yml to gitleaks.toml
Revanthathreya Jun 10, 2025
16c0259
Merge pull request #183 from skyflowapi/Revanthathreya-patch-2
skyflow-bharti Jun 17, 2025
61e368f
SK-971 file upload support (#191)
raushan-skyflow Sep 10, 2025
3cafb58
[AUTOMATED] Private Release 2.1.0b1.dev0+61e368f
raushan-skyflow Sep 10, 2025
dc258e0
SK-2292 retry on connection
skyflow-bharti Sep 12, 2025
adafc1c
SK-2292 set default retry
skyflow-bharti Sep 12, 2025
b3d791c
SK-2293 max retry set to 3
skyflow-bharti Sep 12, 2025
3313c54
Merge pull request #195 from skyflowapi/SK-2293-retry-records-on-conn…
skyflow-bharti Sep 12, 2025
bb6c874
[AUTOMATED] Public Release - 1.15.2
skyflow-bharti Sep 12, 2025
5700874
SK-2293 retry on every exception
skyflow-bharti Sep 12, 2025
8aa2871
Merge pull request #196 from skyflowapi/SK-2293-retry-records-on-conn…
skyflow-bharti Sep 12, 2025
3b47b5c
[AUTOMATED] Public Release - 1.15.3
skyflow-bharti Sep 12, 2025
922f98c
SK-2293 retry on every exception
skyflow-bharti Sep 12, 2025
f895680
Merge pull request #197 from skyflowapi/SK-2293-retry-records-on-conn…
skyflow-bharti Sep 12, 2025
450db9d
[AUTOMATED] Public Release - 1.15.4
skyflow-bharti Sep 12, 2025
8e1d653
SK-2270 fix inconsistencies in python SDK V2 (#193)
raushan-skyflow Sep 17, 2025
aaa18c7
[AUTOMATED] Private Release 1.15.4.dev0+8e1d653
raushan-skyflow Sep 17, 2025
57d594b
SK-2296: Release/25.9.14 (#200)
skyflow-shravan Sep 18, 2025
d2979c3
[AUTOMATED] Public Release - 1.15.5
skyflow-shravan Sep 18, 2025
dab7b66
SK-2270: revert the fern generated detect changes (#201)
raushan-skyflow Sep 19, 2025
bdcc5ac
[AUTOMATED] Private Release 1.15.5.dev0+dab7b66
raushan-skyflow Sep 19, 2025
3bb2b6a
[AUTOMATED] Public Release - 1.15.6
skyflow-shravan Sep 22, 2025
c117285
SK-2313 Release/25.9.15 (#203)
skyflow-shravan Sep 22, 2025
ada8b74
SK-2131 minor fix for remote disconnect error (#204)
skyflow-shravan Sep 23, 2025
b4bd129
[AUTOMATED] Public Release - 1.16.0
skyflow-shravan Sep 23, 2025
10350bc
SK-2131 retry for errors in insert (#205)
skyflow-shravan Sep 23, 2025
abd48ff
[AUTOMATED] Public Release - 1.15.7
skyflow-shravan Sep 23, 2025
44e611c
SK-2325 Fix retry logic when continueOnError is set to true
skyflow-vivek Sep 29, 2025
cce9db4
Merge pull request #206 from skyflowapi/skyflow-vivek/SK-2325-fix-ret…
skyflow-vivek Sep 29, 2025
d6cbc68
[AUTOMATED] Private Release 1.15.7.dev0+cce9db4
skyflow-vivek Sep 29, 2025
b69c357
SK-2325 Added unit tests for fix
skyflow-vivek Sep 30, 2025
1c6553f
[AUTOMATED] Private Release 1.15.7.dev0+b69c357
skyflow-vivek Sep 30, 2025
f3c530e
SK-2325 Updated new env secret in workflows
skyflow-vivek Sep 30, 2025
35916c2
[AUTOMATED] Private Release 1.15.7.dev0+f3c530e
skyflow-vivek Sep 30, 2025
069cf02
Merge pull request #207 from skyflowapi/release/25.9.16
skyflow-bharti Sep 30, 2025
706d0d2
[AUTOMATED] Public Release - 1.15.8
skyflow-vivek Sep 30, 2025
08b535a
SK-2353: Update generated code with updated openapi spec file. (#211)
saileshwar-skyflow Nov 6, 2025
ac51edf
[AUTOMATED] Private Release 1.15.8.dev0+08b535a
saileshwar-skyflow Nov 6, 2025
4c27775
SK-2353: fix indentation
saileshwar-skyflow Nov 6, 2025
9e85bba
[AUTOMATED] Private Release 1.15.8.dev0+4c27775
saileshwar-skyflow Nov 6, 2025
ba73553
SK-2385: add locations to the generated code
saileshwar-skyflow Nov 6, 2025
155a689
[AUTOMATED] Private Release 1.15.8.dev0+ba73553
saileshwar-skyflow Nov 6, 2025
3e794e3
SK-2385: update output transcriptions enum
saileshwar-skyflow Nov 7, 2025
342df8b
[AUTOMATED] Private Release 1.15.8.dev0+3e794e3
saileshwar-skyflow Nov 7, 2025
593ae32
SK-2385: Merge branch 'main' into release/25.9.1
saileshwar-skyflow Nov 7, 2025
a22b9c6
SK-1773 Add str for file upload response
skyflow-vivek Nov 11, 2025
255c8ec
[AUTOMATED] Private Release 1.15.8.dev0+a22b9c6
skyflow-vivek Nov 11, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/shared-build-and-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ jobs:
python -m pip install --upgrade pip
pip install setuptools wheel twine

- name: Build and install skyflow package
run: |
python setup.py sdist bdist_wheel
pip install dist/skyflow-*.whl

- name: Resolve Branch for the Tagged Commit
id: resolve-branch
if: ${{ inputs.tag == 'beta' || inputs.tag == 'public' }}
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/shared-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@ jobs:
with:
name: "credentials.json"
json: ${{ secrets.VALID_SKYFLOW_CREDS_TEST }}

- name: Build and install skyflow package
run: |
pip install --upgrade pip setuptools wheel
python setup.py sdist bdist_wheel
pip install dist/skyflow-*.whl

- name: 'Run Tests'
run: |
Expand Down
49 changes: 34 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ table_name = '<SENSITIVE_DATA_TABLE>' # Replace with your actual table name

# Create Insert Request
insert_request = InsertRequest(
table_name=table_name,
table=table_name,
values=insert_data,
return_tokens=True, # Optional: Get tokens for inserted data
continue_on_error=True # Optional: Continue on partial errors
Expand Down Expand Up @@ -273,7 +273,7 @@ options = InsertOptions(

```python
insert_request = InsertRequest(
table_name=table_name, # Replace with the table name
table=table_name, # Replace with the table name
values=insert_data,
return_tokens=False, # Do not return tokens
continue_on_error=False, # Stop inserting if any record fails
Expand Down Expand Up @@ -474,7 +474,7 @@ try:

# Step 2: Create Insert Request
insert_request = InsertRequest(
table_name='table1', # Specify the table in the vault where the data will be inserted
table='table1', # Specify the table in the vault where the data will be inserted
values=insert_data, # Attach the data (records) to be inserted
return_tokens=True, # Specify if tokens should be returned upon successful insertion
continue_on_error=True # Optional: Continue on partial errors
Expand Down Expand Up @@ -551,7 +551,7 @@ try:

# Step 2: Build an InsertRequest object with the table name and the data to insert
insert_request = InsertRequest(
table_name='<TABLE_NAME>', # Replace with the actual table name in your Skyflow vault
table='<TABLE_NAME>', # Replace with the actual table name in your Skyflow vault
values=insert_data, # Attach the data to be inserted
)

Expand Down Expand Up @@ -608,7 +608,7 @@ try:

# Step 4: Build the InsertRequest object with the data records to insert
insert_request = InsertRequest(
table_name='table1', # Specify the table in the vault where the data will be inserted
table='table1', # Specify the table in the vault where the data will be inserted
values=insert_data, # Attach the data (records) to be inserted
return_tokens=True, # Specify if tokens should be returned upon successful insertion
continue_on_error=True # Specify to continue inserting records even if an error occurs for some records
Expand Down Expand Up @@ -686,7 +686,7 @@ try:

# Step 3: Build the InsertRequest object with the upsertData
insert_request = InsertRequest(
table_name='table1', # Specify the table in the vault where the data will be inserted
table='table1', # Specify the table in the vault where the data will be inserted
values=insert_data, # Attach the data (records) to be inserted
return_tokens=True, # Specify if tokens should be returned upon successful insertion
upsert='cardholder_name' # Specify the field to be used for upsert operations (e.g., cardholder_name)
Expand Down Expand Up @@ -1897,23 +1897,24 @@ ReidentifyTextResponse(
```

### Deidentify File
To deidentify files, use the `deidentify_file` method. The `DeidentifyFileRequest` class creates a deidentify file request, which includes the file to be deidentified and various configuration options.
To deidentify files, use the `deidentify_file` method. The `DeidentifyFileRequest` class creates a deidentify file request, supports providing either a file or a file path in class FileInput for de-identification, along with various configuration options.

#### Construct a Deidentify File request
```python
from skyflow.error import SkyflowError
from skyflow.utils.enums import DetectEntities, MaskingMethod, DetectOutputTranscriptions
from skyflow.vault.detect import DeidentifyFileRequest, TokenFormat, Transformations, Bleep
from skyflow.vault.detect import DeidentifyFileRequest, TokenFormat, Transformations, Bleep, FileInput
"""
This example demonstrates how to deidentify file, along with corresponding DeidentifyFileRequest schema.
"""
try:
# Initialize Skyflow client
# Step 1: Open file for deidentification
file = open('<FILE_PATH>', 'rb') # Open the file in read-binary mode
file_path="<FILE_PATH>"
file = open(file_path, 'rb') # Open the file in read-binary mode
# Step 2: Create deidentify file request
request = DeidentifyFileRequest(
file=file, # File object to deidentify
file=FileInput(file), # File to de-identify (can also provide a file path)
entities=[DetectEntities.SSN, DetectEntities.CREDIT_CARD], # Entities to detect

# Token format configuration
Expand Down Expand Up @@ -1971,7 +1972,7 @@ except Exception as error:
```python
from skyflow.error import SkyflowError
from skyflow.utils.enums import DetectEntities, MaskingMethod, DetectOutputTranscriptions
from skyflow.vault.detect import DeidentifyFileRequest, TokenFormat, Bleep
from skyflow.vault.detect import DeidentifyFileRequest, TokenFormat, Bleep, FileInput
"""
* Skyflow Deidentify File Example
*
Expand All @@ -1985,7 +1986,7 @@ try:
file = open('sensitive_document.txt', 'rb') # Open the file in read-binary mode
# Step 2: Create deidentify file request
request = DeidentifyFileRequest(
file=file, # File object to deidentify
file=FileInput(file), # File to de-identify (can also provide a file path)
entities=[
DetectEntities.SSN,
DetectEntities.CREDIT_CARD
Expand Down Expand Up @@ -2038,7 +2039,6 @@ DeidentifyFileResponse(
],
run_id='83abcdef-2b61-4a83-a4e0-cbc71ffabffd',
status='SUCCESS',
errors=[]
)
```

Expand Down Expand Up @@ -2121,7 +2121,7 @@ except Exception as error:
print('Unexpected Error:', error) # Print the stack trace for debugging purposes
```

Sample Response
Sample Response:
```python
DeidentifyFileResponse(
file='TXkgY2FyZCBudW1iZXIgaXMgW0NSRURJVF9DQVJEXQpteSBzZWNvbmQ…', # Base64 encoded file content
Expand All @@ -2142,7 +2142,26 @@ DeidentifyFileResponse(
],
run_id='48ec05ba-96ec-4641-a8e2-35e066afef95',
status='SUCCESS',
errors=[]
)
```

Incase of invalid/expired RunId:

```python
DeidentifyFileResponse(
file_base64=None,
file=None,
type='UNKNOWN',
extension=None,
word_count=None,
char_count=None,
size_in_kb=0.0,
duration_in_seconds=None,
page_count=None,
slide_count=None,
entities=[],
run_id='1e9f321f-dd51-4ab1-a014-21212fsdfsd',
status='UNKNOWN'
)
```

Expand Down
4 changes: 2 additions & 2 deletions samples/detect_api/deidentify_file.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from skyflow.error import SkyflowError
from skyflow import Env, Skyflow, LogLevel
from skyflow.utils.enums import DetectEntities, MaskingMethod, DetectOutputTranscriptions
from skyflow.vault.detect import DeidentifyFileRequest, TokenFormat, Transformations, DateTransformation, Bleep
from skyflow.vault.detect import DeidentifyFileRequest, TokenFormat, Transformations, DateTransformation, Bleep, FileInput

"""
* Skyflow Deidentify File Example
Expand Down Expand Up @@ -39,7 +39,7 @@ def perform_file_deidentification():
file = open(file_path, 'rb')
# Step 5: Configure Deidentify File Request with all options
deidentify_request = DeidentifyFileRequest(
file=file, # File object to deidentify
file=FileInput(file), # File to de-identify (can also provide a file path)
entities=[DetectEntities.SSN, DetectEntities.CREDIT_CARD], # Entities to detect
allow_regex_list=['<YOUR_REGEX_PATTERN>'], # Optional: Patterns to allow
restrict_regex_list=['<YOUR_REGEX_PATTERN>'], # Optional: Patterns to restrict
Expand Down
2 changes: 1 addition & 1 deletion samples/vault_api/insert_byot.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ def perform_secure_data_insertion_with_byot():
]

insert_request = InsertRequest(
table_name=table_name,
table=table_name,
values=insert_data,
token_mode=TokenMode.ENABLE, # Enable Bring Your Own Token (BYOT)
tokens=tokens, # Specify tokens to use for BYOT
Expand Down
2 changes: 1 addition & 1 deletion samples/vault_api/insert_records.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ def perform_secure_data_insertion():

# Step 5: Create Insert Request
insert_request = InsertRequest(
table_name=table_name,
table=table_name,
values=insert_data,
return_tokens=True, # Optional: Get tokens for inserted data
continue_on_error=True # Optional: Continue on partial errors
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

if sys.version_info < (3, 8):
raise RuntimeError("skyflow requires Python 3.8+")
current_version = '2.1.0b1'
current_version = '1.15.8.dev0+a22b9c6'

setup(
name='skyflow',
Expand Down
Loading