Skip to content

fix: Improve the error message when Application Default Credentials are not configured #173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tim-u wants to merge 3 commits into
base: main
Choose a base branch
from
Open

fix: Improve the error message when Application Default Credentials are not configured #173

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9007ab4
Improve the error message when User Default Credentials are not provided
tim-u Dec 16, 2025
9348464
Improve the error message when User Default Credentials are not provided
tim-u Dec 16, 2025
7858875
Merge remote-tracking branch 'refs/remotes/origin/improve-missing-cre…
tim-u Dec 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions google/cloud/dataproc_spark_connect/session.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
PermissionDenied,
)
from google.api_core.future.polling import POLLING_PREDICATE
from google.auth.exceptions import DefaultCredentialsError
from google.cloud.dataproc_spark_connect.client import DataprocChannelBuilder
from google.cloud.dataproc_spark_connect.exceptions import DataprocSparkConnectException
from google.cloud.dataproc_spark_connect.pypi_artifacts import PyPiArtifacts
Expand Down Expand Up @@ -456,6 +457,15 @@ def create_session_pbar():
raise DataprocSparkConnectException(
f"Error while creating Dataproc Session: {e.message}"
)
except DefaultCredentialsError as e:
Copy link
Contributor

@medb medb Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we check whether this will fix also #142 issue?

stop_create_session_pbar_event.set()
if create_session_pbar_thread.is_alive():
create_session_pbar_thread.join()
DataprocSparkSession._active_s8s_session_id = None
DataprocSparkSession._active_session_uses_custom_id = False
Comment on lines +461 to +465
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This cleanup logic is duplicated in three different except blocks (this one, the one for InvalidArgument/PermissionDenied, and the generic Exception one). To improve maintainability and reduce redundancy, consider factoring this logic out. You could, for example, use a single except Exception as e: block that contains the cleanup logic and then uses isinstance checks to raise the appropriate specific exception.

raise DataprocSparkConnectException(
f"Error while creating Dataproc Session: {e}"
Copy link
Contributor

@medb medb Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need to add a link to the ADC docs so users can fix it up more easily: https://docs.cloud.google.com/docs/authentication/provide-credentials-adc

)
except Exception as e:
stop_create_session_pbar_event.set()
if create_session_pbar_thread.is_alive():
Expand Down
10 changes: 10 additions & 0 deletions tests/unit/test_session.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1487,6 +1487,16 @@ def test_create_session_without_location(self):
except DataprocSparkConnectException as e:
self.assertIn("location is not set", str(e))

def test_create_session_without_application_default_credentials(self):
"""Tests that an exception is raised when application default credentials is not provided."""
os.environ.clear()
try:
DataprocSparkSession.builder.location("test-region").projectId(
"test-project"
).getOrCreate()
except DataprocSparkConnectException as e:
self.assertIn("Your default credentials were not found", str(e))
Comment on lines +1493 to +1498
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For better readability and to follow common testing patterns, consider using the assertRaises context manager to test for exceptions. This makes the intent of the test clearer.

        with self.assertRaises(DataprocSparkConnectException) as e:
            DataprocSparkSession.builder.location("test-region").projectId(
                "test-project"
            ).getOrCreate()
        self.assertIn("Your default credentials were not found", str(e.exception))



class DataprocSparkConnectClientTest(unittest.TestCase):

Expand Down