Skip to main content
Version: v2.10.0

Using Secrets in Yeedu

Overview​

Secrets provide a secure way to manage credentials and sensitive information across different components in Yeedu. This guide demonstrates how to use secrets with metastores, notebooks, and jobs.

Using Secrets in Metastores​

How It Works​

Secrets can be attached to metastores to securely connect to external data sources without exposing credentials in configurations.

Connection Flow:

  1. Create or select an existing secret containing connection credentials.
  2. Attach the secret to a metastore configuration.
  3. The metastore uses the secret's credentials for secure remote access.

Metastore

Benefits​

  • No Credentials in Configs – Connection details remain hidden from code and configuration files.
  • Secure External Access – Credentials are securely stored and only accessed when needed.
  • Centralized Management – Update credentials in one place without modifying metastore settings.

Best Practice​

Attach secrets to metastores when connecting to databases, data warehouses, or cloud storage services.

Using Secrets in Notebooks​

How It Works​

Notebooks access secrets through attached metastores and clusters:

  1. Secrets are attached to the metastore.
  2. The metastore is attached to a cluster.
  3. The notebook runs on that cluster and gains access to the secrets.

Metastore Notebook

Benefits​

  • No Hardcoding – Credentials are never written in notebook code.
  • Secure Execution – Notebooks access credentials only at runtime.
  • Easy Credential Rotation – Update secrets without modifying notebooks.

Using Secrets in Notebooks​

When accessing external data sources in a notebook:

  1. Reference the metastore attached to your cluster.
  2. The metastore provides secure access to the secret credentials.
  3. Execute queries or data operations using the secure connection.

Using Secrets in Jobs​

How It Works​

Jobs access secrets through the same secure mechanism as notebooks:

  1. Secrets are attached to the metastore.
  2. The metastore is attached to a cluster.
  3. The job runs on that cluster with secure access to secrets.

Metastore Job

Benefits​

  • Secure Automation – Jobs execute with encrypted credential access.
  • No Exposure in Configs – Job configurations remain credential-free.
  • Audit Trail – Secret access is logged for compliance and security monitoring.

Using Secrets in Jobs​

When configuring jobs that require external credentials:

  1. Ensure the cluster has the necessary metastore attached.
  2. The metastore contains the attached secrets.
  3. The job automatically uses these credentials during execution.

Security Considerations​

Access Control​

  • Follow the principle of least privilege—attach only necessary secrets to metastores.
  • Review which jobs and notebooks have access to sensitive credentials.

Credential Rotation​

  • Regularly update secrets to maintain security.
  • Changes take effect immediately for new operations.

Monitoring and Compliance​

  • Monitor secret usage in audit logs.
  • Ensure compliance with your organization's security policies.