Job Enhancements

Overview

Yeedu jobs now include enhanced file management, improved path handling, and per-job YCU (Yeedu Compute Units) cost tracking.

File Picker Enhancements

Upload Files

Upload files directly during job creation:

1. In job configuration, find Choose file
2. Click Upload button
3. Select file/folder from computer
4. File/folder uploaded to workspace files
5. Reference in job configuration

Create Folder

Create folder in file picker:

1. Click Create Folder button
2. Enter folder name
3. Folder created in workspace files
4. Use to organize job files

Copy Path

Quick path copying for easy reference:

1. Right-click on file/folder
2. Select Copy Path
3. Path copied to clipboard:
   • /workspace/path/to/file.txt
4. Paste the copy path after clicking on custom path
5. Click on save and it is added to job configuration

Custom Path Support

Use flexible path definitions for jobs, including workspace variables, absolute targets, and date-based dynamic paths.

• Path variables

  • ${WORKSPACE_PATH}/data/input
  • ${TEMP_PATH}/outputs
  • ${CLUSTER_PATH}/scripts

• Absolute paths

  • s3://my-bucket/data/input.parquet
  • /mnt/shared-storage/files

• Dynamic paths with DateFormat

  • /data/${YEAR}/${MONTH}/${DAY}/input.csv
  • /archive/${DATE}/${JOBID}/output.parquet

YCU Per-Job Tracking

Approx YCU Display

See estimated compute cost per job:

1. Go to Runs
2. Find Approx YCU column
3. Shows approximate cost (YCU units)
4. Represents resource consumption

Import Job Enhancements

Mandatory Language Field

When importing data, language is now required:

1. In import job configuration
2. Language field is mandatory
3. Select:
   • Python: For Python transformations
   • SQL: For SQL queries
   • RAW_SCALA: For Scala operations and raw Scala code
   • Jar: For Java/Scala packaged JAR jobs

Why Language Required

• Proper syntax highlighting
• Type checking
• Optimization
• Error detection
• Library resolution