Jobs: Overview

Modified on Fri, 22 Nov at 2:50 PM

Jobs: Overview

Introduction

Jobs is a feature of DSim Cloud that enables you to automate simulations. A job can define any number of commands to perform compliation and simulation in the cloud. A job is defined in the local workspace of your local disk and is executed in the remote workspace of the remote disk. The local workspace is the contents of the folder where you initialized a local workspace by executing mdc initialize --local-only.

All the files needed for your job must be within your local workspace. The entire local workspace is automatically synchronized to the remote workspace /remote/work everytime you execute mdc job submit. If you want certain files to not be synchronized, modify the file mdc_ignore, as described in the section Ignoring File Patterns of DSim Cloud File Synchronization.

See Understanding Workspaces for details.

Unlike Interactive workspaces, you do not need to start a remote workspace to use jobs. Remote cloud resources will be allocated and managed for you automatically when jobs are submitted and executed.

Job File

Jobs are created with job files, which are YAML files following YAML conventions. For example, let's examine carry_lookahead_adder.yml, shown below and can be downloaded from the carry_lookahead_adder design. Note the indentation and that all syntax is case-sensitive.

Sample Job File carry_lookahead_adder.yml

name: carry-lookahead-adder
keep_for_support: false
tasks:
  - name: compile-sv
    depends: [sim-vhdl-sv]
    commands:
      - dsim -genimage image_sv -F SystemVerilog/sim/filelist.txt +acc+b
    inputs:
      working:
        - name: sim-vhdl-sv.source
          path: ./
    outputs:
      artifacts:
        - name: log-sv
          path: dsim.log
      working:
        - name: image-sv
          path: dsim_work/image_sv.so
  - name: sim-vhdl
    depends: [run-sv-3, sim-vhdl-sv]
    commands:
      - dvhcom -F VHDL/sim/filelist.txt
      - dsim -top work.carry_lookahead_adder_tb +acc+b -waves waves_vhdl.mxd
    inputs:
      working:
        - name: sim-vhdl-sv.source
          path: ./
    outputs:
      artifacts:
        - name: waves-vhdl
          path: waves_vhdl.mxd
        - name: log-vhdl
          path: dsim.log
  - name: sim-vhdl-sv
    compute_size: s8
    mdc_work: VHDL_SystemVerilog/sim/
    commands:
      - dvhcom -F filelist_VHDL.txt
      - dvlcom -F filelist_SV.txt
      - dsim -top work.carry_lookahead_adder_tb +acc+b -waves waves_vhdl_sv.mxd
    outputs:
      working:
        - name: source
          path: ./
      artifacts:
        - name: waves-vhdl-sv
          path: VHDL_SystemVerilog/sim/waves_vhdl_sv.mxd
        - name: log-vhdl-sv
          path: VHDL_SystemVerilog/sim/dsim.log
  - name: run-sv-1
    depends: [compile-sv]
    commands:
      - dsim -image image_sv -waves waves_sv_1.mxd -sv_seed 1
    inputs:
      working:
        - name: compile-sv.image-sv
          path: dsim_work/image_sv.so
    outputs:
      artifacts:
        - name: waves-sv-1
          path: waves_sv_1.mxd
        - name: log-sv-1
          path: dsim.log
  - name: run-sv-2
    depends: [compile-sv]
    commands:
      - dsim -image image_sv -waves waves_sv_2.mxd -sv_seed 2
    inputs:
      working:
        - name: compile-sv.image-sv
          path: dsim_work/image_sv.so
    outputs:
      artifacts:
        - name: waves-sv-2
          path: waves_sv_2.mxd
        - name: log-sv-2
          path: dsim.log
  - name: run-sv-3
    depends: [compile-sv]
    commands:
      - dsim -image image_sv -waves waves_sv_3.mxd -sv_seed 3
    inputs:
      working:
        - name: compile-sv.image-sv
          path: dsim_work/image_sv.so
    outputs:
      artifacts:
        - name: waves-sv-3
          path: waves_sv_3.mxd
        - name: log-sv-3
          path: dsim.log

Job File Properties

name: carry-lookahead-adder

This is the name of the job. It does not have to match the job file name. Every time you submit the job file to run the job, a new auto-generated hash will be added to the name to form the unique job-id. The value must be made up of no whitespace, lowercase letters, numbers and '-' characters, but it cannot start or end with a '-'.

keep_for_support: false

Normally, the files synced from your local workspace to the remote workspace are removed once the job finishes. If Metrics Support requires access to these files to help you, you can set this optional property to true to save the files for customer support access only. To disable this, either set the property to false, or omit the property altogether.

tasks:  
  - name:

A job is comprised of tasks, which are executed in the remote workspace. The status of these tasks is obtained with the job-id. If there is an error in executing a task, all the following tasks are omitted and not executed. See Jobs: Tasks for details. The value must be made up of no whitespace, uppercase and lowercase letters, numbers, and '-' characters, but it cannot start with a '-'.

compute_size:

The remote cloud compute configuration for the named task. Current options are s4 and s8. If this optional line is absent, the task will be executed on an s4.

commands:

The following commands can be executed in the remote workspace in the order they appear in your task:

DSim simulator commands
mkdir to create directories
ls to list files and folders in the remote workspace directory
envs to list out all the environment variables (without mdc)

Note that each command is preceded by a -.

mdc_work:

The path from where the commands should be executed, relative to the root of the local workspace. If this optional line is absent, all commands are executed from the root of the remote workspace. Directory traversal is not permissable and the value cannot include '..'.

outputs:
  artifacts:
    - name: log-sv
      path: dsim.log
  working:
    - name: image-sv
      path: dsim_work/image_sv.so

Tasks can have outputs - artifact outputs and/or working outputs. Artifact outputs are outputs saved on the remote for access after the job is complete, such as waveform files, simulation logs, and folders. Each artifact should have a unique name and path. The path is relative to the remote work directory. Artifacts can be downloaded using mdc job download. The file or folder will be compressed into a tar file before being downloaded to \<artifact-name>.tar.gz.

Working outputs are outputs that are only used for the duration of the job and are removed from the remote upon job completion. They define files for use in subsequent tasks, such as image files or source code. Each working output should have a unique name and path. The path is relative to the remote work directory. Working output names must be alphanumeric and may include '-', and '_'. See Task Dependencies for details about working outputs.

depends:

If a task has dependencies, it must have this property. See Task Dependencies for details.

inputs:

Tasks with dependencies can have inputs, which are outputs from previously executed tasks. Tasks without dependencies do not have inputs since they have access to all the files from the local workspace upon job submission. See Task Dependencies for details. The name of the input artifact must be alphanumeric and may include '-', '_', and '.'.

Job File Properties Notation

In the user documentation, job file properties are described using the dot (.) notation. The dot notation shows the path to, or hierarchy of, the property. For example, in the above job file, the task compile-sv has these properties:

One working input name inputs.working.name is sim-vhdl-sv.source.
One artifact output name outputs.artifacts.name is log-sv with a path outputs.artifacts.path of dsim.log.
One working output name outputs.working.name is image-sv with a path outputs.working.path of dsim_work/image_sv.so.

Predefined variables are available to use in the following job file properties:

commands
input path inputs.working.path
output working path outputs.working.path
output artifacts path outputs.artifacts.path

The variables can be referenced as $( variable ). Currently, variable can be:

mdc_work: value of computed mdc_work
compute_size: value of computed compute_size

For example, in your job file:

tasks:
  - name: library
    mdc_work: tb/axi2mba/work.regress
...
    outputs:
      working:
        - name: work-dsim_work
          path: tb/axi2mba/work.regress/dsim_work/
      artifacts:
        - name: dvlcom-log-1
          path: $( mdc_work )/dvlcom-1.log
...

For more information on YAML, see The Official YAML Web Site.

Using Job Files

Create job files as YAML files with the extension .yml or .yaml using the above template. Put them anywhere in your local workspace directory. Use job commands to manage your jobs for regressions. Use job files for your regressions as shown in Tutorial: Regressions.