Jobs: Overview

Introduction

Jobs is a feature of DSim Cloud that enables you to automate simulations. A job can define any number of commands to perform compliation and simulation in the cloud. A job is defined in the local workspace of your local disk and is executed in the remote workspace of the remote disk. The local workspace is the contents of the folder where you initialized a local workspace by executing mdc initialize --local-only.

All the files needed for your job must be within your local workspace. The entire local workspace is automatically synchronized to the remote workspace /remote/work everytime you execute mdc job submit. If you want certain files to not be synchronized, modify the file mdc_ignore, as described in the section Ignoring File Patterns of DSim Cloud File Synchronization.

See Understanding Workspaces for details.

Unlike Interactive workspaces, you do not need to start a remote workspace to use jobs. Remote cloud resources will be allocated and managed for you automatically when jobs are submitted and executed.

Job File

Jobs are created with job files, which are YAML files following YAML conventions. For example, let's examine carry_lookahead_adder.yml, shown below and can be downloaded from the carry_lookahead_adder design. Note the indentation and that all syntax is case-sensitive.

Sample Job File carry_lookahead_adder.yml

name: carry-lookahead-adder
keep_for_support: false
tasks:
  - name: compile-sv
    depends: [sim-vhdl-sv]
    commands:
      - dsim -genimage image_sv -F SystemVerilog/sim/filelist.txt +acc+b
    inputs:
      working:
        - name: sim-vhdl-sv.source
          path: ./
    outputs:
      artifacts:
        - name: log-sv
          path: dsim.log
      working:
        - name: image-sv
          path: dsim_work/image_sv.so
  - name: sim-vhdl
    depends: [run-sv-3, sim-vhdl-sv]
    commands:
      - dvhcom -F VHDL/sim/filelist.txt
      - dsim -top work.carry_lookahead_adder_tb +acc+b -waves waves_vhdl.mxd
    inputs:
      working:
        - name: sim-vhdl-sv.source
          path: ./
    outputs:
      artifacts:
        - name: waves-vhdl
          path: waves_vhdl.mxd
        - name: log-vhdl
          path: dsim.log
  - name: sim-vhdl-sv
    compute_size: s8
    mdc_work: VHDL_SystemVerilog/sim/
    commands:
      - dvhcom -F filelist_VHDL.txt
      - dvlcom -F filelist_SV.txt
      - dsim -top work.carry_lookahead_adder_tb +acc+b -waves waves_vhdl_sv.mxd
    outputs:
      working:
        - name: source
          path: ./
      artifacts:
        - name: waves-vhdl-sv
          path: VHDL_SystemVerilog/sim/waves_vhdl_sv.mxd
        - name: log-vhdl-sv
          path: VHDL_SystemVerilog/sim/dsim.log
  - name: run-sv-1
    depends: [compile-sv]
    commands:
      - dsim -image image_sv -waves waves_sv_1.mxd -sv_seed 1
    inputs:
      working:
        - name: compile-sv.image-sv
          path: dsim_work/image_sv.so
    outputs:
      artifacts:
        - name: waves-sv-1
          path: waves_sv_1.mxd
        - name: log-sv-1
          path: dsim.log
  - name: run-sv-2
    depends: [compile-sv]
    commands:
      - dsim -image image_sv -waves waves_sv_2.mxd -sv_seed 2
    inputs:
      working:
        - name: compile-sv.image-sv
          path: dsim_work/image_sv.so
    outputs:
      artifacts:
        - name: waves-sv-2
          path: waves_sv_2.mxd
        - name: log-sv-2
          path: dsim.log
  - name: run-sv-3
    depends: [compile-sv]
    commands:
      - dsim -image image_sv -waves waves_sv_3.mxd -sv_seed 3
    inputs:
      working:
        - name: compile-sv.image-sv
          path: dsim_work/image_sv.so
    outputs:
      artifacts:
        - name: waves-sv-3
          path: waves_sv_3.mxd
        - name: log-sv-3
          path: dsim.log

Job File Properties

name: carry-lookahead-adder

This is the name of the job. It does not have to match the job file name. Every time you submit the job file to run the job, a new auto-generated hash will be added to the name to form the unique job-id.

keep_for_support: false

Normally, the files synced from your local workspace to the remote workspace are removed once the job finishes. If Metrics Support requires access to these files to help you, you can set this optional property to true to save the files for customer support access only. To disable this, either set the property to false, or omit the property altogether.

tasks:  
  - name:

A job is comprised of tasks, which are executed in the remote workspace in parallel unless otherwise specified by the depends property. Each task should have a unique and descriptive name made up of uppercase and lowercase letters, numbers and the character -, but cannot start with a - or have any whitespace. The status of these tasks is obtained with the job-id. If there is an error in executing a task, all the following tasks are omitted and not executed.

compute_size:

The remote cloud compute configuration for the named task. Current options are s4 and s8. If this optional line is absent, the task will be executed on an s4.

commands:

These are the DSim simulator commands to be executed in the remote workspace in the order they appear. Note that each DSim command is preceded by a -.

mdc_work:

The path from where the commands should be executed, relative to the root of the local workspace. If this optional line is absent, all commands are executed from the root of the remote workspace.

outputs:
  artifacts:
    - name: log-sv
      path: dsim.log
  working:
    - name: image-sv
      path: dsim_work/image_sv.so

Tasks can have outputs - artifact outputs and/or working outputs. Artifact outputs are outputs saved on the remote for access after the job is complete, such as waveform files, simulation logs, and folders. Each artifact should have a unique name and path. The path is relative to the remote work directory. Artifacts can be downloaded using mdc job download. The file or folder will be compressed into a tar file before being downloaded to \<artifact-name>.tar.gz.

Working outputs are outputs that are only used for the duration of the job and are removed from the remote upon job completion. They define files for use in subsequent tasks, such as image files or source code. Each working output should have a unique name and path. The path is relative to the remote work directory. See Task Dependencies for details about working outputs.

depends:

If a task has dependencies, it must have this property. See Task Dependencies for details.

inputs:

Tasks with dependencies can have inputs, which are outputs from previously executed tasks. Tasks without dependencies do not have inputs since they have access to all the files from the local workspace upon job submission. See Task Dependencies for details.

Job File Properties Notation

In the user documentation, job file properties are described using the dot (.) notation. The dot notation shows the path to, or hierarchy of, the property. For example, in the above job file, the task compile-sv has these properties:

• One working input name inputs.working.name is sim-vhdl-sv.source.
• One artifact output name outputs.artifacts.name is log-sv with a path outputs.artifacts.path of dsim.log.
• One working output name outputs.working.name is image-sv with a path outputs.working.path of dsim_work/image_sv.so.

For more information on YAML, see The Official YAML Web Site.

Using Job Files

Create job files as YAML files with the extension .yml or .yaml using the above template. Put them anywhere in your local workspace directory. Use job commands to manage your jobs for regressions. Use job files for your regressions as shown in Tutorial: Regressions.

Job files are limited to a maximum of 25 tasks. Exceeding this limit will result in an error during job submission. If this occurs, consider using Task Fanout or breaking up the job file into multiple smaller job files.