Creating a New Stage

Creating a New Stage

Creating a Stage


Under the mappings tab there is a bar titled stages. At the very end of the stages bar (on the right of the screen) there is a button with a + and what looks like a bulleted list, click this button and the New StarfishETL Stage menu will appear.


Configuring your Stage

There are a variety of stage types and options to choose from for a stage, here are descriptions of what each stage type and option does.

Option Name
Description
Stage Type - Update
Will perform an update-only operation on matched records for each Origin row. Unmatched records will be skipped.
Stage Type - Insert
Will perform an insert-only operation for each Origin row. Use this operation when performing a “dump” of the data.
Stage Type - Delete
Will perform a delete operation on the Destination database for each Origin row, based on the match.
Stage Type - Update/Insert
Will first perform a lookup, based on matches. If a match is found, an update operation will be performed. If no match is found, an insert operation will be performed.
Stage Type - Skip/Insert
Will first perform a lookup, based on matches. If a match is found, the record will not be modified. If no match is found, an insert operation will be performed.
Use Origin Connection
By default all stages write to the destination connection, use this checkbox to override this behavior and instead select a table from your origin connection to write to. This is useful for setting things such as a "process flag" or "last processed date".
Omit Blank Field Value Writes
If checked, the insert/operation will ignore NULL/blank fields while building the SQL statement – resulting in NULL values on the destination. Said another way, if a mapping on a particular field results in a blank or null value, it won’t bother to write that value to the destination at all. This way, if you had a value in the destination, it won’t be overwritten with a blank/null.
Skip Behavior
This is used in conjunction with the "Skip If Blank" checkbox column in the mapping screen. When a mapping is blank this setting will control whether just the current stage gets skipped or if the entire row gets skipped.
Multiple Skip Logic
When multiple mappings are being checked for blank this controls when a row is skipped. If "Or" is used then the record will be skipped if any one of the mappings are blank. If "And" is used then the record will only be skipped if all the mappings are blank.

    • Related Articles

    • Creating a New Job

      Jobs are a series of tasks that can be performed against a singe data source. Each Jobs may contain different Stages, which allow you to move the data to different destinations. You may have an unlimited number of Stages per Job, and an unlimited ...
    • Creating a new project from a pre-built map

        Step Snapshot Go to connections to create your 2 connections if you have not added the connections already Please utilize the Knowledge Base to learn more about the types of connectors and how to configure them. 3.  Go to Projects -> Import to ...
    • Creating a Project

      To create a new project navigate to the projects tab and select the plus sign (new) icon on the right.
    • Repeating a Stage using Queued Datasets

      Sometimes when building maps, it becomes necessary to call a stage multiple times to process a subset of nested data. Take for example an Origin which may return an Account, but then also a list of Contacts in a structured format (such as JSON). To ...
    • Creating a Global Script

      To create a global script, click on the globe shaped icon and choose from VBScript, JavaScript, or C# global in the mappings ribbon.