Queued Stages

Queued Stages are a useful feature for aggregating data or supplying "structured" data. Often systems will expect to have data bundled up and submitted all at once. A good example of this is how many ERP systems expect Sales Orders to be submitted. Many will expect that the Order Header be submitted to the API along with all the Order's Details (Line Items). You can accomplish this by using a Queued Stage for the Line Items. in the example below, we submit multiple Contacts at the same time with and Account. Queued data can be returned in JSON, XML, or as a C# DataTable object.

Create a Stage like normal, and change the "Queued Behavior" to "Queued". When a stage is Queued, it will not attempt to write data to the Destination when the stage is processed. Instead, the data mapped to that Stage is held in memory, until requested.

Map the stage like normal. Then created a 2nd stage for Accounts. This stage is not Queued.

Within the field where you want to put the queued data, call GetQueuedJSON to fetch it.

The mapped field now contains a JSON array of queued data.

For more help on the functions to retrieve queued data, see here.

• Related Articles

• Repeating a Stage using Queued Datasets

  Sometimes when building maps, it becomes necessary to call a stage multiple times to process a subset of nested data. Take for example an Origin which may return an Account, but then also a list of Contacts in a structured format (such as JSON). To ...

• Scripting Class Object

  From within VBScript operations (ScriptedVariable, VBScriptProcedure, ScriptedField) in additional to normal VBScript functions, the developer has access to a custom “Starfish” class. The tables below define its usage. This class may also be used ...

• Creating a New Job

  Jobs are a series of tasks that can be performed against a singe data source. Each Jobs may contain different Stages, which allow you to move the data to different destinations. You may have an unlimited number of Stages per Job, and an unlimited ...

• Write Null instead of Blank

  By default, Starfish will write a blank, "", value to the destination. If you instead want to write Null, you must use a script for this. Function ScriptedField ScriptedField = "@@ORG:law_firm_c@@" if len(ScriptedField) = 0 then ScriptedField = ...

• Creating a New Stage

  Creating a Stage Under the mappings tab there is a bar titled stages. At the very end of the stages bar (on the right of the screen) there is a button with a + and what looks like a bulleted list, click this button and the New StarfishETL Stage menu ...