Queued Stages

Queued Stages are a useful feature for aggregating data or supplying "structured" data. Often systems will expect to have data bundled up and submitted all at once. A good example of this is how many ERP systems expect Sales Orders to be submitted. Many will expect that the Order Header be submitted to the API along with all the Order's Details (Line Items). You can accomplish this by using a Queued Stage for the Line Items. in the example below, we submit multiple Contacts at the same time with and Account. Queued data can be returned in JSON, XML, or as a C# DataTable object.

Create a Stage like normal, and change the "Queued Behavior" to "Queued". When a stage is Queued, it will not attempt to write data to the Destination when the stage is processed. Instead, the data mapped to that Stage is held in memory, until requested.

Map the stage like normal. Then created a 2nd stage for Accounts. This stage is not Queued.

Within the field where you want to put the queued data, call GetQueuedJSON to fetch it.

The mapped field now contains a JSON array of queued data.

For more help on the functions to retrieve queued data, see here.

• Related Articles

• Repeating a Stage using Queued Datasets

  Sometimes when building maps, it becomes necessary to call a stage multiple times to process a subset of nested data. Take for example an Origin which may return an Account, but then also a list of Contacts in a structured format (such as JSON). To ...

• Creating a New Stage

  Creating a Stage Under the mappings tab there is a bar titled stages. At the very end of the stages bar (on the right of the screen) there is a button with a + and what looks like a bulleted list, click this button and the New StarfishETL Stage menu ...

• Creating a New Job

  Jobs are a series of tasks that can be performed against a singe data source. Each Jobs may contain different Stages, which allow you to move the data to different destinations. You may have an unlimited number of Stages per Job, and an unlimited ...

• Sage CRM Connector

  SageCRM Origin Make sure you update your standard SageCRM URL to include ../webservices at the end. The SageCRM Origin will not work if you have a blank password. Your user must have a password. Origin Filters "SageCRM Filters should work just like ...

• Editing XML

  Your entire mapping file is stored as encrypted XML in the .spd file. You can review and edit this XML directly by clicking on the Edit XML button on the General tab. In here you can edit, remove, copy jobs, stages, etc. If you decide to copy a job, ...