Running DDS Jobs

A set of API functions are available for using the Data Delivery Service (DDS).

Before you run a DDS job, you must define a cloud storage target in the Sage Intacct UI.

DDS is not configured separately for entities and data is always extracted from the top level.

The following is a basic sequence diagram for a DDS job.

Sequence diagram for DDS job

The DDS Manager regularly polls the queue for new jobs and processes them.

The sections that follow show how to use API functions to list available DDS objects, start a job, and monitor the results.

List available objects

You can use the following function to get a listing of all objects, both standard and custom, available for DDS jobs.

<getDdsObjects/>

Run jobs

You can run regular asynchronous DDS jobs for single objects or batches of objects. Consider whether you want to get all the records for the given objects, or only those records that changed since the last run.

You can also run a set of synchronous DDS jobs (for change information only) for up to five objects. Synchronous jobs can help avoid reporting gaps—each job has the same starting timestamp.

Job files are always delivered as separate CSV files. Files can be split based on a maximum number of records. It is your responsibility to process and load the files into your own database.

Single object

The following example runs a DDS job to get all vendor records, then saves the data to the specified cloud storage location:

<runDdsJob>
    <object>VENDOR</object>
    <cloudDelivery>AWS-S3-DDS</cloudDelivery>
    <jobType>all</jobType>
</runDdsJob>

The response provides the DDS job object, for example:

 <data>
     <ddsjob>
         <DDSJOB>VENDOR;all;2017-10-04T18:47:55+00:00</DDSJOB>
         <OBJECT>VENDOR</OBJECT>
         <JOBTYPE>all</JOBTYPE>
         <SUBSCRIPTIONKEY></SUBSCRIPTIONKEY>
         <QUEUETIME>10/04/2017 18:47:55</QUEUETIME>
         <STARTTIME></STARTTIME>
         <ENDTIME></ENDTIME>
         <RECORDSCREATED>0</RECORDSCREATED>
         <RECORDSUPDATED>0</RECORDSUPDATED>
         <RECORDSDELETED>0</RECORDSDELETED>
         <STATUS>Queued</STATUS>
         <ERROR></ERROR>
         <FILELIST></FILELIST>
         <DDSHISTORYKEY>11</DDSHISTORYKEY>
         <RECORDNO>12</RECORDNO>
     </ddsjob>
 </data>

Multiple objects

You can run a job for an unlimited number of objects, as follows:

<runDdsJob>
    <objects>
        <object>APRECORD</object>
        <object>APDETAIL</object>
        <object>APBILLPAYMENT</object>
        <object>VENDOR</object>
        <object>LOCATION</object>
        <object>DEPARTMENT</object>
    </objects>
    <cloudDelivery>AWS-S3-DDS</cloudDelivery>
    <jobType>all</jobType>
</runDdsJob>

The response provides job information for each object, for example:

<data listtype="ddsjob" count="5">
    <ddsjob>
        <DDSJOB>APRECORD;change;2019-10-10T15:48:38+00:00</DDSJOB>
        <OBJECT>APRECORD</OBJECT>
        <JOBTYPE>change</JOBTYPE>
        <SUBSCRIPTIONKEY></SUBSCRIPTIONKEY>
        <QUEUETIME>10/10/2019 15:48:38</QUEUETIME>
        ...
    </ddsjob>
    <ddsjob>
        <DDSJOB>APDETAIL;change;2019-10-10T15:48:38+00:00</DDSJOB>
        <OBJECT>APDETAIL</OBJECT>
        <JOBTYPE>change</JOBTYPE>
        <SUBSCRIPTIONKEY></SUBSCRIPTIONKEY>
        <QUEUETIME>10/10/2019 15:48:38</QUEUETIME>
        ...
    </ddsjob>
    <ddsjob>
        <DDSJOB>APBILLPAYMENT;change;2019-10-14T19:44:25+00:00</DDSJOB>
        <OBJECT>APBILLPAYMENT</OBJECT>
        <JOBTYPE>change</JOBTYPE>
        <SUBSCRIPTIONKEY></SUBSCRIPTIONKEY>
        <QUEUETIME>10/14/2019 19:44:25</QUEUETIME>
        ...
    </ddsjob>
    <ddsjob>
        <DDSJOB>VENDOR;change;2019-10-10T15:48:38+00:00</DDSJOB>
        <OBJECT>VENDOR</OBJECT>
        <JOBTYPE>change</JOBTYPE>
        <SUBSCRIPTIONKEY></SUBSCRIPTIONKEY>
        <QUEUETIME>10/10/2019 15:48:38</QUEUETIME>
        ...
    </ddsjob>
    <ddsjob>
        <DDSJOB>LOCATION;change;2019-10-10T15:48:38+00:00</DDSJOB>
        <OBJECT>LOCATION</OBJECT>
        <JOBTYPE>change</JOBTYPE>
        <SUBSCRIPTIONKEY></SUBSCRIPTIONKEY>
        <QUEUETIME>10/10/2019 15:48:38</QUEUETIME>
        ...
    </ddsjob>
    <ddsjob>
        <DDSJOB>DEPARTMENT;change;2019-10-10T15:48:39+00:00</DDSJOB>
        <OBJECT>DEPARTMENT</OBJECT>
        <JOBTYPE>change</JOBTYPE>
        <SUBSCRIPTIONKEY></SUBSCRIPTIONKEY>
        <QUEUETIME>10/10/2019 15:48:39</QUEUETIME>
        ...
    </ddsjob>
</data>

Filter and order results

You can use the query option in a runDdsJob request in several ways:

Synchronous run

You can run a synchronous DDS job to get change information for up to five objects at a time. For example, the following run gets change information for the given objects. Each of the jobs will have the same timestamp as the first DDS job in the set to run:

    <runDdsJob>
        <objects>
            <object>APRECORD</object>
            <object>APDETAIL</object>
            <object>VENDOR</object>
            <object>LOCATION</object>
            <object>DEPARTMENT</object>
        </objects>
        <synchronized>true</synchronized>
        <cloudDelivery>AWS-S3-DDS</cloudDelivery>
        <jobType>change</jobType>
    </runDdsJob>

Monitor a job

DDS does not actively communicate job status. To get the job status, use the read function on the DDSJOB object and provide the RECORDNO returned by runDdsJob.

<read>
    <object>DDSJOB</object>
    <keys>1</keys>
    <fields>*</fields>
</read>

When the job is complete, the response provides a list of CSV files published to the cloud storage target, for example:

<STATUS>Completed</STATUS>
<ERROR></ERROR>
<FILELIST>VENDOR.all.10.2017-09-29_18.04.59_UTC_00000.csv</FILELIST>

Provide feedback