Module nstream.adapter.dynamodb
Package nstream.adapter.dynamodb

Class DynamoDbIngestingAgent

java.lang.Object
swim.api.agent.AbstractAgent
nstream.adapter.common.NstreamAgent
nstream.adapter.common.ingress.IngestorAgent<S,V>
nstream.adapter.common.ingress.IngestorMetricsAgent<DynamoDbIngressSettings,Map<String,software.amazon.awssdk.services.dynamodb.model.AttributeValue>>
nstream.adapter.dynamodb.DynamoDbIngestingAgent
All Implemented Interfaces:
nstream.adapter.common.schedule.StageService, swim.api.agent.Agent, swim.api.downlink.DownlinkFactory, swim.api.lane.LaneFactory, swim.api.ref.WarpRef, swim.api.store.Store, swim.concurrent.Schedule, swim.util.Log
Direct Known Subclasses:
DynamoDbIngestingPatch
public abstract class DynamoDbIngestingAgent extends nstream.adapter.common.ingress.IngestorMetricsAgent<DynamoDbIngressSettings,Map<String,software.amazon.awssdk.services.dynamodb.model.AttributeValue>>
An abstract Web Agent that can scan a DynamoDB database table.

The DynamoDbIngestingAgent loads agent configuration into a settings object on agent start. The agent uses the configured DynamoDbClient provision and timing properties to periodically poll the database for items, deferring processing of the ingested attribute map to subclasses.

Scanning and ingestion are done asynchronously of the agent thread.

See Also:

  • Nested Class Summary

    Nested classes/interfaces inherited from interface nstream.adapter.common.schedule.StageService

    nstream.adapter.common.schedule.StageService.StageCallable<V extends Object>, nstream.adapter.common.schedule.StageService.StageConsumer<V extends Object>, nstream.adapter.common.schedule.StageService.StageRunnable

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected software.amazon.awssdk.services.dynamodb.DynamoDbClient
    client
     
    protected swim.concurrent.TimerRef
    pollTimer
     

    Fields inherited from class nstream.adapter.common.ingress.IngestorAgent

    ingressSettings

    Fields inherited from class nstream.adapter.common.NstreamAgent

    apiLaneSummary, labeledLog, stageAmenity, summary

    Fields inherited from class swim.api.agent.AbstractAgent

    context

  • Constructor Summary

    Constructors
    Constructor
    Description
    DynamoDbIngestingAgent()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    protected void
    assignClient(software.amazon.awssdk.services.dynamodb.DynamoDbClient client)
    Assign the DynamoDbClient to be used for ingestion.
    protected void
    cancel()
    Cancel the poll timer and therefore ingestion.
    void
    didFailIngest(Map<String,software.amazon.awssdk.services.dynamodb.model.AttributeValue> item, Exception exception)
     
    protected void
    fetchAndIngest()
    Loop through the ScanRequest results provided by the abstract scanRequest() method, ingesting each document asynchronously.
    protected DynamoDbIngressSettings
    parseIngressSettings(swim.structure.Value prop)
     
    protected abstract software.amazon.awssdk.services.dynamodb.model.ScanRequest
    scanRequest()
    The configurable ScanRequest to be executed, looped through and ingested.
    protected void
    stageReception()
     

    Methods inherited from class nstream.adapter.common.ingress.IngestorMetricsAgent

    didStageReception, ingestOrCancel, ingestOrCancel, ingestOrContinue

    Methods inherited from class nstream.adapter.common.ingress.IngestorAgent

    didStart, ingest, loadSettings, loadSettings, loadSettings, willStop

    Methods inherited from class nstream.adapter.common.NstreamAgent

    apiLaneSummaryDecodeRequest, apiLaneSummaryDoRespond, debug, execute, executeNonblocking, fail, handleDeferrableException, info, prepareLoop, schedule, scheduleAtFixedRate, scheduleWithFixedDelay, scheduleWithInformedBackoff, trace, warn

    Methods inherited from class swim.api.agent.AbstractAgent

    agentContext, agentId, agents, asyncStage, cipherSuite, close, closeAgent, closeAgent, command, command, command, command, command, command, command, command, command, command, command, command, command, command, command, command, commandLane, demandLane, demandMapLane, didClose, didFail, didLoad, didOpen, didStop, didUnload, downlink, downlinkHttp, downlinkList, downlinkMap, downlinkValue, downlinkWs, error, geospatialData, geospatialData, geospatialLane, getAgent, getAgent, getAgent, getLane, getProp, getProp, hostRef, hostRef, hostUri, httpLane, isSecure, joinMapLane, joinValueLane, lane, laneRef, laneRef, laneRef, laneRef, link, listData, listData, listLane, localAddress, localCertificates, localIdentity, localPrincipal, mapData, mapData, mapLane, nodeRef, nodeRef, nodeRef, nodeRef, nodeUri, openAgent, openAgent, openAgent, openAgent, openAgent, openAgent, openLane, props, remoteAddress, remoteCertificates, remoteIdentity, remotePrincipal, schedule, securityProtocol, setTimer, spatialData, spatialData, spatialLane, stage, supplyLane, timer, valueData, valueData, valueLane, willClose, willLoad, willOpen, willStart, willUnload, wsLane

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • client

      protected software.amazon.awssdk.services.dynamodb.DynamoDbClient client

    • pollTimer

      protected swim.concurrent.TimerRef pollTimer

  • Constructor Details

    • DynamoDbIngestingAgent

      public DynamoDbIngestingAgent()

  • Method Details

    • assignClient

      protected void assignClient(software.amazon.awssdk.services.dynamodb.DynamoDbClient client)
      Assign the DynamoDbClient to be used for ingestion.
      Parameters:
      client - the DynamoDB client

    • fetchAndIngest

      protected void fetchAndIngest()
      Loop through the ScanRequest results provided by the abstract scanRequest() method, ingesting each document asynchronously.

    • scanRequest

      protected abstract software.amazon.awssdk.services.dynamodb.model.ScanRequest scanRequest()
      The configurable ScanRequest to be executed, looped through and ingested.

      Can be overridden for bespoke queries.

      Returns:
      the scan request

    • cancel

      protected void cancel()
      Cancel the poll timer and therefore ingestion. Will cancel any currently active ingestion task.
      Overrides:
      cancel in class nstream.adapter.common.ingress.IngestorAgent<DynamoDbIngressSettings,Map<String,software.amazon.awssdk.services.dynamodb.model.AttributeValue>>

    • didFailIngest

      public void didFailIngest(Map<String,software.amazon.awssdk.services.dynamodb.model.AttributeValue> item, Exception exception)
      Overrides:
      didFailIngest in class nstream.adapter.common.ingress.IngestorMetricsAgent<DynamoDbIngressSettings,Map<String,software.amazon.awssdk.services.dynamodb.model.AttributeValue>>

    • parseIngressSettings

      protected DynamoDbIngressSettings parseIngressSettings(swim.structure.Value prop)
      Specified by:
      parseIngressSettings in class nstream.adapter.common.ingress.IngestorAgent<DynamoDbIngressSettings,Map<String,software.amazon.awssdk.services.dynamodb.model.AttributeValue>>

    • stageReception

      protected void stageReception()
      Specified by:
      stageReception in class nstream.adapter.common.ingress.IngestorAgent<DynamoDbIngressSettings,Map<String,software.amazon.awssdk.services.dynamodb.model.AttributeValue>>