Amazon S3

You can configure Amazon S3 only as a source.

To configure the connection details for Amazon S3 in your Micro-Integration, see Amazon S3 Connection Parameters

You must also define at least one Micro-Integration Flow that specifies:

the endpoints where data is consumed from and published to. For more information, see Micro-Integration Flow Parameters.
how the header and payload fields should be mapped. For more information, see Mapping Headers and Payloads

For a listing of available message headers, see Amazon S3 Message Headers.

Amazon S3 Connection Parameters

The following table describes the connection parameters for Amazon S3.

Field	Description
Checkpoint Queue	The name of the Last Value Queue to use as checkpoint storage.
Auto Create Checkpoint Queue	Select to automatically create the checkpoint queue if it does not exist.

Micro-Integration Flow Parameters

You must configure the endpoint parameters for each Flow. Each Flow can have different settings, but they all share the connection details of the parent Micro-Integration.

Amazon S3 Source Parameters

The following table describes the parameters for configuring Amazon S3 as a source.

Field	Description
Amazon S3 URI	The URI of the Amazon S3 data source, including the bucket name or ARN. Format: `aws2-s3://<bucketNameOrARN>`
AWS Region	The AWS Region, for example, `us-east-1`.
AWS Access Key	The AWS Access Key ID to use to authenticate with the Amazon S3 service.
AWS Secret Key	The Secret Access Key associated with the Access Key ID.
Delete after read	Select this option to delete the file after reading.
File Format	The format of the files coming from the Amazon S3 service Delimited File Configuration—Events and fields in the file are separated by specific characters. For parameters, see Delimited File Configuration Properties. JSON File Configuration—Events in the file are JSON-encoded. For parameters, see JSON File Configuration Properties. XML File Configuration—Events in the file are encoded as XML. For parameters, see XML File Configuration Properties.
Additional Properties	Additional properties for the Amazon S3 service. For more information, see Additional Properties.

Delimited File Configuration Properties

Field	Description
Event Delimiter	The delimiter used to separate the events in the file.
Field Delimiter	The delimiter used to separate the fields in each event.

JSON File Configuration Properties

Field	Description
JSON Path Filter Expression	A JSONPath expression used to split the JSON file into multiple messages. The default is `$`, which means the entire file is treated as a single message. For example, if a JSON file contains an array of objects `[{"a":1}, {"a":2}]`, setting `jsonPath` to `$.*` will generate two separate messages.

XML File Configuration Properties

Field	Description
XPath Filter Expression	An XPath expression that splits the XML file into multiple messages. The default is `/`, which means the entire file is treated as a single message. For example, if an XML file is `<root><item>1</item><item>2</item></root>`, setting `xPath` to `/root/item` generates two messages.

Additional Properties

Option	Type	Default	Description
autoCreateBucket	boolean	`false`	When true, enables the autocreation of the S3 bucket. You can also use this parameter when the `moveAfterRead` option is enabled to create the `destinationBucket` if it doesn't already exist.
delimiter	String	None	The delimiter to use to consume only objects you are interested in.
forcePathStyle	boolean	`false`	When true, the S3 client uses a path-style URL instead of virtual-hosted-style.
overrideEndpoint	boolean	`false`	When true, enables overriding the endpoint. This option must be used in combination with the `uriEndpointOverride` option.
policy	String	None	The policy for this queue to set in the com.amazonaws.services.s3.AmazonS3#setBucketPolicy() method.
prefix	String	None	A prefix to filter objects in the bucket. Only objects with keys starting with this prefix are consumed. If not set, all objects in the bucket are processed. For example, if `prefix` is `inbound/`, only objects in the `inbound/` folder are processed.
uriEndpointOverride	String	None	The overriding URI endpoint. This option must be used in combination with the `overrideEndpoint` option.
customerAlgorithm	String	None	The customer algorithm to use when CustomerKey is enabled.
customerKeyId	String	None	The id of the Customer key to use when CustomerKey is enabled.
customerKeyMD5	String	None	The MD5 of Customer key to use when CustomerKey is enabled.
destinationBucket	String	None	The destination bucket where an object must be moved when `moveAfterRead` is set to true.
destinationBucketPrefix	String	None	The destination bucket prefix to use when an object must be moved, and `moveAfterRead` is set to true.
destinationBucketSuffix	String	None	The destination bucket suffix to use when an object must be moved, and `moveAfterRead` is set to true.
doneFileName	String	None	If provided, Camel consumes files only if a done file exists.
fileName	String	None	The file name to get the object from the bucket.
maxConnections	int	`60`	The `maxConnections` parameter in the S3 client configuration.
maxMessagesPerPoll	int	`10`	The maximum number of messages as a limit to poll at each polling. The default value is 10. Use 0 or a negative number to set it as unlimited.
moveAfterRead	boolean	`false`	When true, moves objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation, the `destinationBucket` option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved.
removePrefixOnMove	boolean	`false`	When true, removes the prefix from the object key when moving it to the destination bucket.
batchMessageNumber	int	`10`	The number of messages composing a batch in streaming upload mode.
batchSize	int	`1000000`	The batch size (in bytes) in streaming upload mode.
bufferSize	int	`1000000`	The buffer size (in bytes) in streaming upload mode.
deleteAfterWrite	boolean	`false`	When true, deletes the file object after the S3 file has been uploaded.
keyName	String	None	The key name for an element in the bucket through endpoint parameter.
streamingUploadMode	boolean	`false`	When true, the upload to bucket is done in streaming.
streamingUploadTimeout	long	None	When streaming upload mode is true, this option sets the timeout to complete upload.
awsKMSKeyId	String	None	The id of KMS key to use when KMS is enabled.
conditionalWritesEnabled	boolean	`false`	When true, uploads the object only if the object key name does not already exist in the bucket specified.
useAwsKMS	boolean	`false`	When true, KMS must be used.
useCustomerKey	boolean	`false`	When true, Customer Key must be used.
useSSES3	boolean	`false`	When true, SSE S3 must be used.
proxyHost	String	None	The proxy host when instantiating the SQS client.
proxyPort	Integer	None	A proxy port to be used inside the client definition.
proxyProtocol	Protocol	`HTTPS`	The proxy protocol to use instantiating the S3 client. Set to one of: HTTP HTTPS
backoffErrorThreshold	int	None	The number of subsequent error polls that must occur before the `backoffMultiplier` is triggered. Error polls are polls that fail because of an error.
backoffIdleThreshold	int	None	The number of subsequent idle polls that must occur before the `backoffMultiplier` is triggered.
backoffMultiplier	int	None	A multiplier that allows the scheduled polling consumer to back off if there has been a number of subsequent idle polls or errors in a row. The multiplier represents the number of polls that are skipped before the next actual attempt occurs. When you use this option, you must also configure `backoffIdleThreshold` and/or `backoffErrorThreshold`.
delay	long	`500`	The number of milliseconds before the next poll.
greedy	boolean	`false`	When true, the ScheduledPollConsumer runs immediately again if the previous run polled one or more messages.
initialDelay	long	`1000`	The number of milliseconds before the first poll starts.
repeatCount	long	`0`	The maximum number of times the scheduler fires. For example, if you set this parameter to 1, the scheduler fires only once. If you set it to 5, the scheduler fires five times. A value of zero or a negative value means the scheduler fires indefinitely.
startScheduler	boolean	`true`	When true, the scheduler is auto started.
useFixedDelay	boolean	`true`	When true, fixed delay is used. When false, fixed rate is used. See ScheduledExecutorService in the JDK for details.
profileCredentialsName	String	None	The profile name when using a profile credentials provider.
sessionToken	String	None	The Amazon AWS Session Token used when the user must assume an IAM role.
trustAllCertificates	boolean	`false`	When true, all certificates are trusted when overriding the endpoint.
useDefaultCredentialsProvider	boolean	`false`	When true, the S3 client expects to load credentials through a default credentials provider.
useProfileCredentialsProvider	boolean	`false`	When true, the S3 client expects to load credentials through a profile credentials provider.
useSessionCredentials	boolean	`false`	When true, the S3 client expects to use Session Credentials (useful when the user must assume an IAM role for doing operations in S3).

Provide feedback