Buffering mechanism

IBMStreams com.ibm.streamsx.objectstorage Toolkit > com.ibm.streamsx.objectstorage 2.2.5 > com.ibm.streamsx.objectstorage.s3 > S3ObjectStorageSink > Buffering mechanism

The operator buffers the object data prior upload to object storage depending on the selected protocol (client) differently.

cos - disk buffering is used
s3a - memory buffering is used per default. You can change the buffering mechanism with the parameter s3aFastUploadBuffer.

The toolkit user may easily switch from hadoop-aws to stocator client by specifying appropriate protocol in the objectStorageURI parameter of the ObjectStorageSink operator or in the protocol parameter of the S3ObjectStorageSink operator. Concretely, when s3a protocol is specified the toolkit uses hadoop-aws client. When cos protocol is specified the toolkit uses stocator client.