How to configure s3 in AEM ?

How to Configure S3 in AEM 



Configuring s3 data stores in AEM 6

In Adobe Experience Manager (AEM), binary data can be stored independently from the content nodes. The binary data is stored in a data store, whereas content nodes are stored in a node store.

Both data stores and node stores can be configured using OSGi configuration. Each OSGi configuration is referenced using a persistent identifier (PID).


In this article we learn the detailed steps for configuration S3 data store in AEM. 


You can get the architectural level understand of S3 with AEM from article - 

https://www.aemrules.com/2021/06/how-does-s3-works-with-aem.html


 

Amazon S3 Data Store


AEM can be configured to store data in Amazon’s Simple Storage Service (S3). It uses the org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore.config PID for configuration.

 

In order to enable the S3 data store functionality, a feature pack containing the S3 Datastore Connector needs to be downloaded and installed. 


Go to the Adobe Repository and download the latest version from the 1.10.x versions of the feature pack (for example, com.adobe.granite.oak.s3connector-1.10.0.zip) & download the ZIP file.  Additionally, you also need to download and install the latest AEM service pack as listed on the AEM 6.5 Release Notes page.

 

Point to Understand - 


When using AEM with TarMK, binaries will be stored by default in the FileDataStore. To use TarMK with the S3 Datastore, you need to start AEM using the crx3tar-nofds runmode, for example:


 java -jar <<aem-jar-file>>.jar -r crx3tar-nofds 


S3 Configuration Steps - 

  • Make sure you have downloaded S3 Connector feature pack as explained above 
  • Extract the contents of the feature pack zip file to a temporary folder lets say /home/myuser/.
  • Go to the temporary folder (/home/myuser/) and navigate to the following location:
    • /home/myuser/com.adobe.granite.oak.s3connector-1.10.14j/cr_root/libs/system/install/
    • Copy all the contents from the above location to /crx-quickstart/install/
  • If AEM is already configured to work with the Tar or MongoDB storage, remove any existing configuration files from the /crx-quickstart/install folder before proceeding. The files that need to be removed are:
    • For MongoMK: org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService.config
    • For TarMK: org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.config
  • Return to the temporary location where the feature pack has been extracted, and copy the contents of the following folder:
    • /home/myuser/com.adobe.granite.oak.s3connector-1.10.14j/cr_root/libs/system/config/
    • Copy all the contents from the above location to /crx-quickstart/install/
  • Make sure you only copy the configuration files needed by your current configuration. 
    • For both a dedicated data store and a shared data store setup copy the org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore.config file.
    • org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.config file
  • Edit the newly copied config file and add the configuration options required by your setup.
    • /crx-quickstart/install/org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.config
                              accessKey=""                                  ## Input S3 Access key. 
                            connectionTimeout="120000"         ## You can leave default.
                            maxConnections="40"                     ## You can leave default.
                            maxErrorRetry="10"                        ## You can leave default.
                            s3Bucket=""                                     ## Add your S3 Bucket name 
                            s3Region=""                                     ## Add AWS region 
                            secretKey=""                                    ## AWS Secret key 
                            socketTimeout="120000"                ## You can leave default.
                            writeThread="30"                            ## You can leave default.
                            Secret="mydevenv"                         ## this could be any string.  

 

  • Start AEM with no-fds run mode -  java -jar <<aem-jar-file>>.jar -r crx3tar-nofds



NOTE :


In a cluster setup, perform above steps on all nodes of cluster one by one. Also, make sure to use same S3 settings for all nodes.

 


 References : 


 

 

Comments

Popular Posts

how to clear dispatcher cache in aem ?

How Does S3 works with AEM ?

How to Increase Apache Request Per Second ?

AdobeDispatcherHacks ".statfile"

How to Sync HMAC in AEM ?

AEM Security Headers

Dispatcher flush from AEM UI

How Encrypt-Decrypt Data in AEM , AEM Crypto Support ?

Configure/Decoding AEM AuditLogs