Manufacturing

BSH and Commencis leverage AWS Infrastructure to Enhance Product Information Management

BSH

Manufacturing

BSH and Commencis leverage AWS Infrastructure to Enhance Product Information Management

BSH

"With the PIM API system, our goal was to deploy an auto-scaling infrastructure on the AWS Cloud that could handle millions of products and user queries worldwide. Commencis is leveraging its agile and DevOps expertise, automated build, and system deployment, which allows for a reduction in environment provisioning time, zero downtime, and 100% improvement in recovery time objectives. With greater visibility over entire systems and processes, we can see the full business picture and make better-informed business decisions."   Serkan Bayazıt-  IT Manager, Digital Marketing & Customer Experience at BSH

About BSH

BSH is one of the world’s leading home appliance manufacturing companies, with 40+ production sites in Europe, the USA, Latin America, and Asia along with a strong network of 80 sales, production, and service companies in 50+ countries. With outstanding home appliances and useful digital services, BSH meets the needs of consumers worldwide.

Business Challenge

The goal was to develop BSH’s PIM (Product Information Management) application so that BSH’s customers could easily access up-to-date BSH’s product and catalog information from BSH’s corporate website in different countries and other third-party applications.

Within the scope of the PIM application programming interface (API) system, a wide range of information, including the features and documentation of BSH products, can now be queried in several languages. Developing a scalable and high-performance architecture with external APIs that could handle simultaneous systems queries and results within a short period of time was the most challenging non-functional requirement.

Solution

Commencis Cloud Team developed end-to-end applications, conducted performance and load tests, and utilized multiple AWS services in the solution. The solution consists of two main parts: Product Information Data Collection Services & Production Information Data Get Services

Product Information Data Collection Services are a set of services running as Lambda functions to collect BSH’s consumer product information in XML format from ERP systems (SAP) and store the data in a database.

On-premises applications that are the master of such product information data send messages about changes (add product, update product, delete product) in consumer products and put the associated hierarchical product data in XML formatted files into an S3 bucket. Lambda function reading SQS listens for new messages, triggers automatically when new messages are sent, and reads S3 files in XML format. After parsing and converting the files, the content is stored or deleted in the Aurora PostgreSQL database’s production information tables. The database operation triggers another Lambda function which pushes notifications about these database operations to SNS. Applications that are subscribed to SNS topics are being notified about product information data changes, which will allow them to call Production Information Data Get Services to get up-to-date product information data and to update the product information data in their systems.

  • InsertFeedToPostgreSQL: handles processing of XML files.
  • InsertProductResponseToPostgresql: handles insertion of preprocessed response data into Aurora.
  • SNSGeneratorFunction: handles generation of SNS messages about affected products for clients.
  • AsyncDistributeLambda: handles asynchronous invocation of snsGeneratorFunction and InsertProductResponseToPostgresql Lambda functions.

Product Information Data Get Services are a set of services running as Lambda functions behind API Gateway to return up-to-date product information data that is stored in Aurora PostgreSQL. To be able to meet high-performance requirements, the product data is read from the read replica instance of Aurora PostgreSQL. These Lambda functions are called from various intranet and internet applications of BSH through the API Gateway which is secured by API keys.

  • pimApiSpec: Swagger API spec rendering function
  • readProductGo: main function clients use to retrieve product information.

Use of third-party applications or solutions

Bitbucket Cloud: A Bitbucket Cloud product is used to store Lambda function source codes, manage releases, and deploy new versions and fixes.

JMeter: JMeter is used to test and report the performance of the Product Information Data Get Services running as Lambda functions.

Terraform: Terraform scripts are developed to define and maintain the AWS infrastructure. Scripts are stored in Bitbucket Cloud like Lambda functions’ source code.

AWS Services Used as Part of the Solution

Lambda: Lambda functions are developed in GoLang to meet performance requirements for both Product Information Data Collection Services and Product Information Data Get Services.

API Gateway: Lambda functions for Product Information Data Get Services run behind API Gateway and are secured with API keys.

Aurora PostgreSQL: The product information data is stored in Aurora tables. Aurora is deployed as planned. Aurora Serverless v2 deployment is still on the roadmap to meet both high performance requirements and unpredictable traffic patterns. The Aurora read replica serves as a service for retrieving product information data.

SQS: The service is selected for decoupling master product datastore in on-prem and its replica product datastore in AWS. When the PIM is offline due to maintenance, version update, etc. SQS is online and continues to collect product information change messages.

SNS: The service is used to send notifications when the product data store gets updated.

S3: XML files containing product information are stored in an S3 bucket.

Output Architecture

Data about products (new, updated, deleted) is created in on-premises applications, stored in S3 buckets as XML files, and event messages are put into SQS FIFO queues.

Lambda function triggers conversion and storage of product information in Aurora database when event messages are put in SQS.

Database operations event messages are sent to SNS with another Lambda function. External applications that are subscribed to SNS topics are being notified. This triggers external applications to query up to data product information data either individually or as bulk with the API exposed via API Gateway in front of Lambda functions.

Outcomes & Benefits

Commencis Cloud Team developed end-to-end applications, conducted performance and load tests, and utilized multiple AWS services.

BSH now has a product information store in AWS that runs in parallel with the master database and is updated asynchronously. The product information data store can be queried concurrently by many external systems, deployed in a high-performing and highly scalable architecture at AWS.

The solution is planned to be evolved to store other BSH data on the same highly scalable architecture. AWS CloudFront is planned to be installed through the AWS API Gateway. This is because the PIM API system needs to be queried from different countries in the later phases of the project. This will ensure that delays caused by data connections at remote points are avoided.

The developed solution is now integrated with BSH’s corporate web site (Digital Commerce) where product and catalog information is retrieved from it. The performance of the product and catalog pages is reported to be 10 times faster compared to the previous solution.

Let’s start your cloud journey
Get in Touch