SAP Data Quality Solutions

by | Data, Data-Lake | 0 comments

Who likes poor data?

Incorrect, incomplete and inconsistent data can give you horrific nightmares. This can be unsettling when important decisions of your organisation are affected and all your predictions and estimates go awry.

To stay smart and ahead of the race, clear selection of the right technology is absolutely crucial.

This blog is a solution approach of all the quality data products published by SAP for your on-premise, on-cloud systems. The solutions that we are going to discuss are:

  1. SAP Data Quality Management
  2. SAP DQM, microservices

SAP Data Quality Management

Latest version: 4.0 SP09

SAP Data Quality Management version for SAP Solutions (DQM for SAP) provides a pre-packaged native integration of quality data best practices within the SAP Customer Relationship Management (SAP CRM), SAP Enterprise Resource Planning (SAP ERP), and SAP Master Data Governance (SAP MDG) environments.

When you create a new customer, supplier, or partner record using SAP ERP, SAP CRM, or SAP MDG, DQM for SAP software corrects and/or validates the address based on referential data sources, and formats the address according to norms of the applicable country. DQM for SAP also helps prevent duplicate entities entering your SAP ERP, CRM, and MDG applications, and helps search and improve your existing data.

This version of data quality is installed as an add-on to the SAP ERP, SAP CRM, or SAP MDG environments. However, this product also has its dependency on SAP Data services, where you can make use of the standard data quality web services and process your data in real-time.

Feature support

Address Cleansing
A correct address results in correct delivery.  With the SAP Address cleansing feature you can achieve that on the fly. Parse, standardise and cleanse the address data in real-time with effective like, phonetic and fuzzy search options. Consequently, dependency on any third-party providers for address validation is reduced. SAP provides its own reference data directories and has its reference data support for 240+ countries globally.

Data Cleansing
Information governance is essential and every organisation has its own set of data governance rules defined. This generates a more simplified use of the data cleansing feature where you can parse, standardise and correct the name, firm, and operational data (e.g. product, financial, etc.).

Geocoding and Reverse Geocoding Support
This feature helps you to add the geocode information (latitude and longitude) related to the information of the records. Also, you can perform reverse geocoding that enables you to identify the closest address or point of interest based on an input reference location.  This can be one of the following: latitude and longitude, address, point of interest (ATM, train station, bus station, restaurant, etc.).  To achieve this, SAP maintains separate geocode directories for different countries.

Match and Consolidate
This is a very interesting and powerful feature that enables you to eliminate duplicate records in the system that belong to the same entity. Here, you can set rules based on combinations of various elements matching at a certain threshold and decide whether the record is a duplicate or not. On identification of a possible match, business users can club the record as one; reducing the chances of potential duplicates in the system.

Monitoring and Reporting
Data Quality reports and dashboards can be used to continuously monitor and govern your information assets.

SAP DQM, Micro-services

This is the SaaS (Software as a service) solution hosted by SAP for data quality. The DQM micro-services application offers cloud-based micro-services for address cleansing, geocoding, and reverse geocoding. You can embed address cleansing and enrichment services within any business process or application so that you can quickly reap the value of complete and accurate address data.

SAP Data Quality Management microservices allow developers to embed data cleansing and enrichment services within any business process or application to HTTPS/JSON-based services running on the SAP Cloud Platform. Developers can now simply integrate these services into their own applications to provide address cleansing/validation, geocoding, and reverse geocoding capabilities.

This address cleansing solution uses reference data from global postal authorities for 240+ countries and territories to validate or correct addresses. With address cleansing, you can quickly and easily correct, parse, standardise, validate, and enhance address data.
These features support:

  1. Address Cleansing
    • Verify address per postal authority reference data as correct
    • Standardise the way the address looks and is structured
    • Correct components of an address if incorrect
    • Enhance an address with missing attributes
    • Assign diagnostic codes which describe why they are incorrect or what was corrected
  2. Geocoding and Reverse Geocoding
    • Append latitude and longitude to any given address
    • Get an address from the latitude, longitude or any POI (Point of Interest)

These services can easily be integrated into any application. However, SAP is developing pre-built integration of these services in other applications, which will provide a simple way to consume this functionality.

Conclusion

The data quality solutions work best for the SAP environments and its DQM microservices solution is a step towards getting this solution used outside SAP. It’s only a matter of time before more and more SAP features will be open to the outside world.

References

  1. https://help.sap.com/viewer/74d4f1a3a6d6428d9f246f46e5320716/4.0.9/en-US
  2. https://wiki.scn.sap.com/wiki/display/EIM/Data+Quality
  3. https://help.sap.com/viewer/d95546360fea44988eb614718ff7e959/Cloud/en-US
  4. https://blogs.saphana.com/2017/03/22/sap-data-quality-management-microservices-generally-available-to-customers/

Co-author: Srinivas Bandi