Master Person System

The Person Data Entity (hereafter also referred to as “Party”) is usually the key to any business information system. A Party may represent individuals (e.g., leads, customers) or legal entities (e.g., suppliers, customers).

The usual motives for implementing Master Data Management for a Party entity are to address the following problems and needs:

• solving personal Data Quality problems,
• the need for Party uniqueness within the client’s entire information system,
• the need to consolidate Party attributes from multiple systems,
• creating the so-called Party Golden Records as the best representation of persons.

Master Person System is our solution for Master Data Management for the Party entity. Our solution is not a single compact product per se, rather a set of assets, which we have implemented and utilized successfully and will apply in our future engagements.

Key Benefits

• Our approach to the solution reflects the successful delivery of an analogous solution for Master Data Management for the Party domain for a leading Czech financial institution.
• We are ready to implement our approach in both Cloud and On-Premises variants; we assume that the client chooses the more suitable variant according to his specific situation.
• The implementation of our approach is based on enterprise technologies delivered by strong vendors with a global footprint (we do not propose solutions based on local or our technologies).
• For all the proposed technologies, there is, and it can be assumed that there will be, a wide range of professional capacities in a competitive environment.
• The proposed solution options are open in the sense that for future changes and development of the application logic the client will have flexibility in the choice of suppliers (not vendor lock-in).
• We design solutions that meet the specific conditions and requirements of the client (not a “one size fits all” approach).

Key Concepts

Our approach to a systemic solution to the above problems and needs is to implement a centralized consolidation system. The diagram below illustrates the fundamental change in the concept of consolidation that our solution brings – moving from fragmented consolidation to centralized consolidation.

The Master Person System (MPS) approaches the above problems and needs in the following way:

• Data quality: The MPS system takes into account the Data Quality of the source data, reference data (external registers, internal codebooks) and contributes substantially to improving the Data Quality of personal data through centralised consolidation when creating the Golden Record.
• The need for Party uniqueness across the client’s information system: The MPS system is designed as a solution that is integrated with the client’s other systems and creates Golden Records that are unique across the entire information system.
• The need to unify (consolidate) Party attributes from multiple systems and create a Golden Record: The MPS is integrated with primary data sources and implements mechanisms for creating, maintaining and distributing Golden Records.

---

System Integration

The Master Person System processes personal data from primary personal data sources, distributes Golden Records to subscriber systems, and provides an interface for managing Golden Records.

The diagram below shows typical integrations where some systems are only primary data sources, some systems are consolidation consumers (of Golden Records), and some systems are both primary data sources and consolidation consumers.

Personal Data Sources Integration

• Web Services: Find, create and update a Golden Record by calling Web Services methods (see below for a list of methods)
• Near Real-Time: Receiving messages (Kafka) when personal data is changed in primary data sources, for subsequent processing of changes in consolidation
• Batch: Batch integration of personal data changes in primary data sources for subsequent processing of changes in consolidation; Batch integration is optional, complementary to Near Real-Time.

Golden Record Subscribers Integration

• Near Real-Time: When changes are made to the Golden Records (consolidation changes), messages (Kafka) are sent for subsequent processing of the changes in the customer systems.
• Batch: Batch integration of Golden Record changes for subsequent processing of changes in subscriber systems; Batch integration is optional, complementary to Near Real-Time.

Web Services API Methods

For interaction via Web Services, the Master Person System provides an API with the following methods:

• Finding the Golden Record based on the attributes passed
• Creating a Golden Record based on the attributes passed
• Updating the Golden Record according to the attributes passed
• Manually linking two person records as relating to an identical person
• Manually disconnecting person connections (based on additional information)
• Checking Data Quality of the personal data sent against data in the Master Person System and/or data obtained from external registers
• Getting suspicious matches for the assessment and decision on the match
• Confirming suspicious matches (with less than the minimum required trustworthiness)

Methods for managing consolidation (e.g. Manually disconnecting, Confirming suspicious matches) can either be called from a specific Master Person System front-end, or, (which we recommend) consolidation management can be built into the system in which persons are generally managed – typically in a CRM system.

---

Model Architecture

Our solution can be implemented both in an on-premises environment (see Client Story) and in a cloud environment. Below we present a model architecture of a variant of the Master Person System solution in the cloud. The selection of technical components is made according to the specific situation and client preferences.

In the presented model architecture, integration with the CRM system and other source systems and Golden Record consumers is solved using Azure integration technologies and existing on-premises client technologies. Consolidation, the creation of Golden Records, is implemented in the Azure SQL Database, with some of the consolidation application logic in Azure App Service. Interface management, handling queries from surrounding systems, and addressing manual verification is implemented in the Azure App Service.

---

Internal Processing

The diagram below shows a simplified technical view of the consolidation processing.

1. Loading data from source systems into Landing Layer: Batch loading of personal data images from source systems into the input area for downstream processing.
2. Change Detection and Historization to Stage Layer (Ingestion): In the case of batch processing, detected changes in Landing Layer vs Stage Layer are processed and historized to Stage Layer. In the case of Near Real-Time and Web Services, changes are processed and historized directly to the Stage Layer.
3. Transformation of source data: Some source attributes may need to be pre-processed before enriching the personal data with information from external registers.
4. Enrichment of source data from external registers (Enrichment): Using selected source attributes, information about processed person records is searched in external registers and the records are enriched with this information.
5. Data Quality Control & Trust Calculation (DQ Control & Trust Calculation): Data quality checks (DQ Checks) are applied to processed person records to identify record attributes that do not meet the required Data Quality. The output of the DQ Checks is used to calculate the trustworthiness of the input data (Trusts) which affects the Matching, Merging & Splitting processes. The processed changes are then standardized to a single ‘format’ and processed and historized into the Core Layer.
6. Matching: Matching is processed over the transformed, enriched, validated and standardized records to identify groups of personal records, where each group represents an identical person. Matching is controlled by implemented rules (Matching Rules) and computed input data Trusts.
7. Creating Golden Records for each person (Merging & Splitting): For each group of records that represent an identical person, Merging is performed – creating a Golden Record that best represents that group of records. Merging is controlled by implemented rules (Merging Rules) and calculated input Trusts. This step may also include processing the splitting of previously merged entities.
8. Enrichment of source data from external registers (Enrichment): As a result of Merging, new information can generally be obtained for the resulting Golden Record by querying external registers. As part of this enrichment step, new Golden Records are registered to subscribe to updates of information from registers.
9. Loading Golden Records into Core: After the enrichment of changed Golden Records from external registries is updated, these changes are processed and historized into the Core Layer.
10. Distribution to consumers: Changes to Golden Records are distributed to subscriber systems by Near Real-Time and Batch mechanisms. Specific requirements for the structure and format of consolidation information are addressed at the Integration layer.

Configuration is used in the above processing and Operational Metadata is produced.

Configuration

• DQ Checks Configuration: Setting rules for Data Quality checks
• Trusts of Matching Attributes per Source: Source data trust settings
• Matching Rules: Setting up consolidation rules
• Merging & Splitting Rules: Setting rules for creating Golden Records

Operational Metadata

• Data Preparation Log: Record of data changes in the consolidation processing
• DQ Issue Log: Evidence of Data Quality findings (= inconsistencies with DQ Checks)
• Matching Log: Records matches and splits, including manual interventions
• Merging & Splitting Log: Evidence of creating Golden Records
• Suspected Matches – together for both suspicious matches and suspicious mismatches
• Log: Technical log from the processing (Trace, Debug, Info, Warn, Error, Fatal levels)