PC to Cloud utility In Informatica!
March 11, 2019How to Implement Incremental load in Informatica PowerCenter?
March 11, 2019Composite Rest Webservice for IDQ and MDM :
Webservice for IDQ and MDM : Composite rest API combines invocation of individual operations of the web services involved.
- Any system to send data into MDM will be processed before through IDQ. If IDQ validates the data, only those can be sent to MDM.
- Based on the input, 3 actions such as Insert, Update, or Delete should be supported.
- An operation like Insert and Update is done based on the availability of Parent Object in MDM. If Parent is available Insert Operation will be considered else Update Operation will be considered.
Delete Operation will be considered only when Update Operation will be considered.
- Insert Operation –
webservice for IDQ and MDM
- Parent and Address data will be combined and processed in a single transaction.
- For Inserts, the Parent/Address messages should be processed before associated child messages such as Phone or Alternate ID.
- Data that failed validation in DQ or was nullified through the cleanse process should be captured in exception files for data analysts to review. All attribute/data values should be written to the exception files as received from the source
- The following data validation errors will cause the transaction to fail:
Lookup Failure
Invalid DBA Name
Invalid First or Last Name
Invalid Address (applies only to the address that failed). - The DBA Name (for Organization as Parent) or the First Name and Last Name (for Person as Parent) and at least one address for the Organization or Person must pass DQ Validation. Addresses that pass DQ validation but have been identified by Address Doctor as invalid should still be sent to MDM. If all addresses for the organization fail DQ validation then the insert transaction for the organization and any associated child transactions should fail.
- Address updates that pass DQ validation will be processed in MDM, whether the address was validated with Address Doctor, and regardless of the status from Address Doctor validation.
- A child transaction should fail if the main attribute value is NULL. For example:
A Phone upsert transaction should fail if the phone number is NULL.
An Email upsert transaction should fail if the email address is NULL.
An Alternate ID upsert transaction should fail if the alternate id is NULL.
If a child transaction fails, an associated parent transaction can still be processed.
- Update and Delete Operation –
- Every Object will be processed in a separate transaction so that even if one gets failed others should proceed for processing.
- Delete operation is being supported with the attribute in the payload. if the attribute is set to yes, the related instance of the object should be deleted. Delete will always be supported in the Update operation. If the payload contains a delete attribute which has value, yes, but the operation is inserted, delete should never be considered.
Prepare message structure for MDM and call MDM Business Entity Service to write data into Informatica MDM solution
Send the success and failure response with the details of who is calling the web service.
Technical Challenges (to Developers)
A suggested workaround is to develop a REST API which only supports POST operation for Insert, Update and Delete and there will be only one end-point URL for all resources.
Solution: Webservice for IDQ and MDM
Approach: Webservice for IDQ and MDM : A Composite REST Webservice API should be developed so that it will call IDQ webservice for validation before inserting or updating in MDM through MDM BES REST API call
Technical Solution: Develop the REST API that calls IDQ SOAP Webservices for validation and BES REST call to insert and update.
- Any system to send data into MDM will call this composite service.
- Based on the input, composite web service will have 3 actions such as Insert, Update or Delete.
- To support these 3 actions, Composite API calls MDM BES before any operation to check the availability of Parent.
- If Parent is available Insert operation will be considered else Update will be considered.
- Id should be recorded for the child object to insert and every object to update.
- Only POST as HTTP Method will be supported.
- Only one endpoint URL will be there.
- The Organization and Person as the resource will be provided through Header.
- Source System Name and Source System Id should be provided through Header.
- Composite service will apply business validation and lookup checks to form input BES message for MDM, i.e. in case of multiple addresses and children, the one with an invalid will be ignored and a valid one will be processed.
- webservice for IDQ and MDM The standardized output from IDQ which is being supported will be used only to form BES Message for MDM.
Advantage: Webservice for IDQ and MDM: This solution includes the IDQ Validation before gets inserted or updated to the MDM and every transaction will be separate so that even if one fails others still proceed except the parent should not fail in Insert.
Issues: No Issue.
Conclusion: Every Child object will be independent of each other while inserting or updating.