New generation data warehouse solutions for insurance industry

Scheme background

With the rapid development of the Internet, big data and cloud computing technology, the insurance industry, as an important member of the financial field, faces many problems such as how to expand channels, how to change direction timely and how to make rapid innovations. It is the key direction of business promotion and innovation implementation for the current insurance industry to reform and improve existing management methods, information system construction, data integration and analysis methods. As the core of data integration and processing, data warehouse not only undertakes the data ETL function with insurance core system and other business systems, but also bears important data processing functions such as data integration, data analysis and mining. By considering higher requirements for timeliness, faster changes in data demand, and larger data capacity currently, the traditional data warehouse technology architecture has been unable to meet the new demand, so a new generation of data warehouse solution is required urgently.

Scheme content

The new generation of data warehouse in insurance industry adopts the hybrid architecture of big data platform (Hadoop) + distributed database (MPP), including data acquisition layer, big data storage and processing layer, big data mining and display layer, big data application layer, big data control center, operation and management center, etc. Both structured and unstructured data can be collected in real time and non-real time via the platform and stored in distributed file system, which are processed, calculated, and mined based on different computing engines such as offline, memory and real-time stream engines to finally realize the visualization of data value and the data support for different application scenarios.

Scheme value

The new generation of data warehouse solves many technical and cost problems such as real-time streaming data processing, unstructured data processing, low efficiency of massive data calculation and query, data not available all the time, and high expansion cost that cannot be satisfied through traditional data warehouse, and comprehensively responds to new demands: 1. Acceleration of data ETL process: from data acquisition of the source system, various distributed, high-performance, and highly reliable technical components are used to perform the data ETL, so that the changes of business data in the source system can be fed back to the data warehouse in a real-time and quasi-real-time manner. 2. Efficient data integration and processing: the distributed memory computing and other means are applied to easily cope with the integration and processing of massive data, promote the client experience in the application layer, and improve the timeliness of decision support. 3. Real-time data processing: in the face of massive real-time data generated by the application system, the real-time streaming data processing technology is used to complete the data calculation and generate the results in milliseconds and seconds, quickly responding to changes in external demands. 4. Data load reduction and investment protection: the new generation of data storehouse technology can reduce the load of the existing data storehouse and separate data, ensure the storage and flow of different value data and protect the original investment of the enterprise.