The major trend observed across industry and the public sector is artificial intelligence (AI)/machine learning (ML) for automation. This, in turn, plays a major part in any digital transformation journey. The trend grew out of the Bay Area, providing a customer-centric view of data and often involved using data as part of the product or service. This consumer- or customer-centric model assumes data enrichment with data from multiple sources. However, fundamentally, it divides the data into two main areas. Real-time data and historic data. Pivotal, whose initial research forms the basis of this thinking, names this as “fast” data and “big” data.
Streaming or “fast” data
The first part of the chart describes the high value of data as recognized in the sub-second space of finance and into the few-second duration of the mobile user/web shopper. The chart does not describe — but the whiteboard session does highlight — how fiscal information becomes of extreme high value if the organization is required to commit statements of fiscal performance to stock-markets/analysts, etc.
Dell EMC Research (https://www.delltechnologies.com/en-us/perspectives/realizing-2030.htm) found consumers to have something in the region of a 6-second attention span. Delayed data or a poor connection results in direct loss of consumer business and — if consistently delayed or poor — loss of consumer confidence in the product.
Quality = Trust: All done at scale and at speed
Behind the chart is the reality of data-driven business. That in delivering information into the hands of customers and business partners the organization is developing a trust relationship. The better the quality of information, the higher trust exists between parties. Trusted data (accurate information) is used to drive relationships with users and business associates. Indeed, data, as a part of the wider relationship, the fair and trusted exchange of information, is becoming the central model for many consumer services (for example, Uber, Facebook, Amazon, Spotify). Online gambling is an example where accuracy, speed (of data) and trust (reliability) is essential and creates differentiated services. The telco-media and entertainment industry, in general, recognized some time ago that strong trusted consumer relationships based on high-quality personalized data deliver consumer brand loyalty and typically increased spend.
Data value increases over time
The greatest change in the overall value of data is that older data is becoming a high-value asset. This trend became apparent when:
- Web-based business needed to understand a customer that it never actually meets.
- Data scientists, mathematicians and traditional business analysts were employed to build profiles from every customer interaction and, against these, predict and effectively recommend, at high levels of consistency, more or new products.
To make accurate consumer predictions, the technology needed to be fed a lot of data. Different industries began to recognize that corporate wisdom needed to be captured. ING Bank call this its corporate memory.
Success stories about the organizations whose examples make up the poster children for data evangelists retained/stored and used data that was always most likely to result in competitive advantage. They do not necessarily keep all data, instead — and by design — they keep what is most likely to prove to be beneficial.
The impact of automation
The advent of advanced predictive analytics, machine learning and artificial intelligence at scale and with a commodity price has driven the need for the “corporate memory” to be rapidly adopted in many organizations.
Data silo and data lake
The charts, in fact, make no suggestion of where or indeed how to store data. Neither do the charts suggest one type of data storage is preferable.
- Corporate memory is a HIGHLY personal (corporate entity) thing. Therefore, having different parts of the corporate memory in different silos may make perfect sense.
- It may be legacy mandated and may — for security purposes — be a very practical way of segmenting elements that have different corporate levels of security applied.
- Not all data is of identical value — and not all data makes sense to the automated world of AI.
- Loosely coupled silos — with appropriate firebreaks and security gaps — can also contribute to the overall corporate memory and the super-fast data usage in the age of AI-driven data use.
Summary and action
The age of commodity-based AI is extremely close. The pattern of data flow demonstrated in these three charts maps directly into the digital world in which AI is a major automation factor. The corporate memory becomes the governance condition for the super-fast world of machine to machine (M2M) interaction. At a simple level, to avoid a flash crash, AIs need to be built upon real-world examples. This alone creates the high value of both “newly created” and “long term” data.
Action now: Review where you are in the final chart; pinpoint where you are under-indexed in terms of investment. Remedy that — gain capability. And, in terms of building corporate memory, every day lost is a day of data gone. Make sure you can get back the data you store. Start with storing an overkill amount and then pare back. Data you keep today can be disposed of tomorrow. The opposite is not true. Whatever you start with — start today.