Deep Learning Blows Up Your Data Strategy
Deep learning (deep structured learning, hierarchical learning or deep machine learning) is based on a set of algorithms that attempt to model high-level abstractions in data by using multiple processing layers.
The implications for how you manage data are radical. Here is what you need to think about:
Deep learning systems are voracious eaters of data. If you think you have volume issues now, it will only get worse. Traditional integration won't cut it. You need bigger compute on GPUs not CPUs for speed, performance, and efficiency. Don't you want to train your data in 2 hours vs. 2 weeks?
The deep learning algorithm (convolutional neural network - CNN) is the data management tool. Traditional analytic models sit outside the infrastructure. In the case of AI, a generic CNN sits with the processor. Classification and inference happens on data ingest.
The deep learning system is the expert - experts need not apply. Traditional analytics and expert systems relied on coding instructions. This included coding for transformations and mapping. SMEs were roped in to "teach" what data decisions should be made. Today, you say what the data should do, and the system aligns it to the purpose.
Deep learning systems learn data governance. Data governance policies will need to be taught. Stewards won't create business rules. DL systems need to be told what you want them to do and provide training data they can test and learn on.
Deep learning systems have deep memories. CNNs will maintain up to 180 layers (maybe more) that have to be immediately referenceable in real-time. You need to consider volume not only in what you take in, but how you store it. Graph isn't a nice to have any more, it is a necessity to hold contextual memory. Cloud and big data lakes allow data scale at lower cost. Storage and GPU compute are converged.
If there is one thing to keep in mind when embarking on this new journey of deep learning in pursuit of artificial intelligence, it begins with data. Keep these five strategy shifts in mind as you introduce AI compute platforms into your organization.
