Showing posts with label big data usage. Show all posts
Showing posts with label big data usage. Show all posts

Monday, February 26, 2018

Machine Learning, AI, Bots & ???

From last couple of years, we are hearing about Machine learning. What exactly is machine learning in layman’s term? It’s empowering systems / machines to learn and understand if there is an inherent pattern underlying huge pile of similar data. Something like we are trying to prove chaos theory in practicality.  And this can be done through algorithms trying to find similar or dis similar data to map against same plotting paper. Let’s see some ways how these algorithms are working,
  • Deep Learning – This is like an artificial human brain. Where interconnected artificial neurons are stacked as multiple layers, and every two layer gets an additional processing layer which processes the data before passing it to the other layer. This processing layer gives out abstraction, classification, segmentation, prediction etc.
  • SVM (Support Vector Machines) – These are infused with separate learning algorithms. Regression analysis is the backbone of these kind of structure. While the data is analysed, classified and then used for regression analysis to understand the relationships between different data points and how these data points are depended on each other. For an example this analysis defines different kind of variables like independent variables and criterion variable. Then it measures the change in criterion when one of the independent variable value gets changed keeping other independent value as static.
  • Probabilistic Model – As the name suggests it intakes all information’s and runs them through probability models. This model creates a probability distribution map to find out all possible outcome with the degree of certainty.

  • Ensemble Model – This is a kind of probabilistic model, while it gives one single output. Mainly this is used behind scene to help other machine learning programs.


Now these above-mentioned models can be little tech oriented, so let’s generalize them and see how or where we are using it now, 

Deep Learning – Big Data, Voice Recognition, and conversational skills. SVM – Facial or handwriting recognition, Probabilistic model – this getting used by search engines to provide relevancy result, Ensemble Model – any type of machine learning methods

Every day we are creating data which is in data mode only unless we keep a machine to read through them and churn out information’s. These data can be created from anywhere starting from online activity, hand held devices, phones, anything and everything that is connected to the network. Size of these data is simply impossible for humans to scan and understand. That’s why we need these enabled machines. But it’s not about only enabling systems to learn, but effectiveness lies under the method of madness. e.g. what exactly we are looking for and how we are intending to utilize the data. There are few instances where these learning's are getting utilized but there is hardly any example to showcase it’s full potential. As along with the concept the environment and it’s demands are changing with time. But let’s see how we are utilizing these currently.
  • Segmentation – What we marketer used to do by our own, not it’s changed. Machines are doing it for us, and considering the competition and other factors they are doing a bloody good job. Now these segments are more fine-tuned, as per not only behavior but audience’s mood or work pattern. It’s an exciting time indeed, for people who are utilizing this to the fullest. Yes to get the correct fit and finish everyone have to continuously work on, but this the best possibility currently.
  • NLP (Natural Language Processing) – This is my personal favourite. A Natural Language with Machine Learning Algorithm can be a primary customer service agent. Much fast, to the point, and effective (in a debatable way).
  • Market Right Price – If you see an ad which says this slashed price of product X is only for you, consider that as TRUE. Previously e-commerce giants used to reduce the price a product for everyone, but now it’s not the case. They are putting the price segment into ‘market right price’ model (obviously with a range) which finds out what’s the optimum price you would go and buy for. This price is not for everyone, it’s specific to you and persona you fall under. How they are doing it? Simple answer will be by analyzing your behavior
  • Trend Analysis – This is where machine learning's are helping business to the most. It’s single handedly not only takes care of predictions but also dishing out solutions of how to tackle upcoming future.
  • Data Security – This is one segment where machine learning is helping immensely. It’s tracking anomalies to detect security breach or malware detection and future prevention.  
  • Prediction System – Machine learning is getting utilized for financial sector where it’s predictions to identify investment opportunities and probable fraud by pin pointing high risk profiles or cyber surveillance for sings of fraud.  It’s even utilized in healthcare for diagnosis and primary prescription. CAD(Computer Assisted Diagnosis) is an example of such utilization.  Even transportation department is using prediction tools for mapping out patterns and trends to plan for future events and new routes.
Till now some people might be thinking I am confusing AI with Machine Learning. But the answer is no. In simple word Artificial Intelligence is a concept and Machine Learning is just an application under the same concept. While AI is the whole gamete, Machine Learning is the prize crown currently.

Now lets looks into some more directly impact of AI into marketing facet
  1. Content delivery companies are already using AI to curate content for you, it knows what’s your behavior map. As a result, you are seeing news, stocks, product listing, recommendations as per your behavior. It’s a usage of clustering algorithm
  2. Google itself is using RankBrain which is a AI system
  3. Newer AI implementations, such as that used by the United Services Automobile Association (USAA, which provides financial services for ex-military), will identify anomalies in behavior even on the first instance.
  4. We still have somewhat fresh memories of Microsoft's AI chatbot 'Tay' may be for some different reason, but it’s next version with deep learning can overcome the previous issues it faced. Even Facebook AI research is making some news
  5. Many websites are using AI to define their design for you specifically.
  6. USAA used the AI technology built by Saffron, now a division of Intel for predictive customer service. This AI has so far helped USAA improve its guess rate from 50% to 88%, increasingly knowing how users will next contact and for what products.
  7. Programmatic media buy is already optimizing bids for advertisers, algorithms to achieve the best cost per acquisition (CPA) from the available inventory. When it comes to targeting of programmatic ads, machine learning helping to increase the likelihood a user will click. This might be optimizing what product mix to display when re-targeting, or what ad copy to use for what demographics.
  8. Siri and Cortana already using speech recognition. Not only that even some translators like Skype translator is using neutral networks. Speech recognition is only the start AI systems are helping to identify the language too
  9. Google image, facebook face recognition, or snapchat face swap are based on AI systems. Even AI is into AR, it helps all AR systems with sophisticated recognition of landscape.
  10. Last but not the last for obvious reason will be bots. These little pieces of codes have to power to replace many things like 1-800 numbers (a prediction made by TechCrunch), revamping the experience of any customer care. There are lots of usage which many people know here already.
Does these examples shows that these systems are full proof? The simple answer is no. Only reason it human interactions are multi-layered and multi facet. So unless these systems are multi-layered and backed up with enough (how much will be enough is a debatable as every human being is unique and their choices and conversations too), then these systems are bound to fail.

It's worth pointing out that most important piece of this puzzle is human element, because AI and machine learning still need people, such as Google's raters. They are improving accuracy and I turn training algorithms to become perfect. Crowd-sourcing of workforce (e.g. Amazon's Mechanical Turk) will be the next big industry because of increased usage of AI and simultaneously growing requirement of human's guiding hand to adjust data-sets.

However, if you're doing a job that could conceivably be automated, AI could be more and more of a pressing issue.

This is a personal scribble ground. The opinions expressed here are all personal they do not express any direct or indirect linkage to the organization I am attached to. But I love changes so they might be changing time to time. :o)

Wednesday, November 5, 2014

Data – From Big to Small and Real to Old

From last couple of days I am thinking about different marketing campaigns their designs, usage of different channels and most importantly role of data playing in them. During this wasted thinking items I was asking one question to myself.

Is it a good idea to go after real time data or depend on your predictive models for marketing? Now let me tell you why I am behind data this time. If I am correct the most important and expensive piece in any planning is data. More specifically so called real time data, you must be wondering why I am calling it ‘so called’ when the whole world is calling it otherwise?

The main reason behind this is why we call it as real time is the data just came out of the system. It’s as true / valid as it can be. It gives you the knowledge of what your customers / listeners / readers / prospective customers / the WORLD is thinking right now or doing right now or looking for right now. It can make you king / queen as you have the power to use the data as you want to make you successful. Now after all these you must be thinking that if I am agreeing to this then why I am not supporting the concept of it? The idea of using the real time data and become successful is a myth.

Four points to support my points,


  • Real time data are way too expensive and get the actual value becomes hard for some company
  • Getting meaningful information from those huge / small and clustered / un-clustered data takes some time
  • Mold your existing / planned campaign with respect to the market demand and the kind of information coming out of these real time data takes time
  • Decision making process in every company takes its own sweet time depending upon the company size and decision making flexibility


In above four points last three ones can take any time between couple of days to weeks to months, which actually makes the data no more real time but old time. As you know real time data in consumers market changes every day and business market might be not that fast because but it changes every quarter. So by the time you are ready to take an action on the data it became invalid for current time as some vendor will be ready to give you real time data of that date.

Once all these thinking done, I was looking for an easy way out. And guess what I had one which cheaper too. Let’s use the older data (a year may be) and a proven predictive model to give you an answer of what you are looking for. I think the older data are cheaper than real time. And I always feel comfortable working with older data and a proven statistical models to get the answers regarding campaign planning. This plan does work for me (Would love to know your experience on this).

Exclude the artificial intelligence systems which are making use of real time data, because they are best combinations for this concept. Gist of it can be if you have to make use of real time data in proper way, prepare a human free model.

Now the question is whether you are ready to pay, loads for something which you can’t actually use as it’s sold. It would be great if you can let me know your take on this in comments below, as I love to change my concepts if I am wrong.

This is a personal scribble ground. The opinions expressed here are all personal they do not express any direct or indirect linkage to the organization I am attached to. But I love changes so they might be changing time to time. :o)