This weekend I spent hours learning prediction.io. PredictionIO is an open-source machine learning server for developers and data scientists to build and deploy predictive applications in a fraction of the time.
Prediction.io makes machine learning very easy to learn. Its architecture makes it easy to train the machine and deploy it. It is based on templates with a DASE architecture. And since it is built on top of Apache Spark with MLlib, all the MLlib algorithms are available with prediction.io.
I’m not going to tell you about prediction.io in detail — you can read the complete documentation on their site. Instead I want to write about my research so far with prediction.io.
I’m building a recommendation engine to learn a complex set of data. I want to achieve a machine that can learn that data and recommend the most similar items and what users are looking for.
Imagine an eCommerce platform — after a while you’ve viewed and purchased products, the machine could:
- Offer you a similar product while you’re browsing
- Email you promotions based on your purchase history
- Offer you a one-time promotional deal based on products you’re interested in
- Predictive search based on current trends
The applications of recommendation engines, prediction engines, and similar-product engines are many.
What I’ve learned so far from prediction.io:
- The prediction result is not exactly what I wanted, but it is close. And more training data doesn’t necessarily mean more accuracy.
- The training part is still unclear to me — I need more time to learn how it works.
- The machine cannot be trained in real time, which means it cannot receive an event and recalculate the prediction. You need to shut it down, train, and redeploy. There is a script for retrain and redeploy without downtime, but basically the machine needs to retrain frequently based on the events collected.
- Prediction.io uses Apache Spark, and can use a cluster of cloud compute for big data.
- I got it working for recommendation and similar-product prediction based on real data I have. The prediction result is quite fast.
- It takes some time to do pio build and pio train. For large data sets, pio train took 5 minutes to finish.
- Prediction.io is available in a Vagrant version for learning.
I wrote this for my own records, but if it helps you and you have something in mind, just let me know.