Big Data, Data Engineering & Analytics

Roughly one in nine diabetic hospital stays in the dataset below ends with the patient back in hospital within 30 days. Readmissions are expensive, often preventable, and in the US Medicare penalises hospitals with excess readmissions for several common conditions. So the question a hospital actually asks is simple: at discharge, which patients should get extra follow-up? This tutorial answers that question end to end in Python, on a real public dataset of about 100,000 hospital encounters. You will clean the data, engineer features, avoid a common evaluation mistake, compare two models, and turn the scores into risk tiers a care team could use. Every number in this post comes from running the code shown.

Predictive analytics uses historical data, statistics and machine learning to estimate what is likely to happen next: which customers will cancel, how much stock a store will need, which patients are at risk of readmission. It does not predict the future with certainty. It produces probabilities and estimates that help people make better decisions. This guide covers the core concepts, the main techniques and models, a small Python example, business examples, and the mistakes that most often make predictions wrong. 1. Define the decision and target 2. Prepare historical data and features 3. Train models on past outcomes 4. Validate on unseen data with right metrics 5. Deploy score new cases, monitor, retrain Monitoring feeds back: when data or behaviour drifts, go back to step 1

Big data refers to data sets too large, too fast or too varied to be handled well by a single traditional database. Big data analytics is the practice of turning that data into decisions: collecting it, storing it at scale, processing it, and applying statistics, machine learning and visualisation to answer business questions. This guide covers the key definitions, the four types of analytics, a reference architecture, the main tools and techniques, real examples, and common pitfalls. Descriptive What happened? dashboards, KPIs Diagnostic Why did it happen? drill-down, root cause Predictive What will happen? ML models, forecasts Prescriptive What should we do? optimisation, rules Increasing value and difficulty →

Amazon CloudSearch is a fully managed service in the cloud that makes it easy to set up, manage, and scale a search solution. Amazon CloudSearch can search large collections of data such as web pages, document files, forum posts, or product information. CloudSearch makes it possible to search large collections of mostly textual data items called documents to quickly find the best matching results. Search requests are usually a few words of unstructured text. The returned results are ranked with the best matching, or most relevant, items listed first.

Cloudera Distribution including Apache Hadoop ( CDH ) is the most popular Hadoop distribution currently available. CDH is 100% open source. Cloudera quick start VMs include everything that is needed to tryout basic package based CDH installation. This is useful to create initial deployments for proof of concept (POC) or development.