This book is also part of our apprenticeship. Part of the content as well as new content is in a separate document called Addendum. Click here to download the addendum. The book is available on Barnes and Noble. Also, read our article on strong correlations to see how various sections of our book apply to modern data science. If you start from zero, read my data science cheat sheet first: it will greatly facilitate the reading of my book.
My second book - Data Science 2.0 - can be checked out here. The book described on this page is my first book.
About the Author
Dr. Vincent Granville is a visionary data scientist with 15 years of big data, predictive modeling, digital and business analytics experience. Vincent is widely recognized as the leading expert in scoring technology, fraud detection and web traffic optimization and growth. Over the last ten years, he has worked in real-time credit card fraud detection with Visa, advertising mix optimization with CNET, change point detection with Microsoft, online user experience with Wells Fargo, search intelligence with InfoSpace, automated bidding with eBay, click fraud detection with major search engines, ad networks and large advertising clients.
Most recently, Vincent launched Data Science Central, the leading social network for big data, business analytics and data science practitioners. Vincent is a former post-doctorate of Cambridge University and the National Institute of Statistical Sciences. He was among the finalists at the Wharton School Business Plan Competition and at the Belgian Mathematical Olympiads. Vincent has published 40 papers in statistical journals (including Journal of Royal Statistical Society - Series B, IEEE Pattern Analysis and Machine Intelligence, Journal of Number Theory), a Wiley book on data science, and is an invited speaker at international conferences. He also developed a new data mining technology known as hidden decision trees, owns multiple patents, published the first data science eBook, and raised $6MM in start-up funding. Vincent is a top 20 big data influencers according to Forbes, and was also featured on CNN.
Introduction
To find out whether this book might be useful to you, read my introduction.
Table of Content
- Introduction xxi
- Chapter 1 What Is Data Science? 1
- Chapter 2 Big Data Is Different 41
- Chapter 3 Becoming a Data Scientist 73
- Chapter 4 Data Science Craftsmanship, Part I 109
- Chapter 5 Data Science Craftsmanship, Part II 151
- Chapter 6 Data Science Application Case Studies 195
- Chapter 7 Launching Your New Data Science Career 255
- Chapter 8 Data Science Resources 287
- Index 299
Chapter 1 - What Is Data Science? 1
Real Versus Fake Data Science 2
- Two Examples of Fake Data Science 5
- The Face of the New University 6
The Data Scientist 9
- Data Scientist Versus Data Engineer 9
- Data Scientist Versus Statistician 11
- Data Scientist Versus Business Analyst 12
Data Science Applications in 13 Real-World Scenarios 13
- Scenario 1: DUI Arrests Decrease After End of State Monopoly on Liquor Sales 14
- Scenario 2: Data Science and Intuition 15
- Scenario 3: Data Glitch Turns Data Into Gibberish 18
- Scenario 4: Regression in Unusual Spaces 19
- Scenario 5: Analytics Versus Seduction to Boost Sales 20
- Scenario 6: About Hidden Data 22
- Scenario 7: High Crime Rates Caused by Gasoline Lead. Really? 23
- Scenario 8: Boeing Dreamliner Problems 23
- Scenario 9: Seven Tricky Sentences for NLP 24
- Scenario 10: Data Scientists Dictate What We Eat? 25
- Scenario 11: Increasing Amazon.com Sales with Better Relevancy 27
- Scenario 12: Detecting Fake Profiles or Likes on Facebook 29
- Scenario 13: Analytics for Restaurants 30
Data Science History, Pioneers, and Modern Trends 30
- Statistics Will Experience a Renaissance 31
- History and Pioneers 32
- Modern Trends 34
- Recent Q&A Discussions 35
Summary 39
Chapter 2 - Big Data Is Different 41
Two Big Data Issues 41
- The Curse of Big Data 41
- When Data Flows Too Fast 45
Examples of Big Data Techniques 51
- Big Data Problem Epitomizing the Challenges of Data Science 51
- Clustering and Taxonomy Creation for Massive Data Sets 53
- Excel with 100 Million Rows 57
What MapReduce Can’t Do 60
- The Problem 61
- Three Solutions 61
- Conclusion: When to Use MapReduce 63
Communication Issues 63
Data Science: The End of Statistics? 65
- The Eight Worst Predictive Modeling Techniques 65
- Marrying Computer Science, Statistics,and Domain Expertise 67
The Big Data Ecosystem 70
Summary 71
Chapter 3 - Becoming a Data Scientist 73
Key Features of Data Scientists 73
- Data Scientist Roles 73
- Horizontal Versus Vertical Data Scientist 75
Types of Data Scientists 78
- Fake Data Scientist 78
- Self-Made Data Scientist 78
- Amateur Data Scientist 79
- Extreme Data Scientist 80
Data Scientist Demographics 82
Training for Data Science 82
- University Programs 82
- Corporate and Association Training Programs 86
- Free Training Programs 87
Data Scientist Career Paths 89
- The Independent Consultant 89
- The Entrepreneur 95
Summary 107
Chapter 4 - Data Science Craftsmanship, Part I 109
New Types of Metrics 110
- Metrics to Optimize Digital Marketing Campaigns 111
- Metrics for Fraud Detection 112
Choosing Proper Analytics Tools 113
- Analytics Software 114
- Visualization Tools 115
- Real-Time Products 116
- Programming Languages 117
Visualization 118
- Producing Data Videos with R 118
- More Sophisticated Videos 122
Statistical Modeling Without Models 122
- What Is a Statistical Model Without Modeling? 123
- How Does the Algorithm Work? 124
- Source Code to Produce the Data Sets 125
Three Classes of Metrics: Centrality, Volatility, Bumpiness 125
- Relationships Among Centrality, Volatility, and Bumpiness 125
- Defining Bumpiness 126
- Bumpiness Computation in Excel 127
- Uses of Bumpiness Coefficients 128
Statistical Clustering for Big Data 129
Correlation and R-Squared for Big Data 130
- A New Family of Rank Correlations 132
- Asymptotic Distribution and Normalization 134
Computational Complexity 137
- Computing q(n) 137
- A Theoretical Solution 140
Structured Coefficient 140
Identifying the Number of Clusters 141
- Methodology 142
- Example 143
Internet Topology Mapping 143
Securing Communications: Data Encoding 147
Summary 149
Chapter 5 - Data Science Craftsmanship, Part II 151
Data Dictionary 152
- What Is a Data Dictionary? 152
- Building a Data Dictionary 152
Hidden Decision Trees 153
- Implementation 155
- Example: Scoring Internet Traffic 156
- Conclusion 158
Model-Free Confidence Intervals 158
- Methodology 158
- The Analyticbridge First Theorem 159
- Application 160
- Source Code 160
Random Numbers 161
Four Ways to Solve a Problem 163
- Intuitive Approach for Business Analysts with Great Intuitive Abilities 164
- Monte Carlo Simulations Approach for Software Engineers 165
- Statistical Modeling Approach for Statisticians 165
- Big Data Approach for Computer Scientists 165
Causation Versus Correlation 165
How Do You Detect Causes? 166
Life Cycle of Data Science Projects 168
Predictive Modeling Mistakes 171
Logistic-Related Regressions 172
- Interactions Between Variables 172
- First Order Approximation 172
- Second Order Approximation 174
- Regression with Excel 175
Experimental Design 176
- Interesting Metrics 176
- Segmenting the Patient Population 176
- Customized Treatments 177
Analytics as a Service and APIs 178
- How It Works 179
- Example of Implementation 179
- Source Code for Keyword Correlation API 180
Miscellaneous Topics 183
- Preserving Scores When Data Sets Change 183
- Optimizing Web Crawlers 184
- Hash Joins 186
- Simple Source Code to Simulate Clusters 186
New Synthetic Variance for Hadoop and Big Data 187
- Introduction to Hadoop/MapReduce 187
- Synthetic Metrics 188
- Hadoop, Numerical, and Statistical Stability 189
- The Abstract Concept of Variance 189
- A New Big Data Theorem 191
- Transformation-Invariant Metrics 192
- Implementation: Communications Versus Computational Costs 193
- Final Comments 193
Summary 193
Chapter 6 - Data Science Application Case Studies 195
Stock Market 195
- Pattern to Boost Return by 500 Percent 195
- Optimizing Statistical Trading Strategies 197
- Stock Trading API: Statistical Model 200
- Stock Trading API: Implementation 202
- Stock Market Simulations 203
- Some Mathematics 205
- New Trends 208
Encryption 209
- Data Science Application: Steganography 209
- Solid E‑Mail Encryption 212
- Captcha Hack 214
Fraud Detection 216
- Click Fraud 216
- Continuous Click Scores Versus Binary Fraud/Non-Fraud 218
- Mathematical Model and Benchmarking 219
- Bias Due to Bogus Conversions 220
- A Few Misconceptions 221
- Statistical Challenges 221
- Click Scoring to Optimize Keyword Bids 222
- Automated, Fast Feature Selection with Combinatorial Optimization 224
- Predictive Power of a Feature and Cross-Validation 225
- Association Rules to Detect Collusion and Botnets 228
- Extreme Value Theory for Pattern Detection 229
Digital Analytics 230
- Online Advertising: Formula for Reach and Frequency 231
- E‑Mail Marketing: Boosting Performance by 300 Percent 231
- Optimize Keyword Advertising Campaigns in 7 Days 232
- Automated News Feed Optimization 234
- Competitive Intelligence with Bit.ly 234
- Measuring Return on Twitter Hashtags 237
- Improving Google Search with Three Fixes 240
- Improving Relevancy Algorithms 242
- Ad Rotation Problem 244
Miscellaneous 245
- Better Sales Forecasts with Simpler Models 245
- Better Detection of Healthcare Fraud 247
- Attribution Modeling 248
- Forecasting Meteorite Hits 248
- Data Collection at Trailhead Parking Lots 252
- Other Applications of Data Science 253
Summary 253
Chapter 7 - Launching Your New Data Science Career 255
Job Interview Questions 255
- Questions About Your Experience 255
- Technical Questions 257
- General Questions 258
- Questions About Data Science Projects 260
Testing Your Own Visual and Analytic Thinking 263
- Detecting Patterns with the Naked Eye 263
- Identifying Aberrations 266
- Misleading Time Series and Random Walks 266
From Statistician to Data Scientist 268
- Data Scientists Are Also Statistical Practitioners 268
- Who Should Teach Statistics to Data Scientists? 269
- Hiring Issues 269
- Data Scientists Work Closely with Data Architects 270
- Who Should Be Involved in Strategic Thinking? 270
- Two Types of Statisticians 271
- Using Big Data Versus Sampling 272
Taxonomy of a Data Scientist 273
- Data Science’s Most Popular Skill Mixes 273
- Top Data Scientists on LinkedIn 276
400 Data Scientist Job Titles 279
Salary Surveys 281
- Salary Breakdown by Skill and Location 281
- Create Your Own Salary Survey 285
Summary 285
Chapter 8 - Data Science Resources 287
Professional Resources 287
- Data Sets 288
- Books 288
- Conferences and Organizations 290
- Websites 291
- Definitions 292
Career-Building Resources 295
- Companies Employing Data Scientists 296
- Sample Data Science Job Ads 297
- Sample Resumes 297
Summary 298
Index 299
Other links
You need to be a member of Data Science Central to add comments!
Join Data Science Central