3 Simple Rules For Getting Results With Data Science (A Story About Learning From Failure)
Written by Matt Dancho
Are you finding it difficult to show your organization results with data science?
Then let me tell you a story from my consulting experience.
Here’s how I went from bad to good at data science consulting by learning 3 Simple Rules.
Data Science Consulting is Intense
Data Science Consulting is an intense role to be in. Businesses only care about one thing - Results! If you can’t deliver, you get cut.
I wasn't always good at consulting. I got cut.
This is a story about rebounding from failure.
My Bag of Tricks
I began data science consulting having a bag of tricks that I’d developed by through research and experimentation on a lot of business problems.
In my research, I eventually stumbled upon 2 key packages that improved my abilities 10X.
H2O + LIME - The combination was a powerful 1-2-punch that enabled me to get amazingly high model performance plus explainability in minutes.
H2O Automatic Machine Learning (AutoML)
H2O Automatic Machine Learning - Get 30+ Models in Minutes
H2O is great. The open source H2O AutoML Algorithm builds 30+ models in minutes, an efficiency that would take me hours or days to match.
H2O quickly became a core tool in my toolbox.
But, in business, explanations are most critical, which is what led me to…
Local Interpretable Model-Agnostic Explanations (LIME)
LIME - Explain Business Insights
I needed a way to convert the high-complexity models that H2O produced into results that the business could gain insights from (after all, without insights, I wasn’t going to help the business very much).
LIME turned out to be the perfect tool. LIME explained the H2O models so I could tell business people what levers they have available to adjust.
Speed + Power + Explainability. Win-Win-Win.
LIME - Explain Business Insights
The combo of H2O + LIME ended up being so good that I decided to commercialize it through offering a consulting service beyond just being a handy coder.
This leads me to my first rule.
Rule #1: Learn Tools That Deliver High Impact
The tools that have delivered the most impact for me in my career have hands-down been H2O and LIME. These are high-impact tools that deliver speed + power + explainability.
I began publishing what I was learning on the Business Science Blog (now Learning Hub) as both a way to help others and also to get the word out that I could solve complex business problems with data science.
I wrote articles on Human Resource Analytics / Employee Attrition, Customer Churn, Sales Back Order Analysis and more - Each of these diverse problems could all be solved by combining H2O and LIME.
Next Is My Struggle
I eventually was noticed by a newly-appointed Director of HR working at a Fortune 500 Firm. She contacted me because of my Business Science articles (now Learning Hub) - this was very exciting.
As I took on my first major project with the Fortune 500 firm, I noticed that this engagement was different.
I was given a very vague request (I later found out this is incredibly common):
“We have some HR data, we have a meeting with our Board in 1 week, and we want to show that we are implementing predictive analytics.”
~Fortune 500 Firm
What did I do?
I took the engagement confident I could deliver amazing results with my bag of tricks.
I received the data on Friday, and was requested to have results Monday morning.
I had hired an expert in the HR field to help me out, and we worked through the weekend to deliver a report with H2O + LIME.
The Delivery
We delivered the executive report on Monday.
The report was awesome. It identified people that were not being currently identified as “executive leadership potential”. It showed why they should be flagged for leadership potential.
We had our meeting on Monday. We delivered the report to the HR Exec’s, gave them the results, and then there was…
silence.
For anyone delivering a data science product, silence is NEVER a good thing.
Epic Fail
FAILURE. It's one of the most difficult things to deal with.
It turns out they didn’t like the report, and I never found out why.
I stopped taking on engagements.
For days I just kept asking myself what I had done wrong.
I hired an expert, I successfully applied predictive analytics to their data, my consultant and I turned around a great report in 3 days.
What could it be?!?!
After about 10-days, it hit me like a ton of bricks:
-
I was operating without a process
-
I had an unrealistic time schedule
-
(Most Importantly) I had no clue what the client actually wanted (and I still don’t).
This leads me to Rule #2.
Rule #2: Learn From Failure
We all get knocked down. It's how we get back up. If you deconstruct your failures, you will learn, and you will be able to prevent future failures.
With an understanding of my failure in hand, I had the ingredients to build a system that would change my consulting performance overnight.
This deconstruction of the failure spawned a major epiphany including the creation of the most important Data Science Tool in my toolbox - The Business Science Problem Framework.
The Most Important Tool in My Arsenal - The Business Science Problem Framework (Free Download)
Do you want to hear something crazy?
I implemented the BSPF system as part of my consulting engagements, and overnight my client satisfaction skyrocketed!
Why Was the BSPF So Effective?
There are really 3 reasons.
First, the BSPF system laid out an entire plan for a data science project that clients understood.
Client’s appreciate understanding the entire scope. The BSPF was a roadmap showing the direction we were headed.
Second, the BSPF showed every step in the process.
Client’s saw the steps required for project success. Getting to project success was a team effort with stages and checkpoints to prevent project failure. They understood why it takes weeks to complete a data science project.
Third, the BSPF opened up the communication channel
Client’s saw exactly where their input was needed, and why their input was so valuable to delivering a successful data science product.
The bottom line was that the BSPF was delivering unparalleled value - client’s were more satisfied than ever.
This leads me to Rule #3
Rule #3: Have a Plan that Incorporates Your Stakeholder's Problem Understanding
My struggle was because I didn't have a plan that took into account the stakeholder's input. Without this, my project was doomed from the beginning. Applying the BSPF changed my client's satisfaction overnight.
Fast forward to today…
Nowadays I Teach Data Scientists
I see a lot of Data Scientists in my shoes.
Data Scientists are finding it difficult to deliver business value.
This leads to a lot of…
Frustration
The most difficult part of data science is having an immense set of tools, but not knowing how to connect data science with the business.
The most common sources of frustration I see are…
Which problems should I focus my efforts on?
The business is faced with an endless stream of challenges. Which ones should you take on and with what priority?
Which algorithms should I use to solve a problem?
Some business challenges are not good candidates. Should you use data science to solve a problem? Where does machine learning fit? Which algorithm should you use?
How do I explain data science to business stakeholders?
The business stakeholders are not typically proficient in the data science tools (that’s your job). What are the stakeholders interested in? How can you convince the business to take on your project, to accept your solution, and to take your solution into production?
What Helped Me Learn Data Science for Business?
One of the reasons I enjoy consulting is that it allows me to work with the people that are on the front lines solving real problems that matter to the business. And I got to work with a lot of them.
I was able to experience their struggles and successes.
This consulting experience has been very beneficial in understanding business problems and how to generate value within a diverse set of organizations.
Now that I teach, I get to share my experience with you! :)
I Began Offering My Knowledge Through Courses
I was drawn to education because I was helping organizations on a small level, but I wanted to help data scientists on a larger level.
A natural extension was to create a course that addressed the needs of Data Scientists seeking to apply data science within organizations to drive business value (through Return-On-Investment (ROI)).
I focused on the Business Consulting piece first because this is where I saw the biggest need within organizations.
Data Scientists were struggling. But with a little help, they could become amazing! They just needed to know what I knew - How to connect data science with business.
I created a course that detailed the Business Consulting + Machine Learning process from beginning to end
Most Data Scientists needed to learn how to apply cutting-edge tools (i.e. H2O and LIME) to real Business Problems (i.e. Churn). So this is what I created.
The product of a year of hard work was my Data Science for Business with R (DS4B 201-R) Course.
Machine Learning + Business Consulting Course
DS4B 201-R follows the exact same consulting framework that worked so well for me - The Business Science Problem Framework - And the student applies the framework and machine learning to a real business problem.
This is an effective approach for a few reasons:
-
The student gains experience applying a repeatable methodology for generating business value through data science.
-
The student learns high-impact tools - H2O and LIME
-
The course incorporates industry best practices - My learnings from failure, and years of experience consulting with many organizations.
But, it’s when I launched DS4B 201-R that I began to understand the true impact.
Real Results Through Applied Business Problem-Solving
The true measure of any course or solution is…
Does It Deliver Results Fast?
If an educational program can't be applied within 1-hour of beginning, it's a waste of time.
The first major breakout result was when one of my students contacted me to explain his situation and what the course did for him.
When I spoke with Rodrigo, he had already applied the BSPF framework to 3 projects. And he was using the tools, H2O and LIME, to solve client problems.
The course doubled his productivity - He was able to complete data science projects in half the time!
You can check out the full interview here.
As Word-Of-Mouth Spread, The Results Continued
As word of mouth spread, we began getting more data scientists into the DS4B 201-R program, and more importantly we began gaining more feedback from students.
This feedback was critical.
-
Feedback led to improvements like the Community Slack Channel, where students are able to connect to each other.
-
Feedback led to numerous other iterative improvements that drove the course success - We have a survey at the end of Week 3 that has been completed by 90+ students.
Each student has an opportunity to provide feedback. I personally read every response.
Our overall rating for DS4B 201-R is a 9 of 10, which is amazing!
9 of 10 Rating
Feedback is critical to improvement. Progress happened iteratively. We improved overall feedack over time to a 9 of 10 rating (90+ ratings), which is very good for an advanced data science course.
But, the details of the responses were more amazing…
Data Scientist are confidently applying data science to business
More recently we began seeing results like Ben Moretti’s.
Ben was apprehensive about applying data science to business.
He took our DS4B 201-R course, and learned the BSPF Framework and H2O. Ben’s confidence increased. He now was able to apply a repeatable system and tools to generate business value.
Conclusion
I’ve learned a lot on my data science journey. I’ve made mistakes, I’ve improved through trial and error, and I’ve become a much better data scientist because of it.
If I boil this learning down, I can sum it up into 3 Simple Rules:
-
Learn Tools That Deliver High Impact - For me and many other data scientists, these are H2O and LIME. It’s a powerful one-two punch for high performance and explainability, both of which help generate business value.
-
Learn From Failure - When your projects fail, learn from it. Deconstruct the problem. Find out why. Figure out what could have been done differently. Implement it!
-
Have a Plan that Incorporates Your Stakeholder’s Problem Understanding - A tool like the Business Science Problem Framework is priceless because it gets everyone on the same page while showing them a roadmap for the entire problem-solving process.
Finally, just because you are learning does not mean that you have to go at it alone. You can accelerate and improve by learning from others that have been there. That’s where I fit in.
I will help.
The Data Science For Business with R (DS4B 201-R course) is the solution I created for you. You learn:
-
A Repeatable Methodology: The BSPF
-
High-Impact Tools: H2O, LIME, and more
-
Industry Best Practices: Stakeholder communication and project involvement, ROI analysis, and more.
DS4B 201-R is an amazing program that will help you benefit from my experience while you experience first-hand how to apply data science to business. Many others have committed to learning with DS4B 201-R, and it has helped them double, tripple, or even quadruple thier productivity and confidence. It will help you too.
If you have questions, feel free to reach out. I’m invested in your success.
Share this article if you like it so others can benefit.