Data Engineering is like the Offensive Line in Football

In Football, QBs & RBs get all the glory, but they are nothing without their Offensive Line. In Advanced Analytics, Data Scientists capture all the headlines, but the Data Engineering team is quietly of one of the most important pieces of an Enterprise.

You want to be Data Driven? …you need Data.

Quality Data Data that can be trusted and governed

And available when needed

A Data Scientist can build the most valuable model, but it is always dependent on the data.

As teams evolve to adopt many multiple AI/ML models that depend on the same underlying data, the data pipeline(s) becomes the critical path of the business.

A strong Data Engineering team can make a huge impact on a Data/Decision Science team. Conversely, they can also create a huge drag.

The longest cycles in a ML project is the data wrangling and the production-ization of a model. Data Engineers have the opportunity to drastically reduce these areas and free the Data Scientists to focus on what they do best.

The best QBs take care of their O-Line and are known for giving extravagant gifts (cars, watches, etc.). Data Scientists aren’t star QBs making tens-of-millions, but a simple “Thank You” and acknowledgement of appreciation this holiday season can go a long way!

#TeamWork

AI Wins Again!

AI beats humans again!

https://mashable.com/2018/02/26/ai-beats-humans-at-contracts/#Sg3c4D_8TkqS

These victories are to be celebrated as great feats of technology and advancement, until one day… :/

In this competition, lawyers we’re given 5 NDAs to review and identify 30 legal issues.

Humans averaged 85% accuracy rate; AI achieved 95 percent accuracy.

AI also achieved 100% accuracy in one contract, whereas which highest-scoring human lawyer score was 97%.

So, the tech works, but what is the business case?

Human lawyers took an avg of 92 minutes; AI completed the task in 26 seconds!!!

That is at least several hundred dollars of savings.

I’m not a lawyer, but if I was, this isn’t something that I’d be concerned with at all. I’d welcome this with open arms and run to this now, as a potential competitive advantage to provide a better service at a lower cost to my clients. In theory, I’ll be able to serve more clients as well and/or be able to devote more attention to higher valued services.

As a consumer, I’d look for lawyers that have adopted this kind of technology because I will feel more confident in the quality of the service. And presumably, the service might be a little cheaper (relative to others’ that don’t leverage technology).

How Much Time do you REALLY Have?

Would you change anything if you, literally, knew when your time is up? Seems we each have an explicit biological clock embedded in our DNA.

This article talks about how researchers found that your Epigenetic clock can calculate biological age and predict your lifespan.

“Some individuals who fill their lives with fitness and healthy habits die younger than peers who live a much less healthy life. New research into the epigenetics of aging sheds some fresh light on the perplexing phenomenon of premature aging.”

It’s based on this research paper.

I thought this was super interesting.

Loads of implications:

  • Would you retire earlier or later based on your biological age?
  • Surely, this is going to have a tremendous effect on the Insurance industry.
    • I wonder how long until the industry adopts this as common practice to set your premiums.
  • Now that we know the marker (or measure), can we work to improve or manipulate it. (this is all over my head, I don’t even know if that’s a valid question)

Why you should Build your CDP

Customer Data Platform (CDP). The more I read and learn about CDPs, the more I am convinced that most, large companies should have one. The CDP hype is real. BUT (I like big BUTs and I cannot lie), I’m a huge advocate for BUILD (Vs. Buy), in this case. I would not go with a CDP SaaS vendor.

Of course, there is always exception to the rule and, as with anything in technology fitting, it depends. But I would strongly consider to build in-house, by default.

This is a shift from my norm. I’ve always been on the vendor-side of things, in favor of the business case to BUY solutions. But, I see this as so strategic and core to a company, that it’s worth the investment to build it. There are tools on the market today to make this process feasible and achievable (more on that later).

When Caesars Entertainment declared bankruptcy a few years back, “The most valuable of the individual assets being fought over by creditors is the data collected over the last 17 years through the company’s Total Rewards loyalty program”. I’m going to argue that that is their CDP. This is a good write-up about it: https://www.forbes.com/sites/bernardmarr/2015/05/18/when-big-data-becomes-your-most-valuable-asset/#117b5a741eef

When you build a CDP, you are building an incredibly valuable asset.

Value is built through asset ownership, not renting.

Taken from this article (https://www.martechadvisor.com/articles/data-management/what-is-a-customer-data-platform-and-what-are-the-benefits/), the CDP has three primary functions:

  1. They pull in customer data from the disparate data systems of your choice
  2. They match, merge and cleanse this data into a unified record for each customer
  3. They make these records visible to your other marketing tools to ensure the consistent treatment of customers

I’m going to add #4, from this Gartner blog (https://blogs.gartner.com/martin-kihn/what-is-this-thing-we-call-a-cdp/):

4. It is owned and operated by marketers

I like the way the author opened the article:

A new technology appears, seemingly from the ether, and promises to change our lives. Customer data integration, labeling and storage problems will disappear. Identities will merge. You’ll be able to find new audiences until your ribs squeak and deliver them to any execution system in the barn.

Oh, and it will scale, rarely fail and enable (yes) true one-to-one marketing.

The name of this magic machine is … CRM. It was 1998. Companies piled in, dropping $3.5 billion a year on apps and databases alone – consulting fees not included – and yet, by 2001, 50% of CRM projects “failed.”

The same thing happened, on a lesser scale, during the great marketing automation boom of the 2000s. And it’s happening again.

So, the CDP is supposed to accomplish what the CRM cannot do and what the DMP does not do. But we cannot forget Primary Function #4: It is owned and operated by Marketers.

If you thought the relationship between Sales and Marketing was challenging, it pales in comparison between the diatribe between Marketing and IT; hence, the rise of Marketing Technology teams? I wrote a post about this recently and highlighted this article by on chiefmartec by Scott Brinker, written in 2009 (the classics never die) .

Let’s be honest, these are massive projects; however, the potential value is huge. It’s no surprise that there are over 60 vendors in just a few short years, with a few of them pivoting to adopt this acronym as their primary identity. This is a project you only want to do once.

Do you want to be vendor-locked?

Many CMOs may not even live to reap the full benefits of their investment, given that the median tenure-ship of a CMO is 31 months (according to this source )

You have a CRM.

You have a DMP.

You have a Marketing Automation platform.

You have an EDW and possibly a Data Lake.

You have an enterprise BI solution.

Do you really need a CDP vendor?

(And I’ll double down on that question, if you’ve already implemented a SaaS Customer Journey solution, which most CDPs can/should be able to provide)

The one case where I’d fold quickly on the CDP vendor case is Datorama, if and only if, you’re already full stack Salesforce —negotiate hard at renewal time! 🙂

Here’s the kicker, the most expensive (and important) component of the CDP is your 1st-party behavioral data. The storage, but mainly the processing, of your digital analytics clickstream to match, merge, and cleanse this data into a unified record for each customer. That’s a lot of data! And you’re likely already storing it somewhere.

And if you’re a brand conglomerate, do you have a CDP per brand or do you aggregate or can you have both? I think you should have both, but this would be cost prohibitive (or hugely wasteful) with a SaaS vendor.

I’m going to circle back to Primary Function #4 because this a big reason folks choose a CDP vendor. …“My IT is backlogged; they can’t deliver in my required timeline; I want control”. All very likely true.

If I put my CMO hat on, I would find a consultant to architect the system. If done properly, this system would be a silo-buster and could help democratize A LOT of data throughout the org (see post on Data Silos here). Data/BI Analysts will go to town. Data Science can reap huge time savings. Many internal projects can spawn from a CDP; E.g., Marketing attribution. Endless possibilities, really.

Now, that then brings up the question of “Who’s budget?”. Ah, the joy of politics.

I will argue that the CMO should take this initiative on. They are the first and primary use-case. If I’m correct, that this will benefit multiple orgs, then that’s a big win for the CMO. That’s a CMO with enterprise-wide vision. That’s a future CEO.

In terms of time, there are many tools on the market that can be used to dramatically accelerate this project. In fact, I’ll put it out there, that with the right tools, this can be accomplished within several months.

If IT can’t handle it, I would consider starting with a consulting firm that can operate as a managed service, knowing that I can bring the resources in-house later, if that makes sense. This would be hosted in a virtual private cloud (and hopefully a solution would be flexible enough to not lock me into any one cloud vendor, either).

To conclude, if you don’t have a CDP project going, you’d be remiss if you didn’t get one going as soon as possible. But look inward first.

Why IT and Marketing are Diametrically Opposed [commentary]

[LinkedIn Post]

When Business and IT clash, it’s the customer who loses. Everyday you stagnate, you put your customers at risk. When you keep your data siloed, it’s your customer’s experience that suffers.

Amazon has over 500K employees and they are foster a culture with  “Day 1” mentality; a mantra meant to convey that the company will never stop being a start-up.

Has any company in history disrupted as many individual companies and distinct industries as Amazon?

I’ve been on the vendor side for over 12 years now, selling technology into various businesses. Business wants to be nimble and move fast. IT is often opposed and it’s not their fault, necessarily.

It’s a classic scenario.

This old article by Scott Brinker absolutely nails it on the head.

I believe these opposing goals has helped fuel the rise of SaaS, as businesses could by-pass IT. But with Data being such a huge asset (and security risk), the pendulum is swinging back a bit. Business must work with IT.

“We’re a big company, we move slowly”

Okay, but who suffers? YOUR CUSTOMER.

 

Last I checked Amazon is a big company too.

They put their customer first.

It’s always Day 1 at Amazon.

[Article Thoughts] Want to Really Understand Your Customer Data? Try Hiring a Scientist

[Posted on LinkedIn]

Here is the article: http://www.adweek.com/digital/blinded-by-data-science/

I bet Nike accelerated ROI from their customer data by years by buying Zodiac.

Here’s the typical path…

Execs read that Data Science is hot, so they start to build out a small team. Some value is seen, but putting models into production, specifically ones with high data volume, velocity, and variety (e.g., Customer-related data) poses a significant engineering challenge. It’s hard.

It’s why Uber built Michelangelo, their ML platform. Putting this apps into production is hard. It seems to have taken Uber ~1.5 yrs to build Michaelangelo and I’m sure it’s constantly evolving, as it has a dedicated product team assigned to it.

By acquiring Zodiac, Nike inherits a Data Science platform or toolset centered around Customer data. Plus they get some (presumably good) data science talent, which is seemingly scarce. They’ve accelerated initiatives by a year+.

“Before analysts can crunch the data, apply machine learning and deliver new insights, brands need engineers who know how to store, manage and clean the data. And they need tech-savvy communicators who can take that analysis and translate it into business reality, Purcell says.”

SYNTASA can help you with this in less than one month!

#WorkSmarter #Clickstream

The Internet is the Battleground for Perpetual Warfare

Norse has a really, really cool real-time visualization of cyber attacks. It’s quite mesmerizing to watch.

Most of the attacks stem from US and China, at least in the few minutes that I watched. What’s interesting is that you see big name corporations as the Attackers. I happened to notice A LOT of Microsoft Corporation. I have no idea what that means; I’m very from being any kind of security expert. I just noticed this and thought, “hmmm”.

I’m sure this is just a fraction of what goes on out there. Which makes this pretty scary. I’ve been in software a long time and I’ve never come across or heard of code that didn’t have bugs or couldn’t be hacked.

Add to that the fact that we have assigned some of the best and brightest to find all these holes (re: Wikileaks CIA hacking tools release and NSA leaked hacking tools).

And then of course I’m always reading about how fragile, old and outdated our infrastructure is. We’re so exposed. Is a nuclear threat really the biggest threat out there?

Hold Hold Hold … It’s So Hard Not to Sell

BitCoin and Ethereum (as well as other cryptocurrencies) are on a parabolic trajectory. It’s so hard not to sell here. I mean, when you’ve 3x and 6x your investment and you see this trajectory, isn’t it prudent to take some winnings off the table?

I would say normally, taking some off the table (at least lock in the initial investment so that what’s left is “house” money). But here is my investor greed setting in, perhaps.

I truly believe that this is really only the very very beginning and HUGE gains are ahead because the blockchain technology is that revolutionary. I don’t want to get into the mindset of trading in and out because the Fear Of Missing Out (FOMO) if I sell will kill me more.

I don’t have a huge portion of my portfolio allocated into these, so I can afford to lose it all and still sleep at night. These really are like lottery tickets, but I like the odds here.

I’m considering diversifying and buying into some Ripple. I’ve been reading that Tezos is also an ICO to watch for. These aren’t as simple to buy, though. I mean, they aren’t hard, but I’d have to set up other wallets.

Interesting times…

3 Little-Known Companies Helping Shape the Movement to Autonomous Vehicles

Quanergy, Velodyne, and Luminar Technologies. They are all LiDAR technology companies.

LiDAR is how the vehicles detect what is around them. LiDAR sensors measure distances by measuring the Time of Flight (TOF) that it takes a short laser pulse to travel from the sensor to an object and back, calculating the distance from the known speed of light.

Each of those 3 companies has raised many millions in early rounds (none beyond Series B). According to James Altucher sources, they each carry a $1B+ valuation which puts them in the Unicorn classification. (I can’t link to the source because it is an insider newsletter)

James believes that Velodyne is the clear leader at this point. Velodyne raised $150M in August 2016 from investors that included Ford, Google, and Baidu. He mentions that Velodyne’s products are currently being used in 25 different self-driving programs.

Of course, as private companies, we can’t invest in these companies. But James is recommending to look at Ford as a way into Velodyne.

Of all the car companies, I do like Ford the best. They have made interesting investments and partnerships. My wife has a Ford Focus and it’s a pretty decent car.

I can’t bring myself to invest in car companies at this time, though. I’ve been reading too much about sub-prime car loans going bust, similar to how sub-prime housing went bust in 2007. Obviously, the car market isn’t nearly as big, but it’s still significant.

I’m currently short GM and that is going pretty well so far…

 

We Can No Longer Trust Any Recorded Audio or Video

We’re in for some scary times. Who knows what or who you can trust.

Check these demos from Lyrebird. The first on there is a conversation between Donald Trump, Barack Obama, and Hillary Clinton.

Lyrebird claims that they can take a 1-minute recording of someone’s voice and compress the voice DNA into a unique key. Then they can generate anything with its corresponding voice.

So, basically take any clip off YouTube for any celebrity and BOOM, you can create an instant celebrity endorsement for whatever.

The voices aren’t​ perfect…yet. I mean, the intonation of the Trump/Obama/Clinton voices sound robotic; but I can definitely say that it sounds like them. Obviously, the technology will only get better.

Now, couple that with this video technology and we’re in for some serious trouble as a society.

In the future, I believe everything will have to have an authentication key of some sort with every piece of media. In other words, people will come to not trust anything without this key. Without such a mechanism, the world will drown in “Fake Media” and these technologies will be used for evil.

This may be yet another application for the blockchain…