The last 7–10 years have in many ways come to be known as the era of big-data. Companies in nearly every industry can now reach the huge potential upsides of good calculated decisions, and industries like marketing have shifted from an art to a data-driven science. While data-scientists rely on structured scripting languages like Hive, Pig, and Python, the data-mentality needs to penetrate all the different decision-making parts of a business in 2018–19.
The Data Challenge for Businesses
This has created a particular headache for IT teams and the corporate leadership, faced with the challenge of effectively collecting, integrating, and distributing information in a way that can be used across the board. Brick-and-mortar businesses and companies with a strong in-person element face this challenge in particular, as they need to find an effective way to collect, store and use data from many sources and environments.
Imagine your local pizza joint trying to extract the nuances of pizza toppings – which bring in the most revenues and what you, the customer, thinks while ordering or eating it. To make matters more complicated, you can’t always ask the customer outright what they think, as “the mind knows not what the tongue wants!”
The Journey of Pizza Preferences
IT and data teams need to gather any helpful or identifiable information available, and they need to translate your thoughts into a standardized language that can be understood by others. The information is sent somewhere (usually in the cloud) to be stored together with the information from thousands of other pizza-lovers. Your information needs to then somehow be interpretable to the management of the pizza chain so that it can actually be effective in helping make your next slice even better than the last. Without this process, the pizza place down the block will clearly put them out of business. Talk about a long and expensive journey for pizza topping information.
In 2018, data is key, and it can be exceptionally hard to access. Not only that, but real-time data is more important than ever. The whole pizza process above needs to be instant. A stale mushroom is all it takes for a customer to instantly write a negative review that can affect an entire chain.
Companies traditionally faced these challenges by either building their data-collecting machines in-house or by using data-integration tools. Oracle has traditionally played at the fore for enterprises with their cloud and particularly the Data Integrator. Microsoft has invested millions pushing out its Data Integration Platform, and Microsoft’s SSIS (SQL Server Integration Services) have carved out their respective pieces of the market. Between those products and SAS, IBM and Informatica, enterprises have plenty of options for integrating data.
While it goes without saying, the shift to the cloud has become completely mainstream for storing all this data, with the main exception being firms who either prefer to keep very confidential information on their servers or those who invested in large on-base storage systems and have yet to shift.
As this shift to the cloud takes place along with companies trying to become more data-centric, startups are leading the way in generating, migrating and interpreting the data.
How Startups Are Leading the Data Journey
In terms of generating or identifying information, you can find AI and machine learning startups focused on nearly every industry, from healthcare to food to travel and more. Running with our example from the food industry, companies like Trigo are tackling the challenges of collecting data inside brick-and-mortar. In their case, they use cameras to identify and understand everything you are doing or which product you are taking in a bodega or supermarket. This can be used to understand real-time information, to automate a store like Amazon Go or to track a store or warehouse from theft or problematic activity.
Once information is collected, the actual migration to the cloud can be viewed in 2 components: the migration of data and the migration of applications. The applications in essence tell the data how to ‘behave.’ While once upon a time this all sat on a machine, or computer, in house, it has undergone a revolution.
Startups like Zerto and CloudEndure have been carving out their place in migrating applications. CloudEndure for example, has a tool for cloud migration combined with disaster recovery, which prevents data loss and downtime due to human errors, network failures, external threats, or any other disruptions.
Migrating the data itself however, in addition to the ‘machine,’ has proven to be the major pain-point for companies – especially those like our pizza and grocery stores trying to collect and make sense of different types of information while sending it to storage. Alooma, a Redwood City based startup with an office in Tel Aviv, initially began to focus on this challenge after conducting in-depth research with 150+ companies and VCs in 2015, according to their co-founder and CTO Yair Weinberger. Their solution has been focused on taking all of a company’s data in from various sources – whether a pizza oven interface, a SaaS application or on an excel spreadsheet – and bringing that decentralized information into one place like BigQuery, Redshift or Snowflake.
According to Yair, big companies already offered the integration, but startups have helped make sense of the data, recover for the unexpected and move it around without requiring a huge in-house team. Once the data is in the cloud, IT teams have more options to control the activities surrounding it.
As an investment platform into venture with hundreds of startups in our rolodex, we are seeing data become the most valuable asset held by enterprises. Companies need to be more cautious than ever in handling this information, but also faster and more efficient. Industries that were traditionally slower to adapt to technology, like your local pizza store, are now searching for ways to innovate and gain an upper-hand. According to Orly Glick, VP of Value Added Services, we’ve literally arranged over 950 meetings between startups and corporations and we expect the interest to grow as these startups managing your data become more sophisticated, larger, and waiting longer for an exit.