Gartner D&A Summit Bake-Offs Explored Flooding Impact And Reasons for Optimism!

Extreme weather events and their devastating impacts are more frequent and severe, but which populations are most affected? Are there mitigation strategies that show reasons for optimism? We explored these questions and more at our Bake-Offs and Show Floor Showdowns at our Data and Analytics Summit in Orlando with 4,000 of our closest D&A friends and family. We did two Bake-Offs this year. The first featured analytics and BI platform Gartner Magic Quadrant leaders while the other showcased high interest data science and machine learning platforms. The Bake-Offs are fast-paced, fun and informative sessions that let you see leading vendors side-by-side using scripted demos and a common data set in a controlled setting.   And this year's Bake-Offs did not disappoint! To  extend the Bake-Off experience to a broader range of vendors, including smaller innovators, we also did a series of Show Floor Showdowns for analytics and BI, data science and machine learning and for data management. The Show Floor Showdowns were held on the Exhibit Hall show floor and were based on the same demo script and data set used in the Bake-Offs. Any vendor that Gartner has covered in any of its research and all vendors that had a booth in the Data and Analytics Summit, Orlando Exhibition Hall could apply to participate in Show Floor Showdowns. From there, participants were randomly selected and invited to present live at the Show Floor Showdown sessions. We also gave the demo script and data set to all vendors in the Exhibit Hall to create demos for their booths and to submit for this blog. For the vendors that participate in the Bake-Off and Show Floor Showdowns, it is in equal measure fun and extremely stressful. It’s a high stakes session where they put everything on the line in front of their arch competitors, revealing what’s coming and coveted sales tactics all while being judged by attendees. It takes a fair bit of preparation, extreme commitment, and an ability to shine under pressure. What Data Did We Use? We use the Bake-Offs and the Show Floor Showdowns as a platform for data for good. This year's scenario was  to understand the likelihood of more frequent and anomalous flooding and the resulting devastating human and economic impact because of unpredictable weather conditions. Are there mitigation strategies that can be implemented successfully that could provide policy guidance and reasons for optimism in the face of ever increasing frequency of extreme weather events? We gave the vendors data from Organization for Economic Co-operation and Development (OECD) data and National Oceanic and Atmospheric Administration (NOAA) from the National Centers for Environmental Information. They could supplement this data with any other relevant data sets. Last year’s topic was the UN’s Sustainable Development Goals.  In 2021, we explored vaccine effectiveness and inequity using global vaccine data. In 2020, we analyzed population health data. In past Bake-Offs, we explored loneliness and happiness data, opioid epidemic data, traffic fatalities, college costs, and homelessness. The results showing the  impact of flooding, predictions for the future and reasons for optimism were fascinating.  This blog highlights some notable findings and the videos from participating vendors. BI Bake-Off Vendors’ Key Findings and Videos Microsoft Here is the link to Microsoft's BI Bake-Off video. Qlik Key Findings: In the US alone, there’s $367 billion in agricultural commodities at risk to flooding in the US alone. There have been 338 weather events that have created over a 1 billion dollars (USD) plus of damage, totaling to $24 trillion in damages altogether and averaging $71 billion per event. For most coastal countries, climate change and flooding are critical topics of policy and development strategies. A large part of under-developed Asian countries ranging from Bangladesh to Vietnam are at high risk of flooding events. Here is the link to Qlik’s BI Bake-Off video. Salesforce (Tableau) Key Findings: Unlike other European countries, the Netherlands has the most square meters of land at risk of flooding, as well as the greatest population at risk of flooding (70%). While there are many factors that can contribute to flood risk such as changes in land-use from urbanization and deforestation, there are also natural geographic features that can put certain populations at increased flood risk. These factors include % of population at risk, average elevation, and % of inland waterways throughout a region. A big factor as to why Netherlands faces extreme flooding risk is that 15% of the region is comprised of inland waterways. As population density increases in major metropolitan areas pre-disposed to flooding, risk to life and livelihood is expected to continue to increase. In 2000, the Netherlands had 8.5 million people at risk of catastrophic, flooding. In 2040 that number is projected to grow to 10.2 million. Here is the link  to Tableau’s BI Bake-Off video. Data Science and Machine Learning Bake-Off Vendors’ Key Findings and Videos Alteryx Key Findings: The task was to analyze the likelihood of more frequent and anomalous flooding and provide mitigation strategies The objective was to identify high-risk areas the US Department of Interior should focus their resources for river flooding over the next 10 years. Climate change may lead to more frequent and severe flooding. Through the analysis we found regions with high risk of flooding, including Big Horn County in Montana, which has a high Native American population at higher risk of safety and economic impact. Recommended mitigation strategies include improved infrastructure, early warning systems, land-use planning, and community engagement focused on the middle Rockies surrounding areas. Here is the link to Alteryx's Data Science and Machine Learning Bake-Off video. Dataiku Dataiku chose to focus on flood risk by geography and flooding’s economic impact on commercial insurance policies and claims. With a business use case of defining a location and insurance strategy for a proposed warehouse in southeast Florida, here are some of the key findings: Extreme rainfall and rising sea levels are driving up both flood frequency and insurance costs, especially in high-risk Florida areas. Southern Florida's Atlantic coast has increased flood risk, leading to more and costlier flood insurance claims in areas like Homestead and Miami compared to West Palm Beach and Port Saint Lucie. Our analysis suggests prioritizing Hobe Sound or Stuart, FL for real estate investment due to no past floods or claims within a 60-minute driving radius, reducing risks to warehouse, contents, and distribution routes. Interpretation of our machine learning model suggests buying $50k in building coverage and $100k in warehouse content coverage, as the latter significantly boosts predicted flood claim payouts. Here is the link to Dataiku's Data Science and Machine Learning Bake-Off video. SAS SAS took the view that mitigating and anticipating flooding events should be focused on the goal of helping people. SAS created, on top of the traditional statistical and machine learning models to predict events, a set of four unique models specifically focused on helping people impacted by flooding: An optimization network model (cost network flow algorithm) to optimally help displaced people reach public shelters and safer areas. An optimization network model (vehicle routing algorithm) to optimize routes and the time it takes to transport displaced people to public shelters and safer areas. An autocorrelation forecasting model to identify parameter estimators, associated with relevant variables, that impact the likelihood of flooding events. Based on these estimators, SAS created an easy to use what-if dashboard. The user can change the variable values to understand how these changes would impact the likelihood of future flooding events. A time series regression on referenced spatial data to estimate future flooding locations within hurricanes/cyclones paths. The hope is governments could prepare for a flooding event and consider mitigating actions to help people in future danger. Here is the link to SAS's Data Science and Machine Learning Bake-Off video. Analytics and BI Show Floor Showdown Vendors’ Key Findings and Videos Course5 Intelligence Key insights from Course5 Intelligence include: The US states with the highest susceptibility to coastal flooding for a 10-year return period are Alaska, Delaware, Maryland, and New Jersey. Louisiana, Arkansas, Mississippi, and West Virginia are the US states with the greatest vulnerability to river flooding for a 10-year return period. During river flooding, Louisiana experiences more severe impacts on built-in lands and crops, with exposure percentages of 50%-60% for built-in lands and 40%-45% for crops, based on data from the OECD on river flooding. Between the years 2000 and 2020, river flooding in Louisiana caused crop damages worth $270 million and property damages worth $9.1 billion. Heavy rainfall along with storms is one of the primary causes of flooding in Louisiana, as evidenced by the graph of flood frequencies vs annual precipitation. • Some of the other general reasons for flooding in Louisiana are hurricanes, poor drainage, levee breaches, and deforestation. Also based on the historical data, there is a very high chance of flooding in Louisiana in the year 2024 with a likelihood of 3:1. According to ChatGPT’s suggestion, some of the flood mitigation strategies that could be deployed by the government in Louisiana are: improve the drainage system, improve building standards, improve land use practices, and improve emergency response. Here is the link to Course5’s Show Floor Showdown video. Datamatics Key Findings: In China, Impact of coastal flooding on built up area exposure has increased from 4.45% in year 2000 to 6.64% in year 2020. In Washington. Impact of coastal flooding on built up area exposure has increased from 0 - 1% in year 2000 to 5 - 7 % in year 2020. The key driver analysis for Coastal flooding in United States has shown significant differences between flooding at a state level and at a country level. Further our analysis shows the significance of exposure to coastal flooding in terms of built-up area, cropland and land exposure. New York - Built-up Area Exposure is significant California - Land Exposure is significant Texas - Land Exposure is significant. The Entire United States - Built-up Area Exposure is significant. Here is the link to Datamatics’ Show Floor Showdown video. Domo Key Findings: Year-over-Year (YoY) analysis has shown that, based on data from the first quarter of 2023, the Northern Gulf Coast of Florida (Leon County) has a hotter average temperature than the previous 4 years. Weather station analysis shows that Leon County, Florida has received the most precipitation by 50% over other key flood zones in Florida, although Miami-Dade county has had the most flood events year to date. Insights driven by regression analysis demonstrate that major storm events (100-year, 50-year, and 25-year) impact zones for built up areas and will continue to increase over the next 5 years as 10-year impact zones stabilize, thus identifying that major storm event activity will continue to grow in size and impact on the surrounding areas of typical flood events. Although Leon and Miami-Dade counties are seeing the largest impact (respectively) due to increased precipitation and flood  events, Lee County has the highest average damage (in dollars) per event due to hurricane activity. Here is the link to Domo’s Show Floor Showdown video. Incorta Here is the link to Incorta’s Show Floor Showdown video. LTIMindtree Lumin picked up the South Asia Region for the river flooding analysis. Some of their key insights include: India has seen the highest population (more than 40 million) impacted by river flooding in the last five years. This river flooding has been caused by severe land precipitation but has been exacerbated by growth in dry grassland cover and artificial surface cover at the cost of vegetative cover. A quick simulation revealed that continued urbanization in India would lead to another 6500 square kilometers of land area being exposed to floods in a five year horizon. A state-wise deep dive uncovered the densely populated West Bengal and Bihar as the most affected regions, with 17 million people exposed to floods in Bihar. There is however, a cause for optimism as the trend of flood mortality or the casualties from flooding events, has decreased steadily. Densely and moderately populated regions in India have seen a significant drop in flood mortality . This decrease in casualties is driven by drop in the number of flooding events and the number of days of flooding but the number of people displaced due to flooding in India continues to remain fairly high each year, causing massive social and economic upheaval. Here is the link to LTI Mindtree’s Show Floor Showdown video. SAP Analytics Cloud Key Findings: River Flood has more impact on Population and Economics. The only exception is in the Netherlands, where Coastal Flood is a significant problem. Population at Risk, Cropland at Risk, and Total Land at Risk all have a linear relationship with Population, Cropland and Land in a country. Due to this, the largest and most populated countries suffer more, typically found in Asia. The most suffering countries are at the same time largest Agriculture producers with a high share of Agriculture in their GDS. Therefore, flooding is not only dangerous for people and infrastructure, but also has a large economic effect, especially affecting food supply through Agriculture. Countries with a high GDP per capita are in the safest positions regarding flood, while low GDP per capita countries are strongly affected by floods. Here is the link to SAP Analytics Cloud’s Show Floor Showdown video. Tellius - Was not selected for the Show Floor Showdown this year, but provided a video by the deadline as per the participation rules. Key Findings: After analyzing global disasters across the past 30 years to understand the impact of floods vs. other natural disasters, we discovered that from 1990 to 1991, there were a lot of land and mud slides in eastern Asia that led to a 388% increase in total affected population due to floods caused by heavy rains. Put simply, indirect flood-related disasters have proven devastating to human life loss. When exploring river and coastal flooding for the USA on a city level, river flooding area had a drastic increase starting from 2015. Finally, by enriching the data with flood Insurance claims, we found that the most recent dollar claim value impact occurring in Ida, Oklahoma. Here is the link to Tellius’s Show Floor Showdown video. Data Science and Machine Learning Show Floor Showdown Vendors’ Key Findings and Videos Aible Key Findings: Predicting floods is not as actionable as focusing our resources on trying to address floods that affect at least 100 people. Such a specific goal enabled a far more useful analysis. We need to balance multiple KPI such as 'lives affected' vs. 'economic impact of the floods' as well as stay within relevant budgets. We need to consider fairness. For example, if we stayed within current budgets and only focused on the economic impact of floods, we would under-invest in under-developed regions. This would be contrary to the goals of a hypothetical UN agency. By trying out different definitions of fairness we can balance our fairness goals against economic ones. A manual process cannot possibly consider so many different constraints, but an AI-first business-outcome-driven approach conducts such analysis in minutes. Here is the link to Aible’s Show Floor Showdown video. eQ Technologic Key Findings: Cropland exposure and Built-up area exposure related parameters play a major role is deciding the flood predictions. As for USA, maximum floods have occurred in Quarter2. The highest cause of floods remains heavy rains and torrential rains. It is observed that Texas has maximum floods in the last 5 years, and they have been successful in reducing the damage due to floods in last 3 years. Arizona has also reduced the damage and fatalities caused due to floods in the last 3 years. Across the world, it is observed that cropland exposure to river floods have gone down over the years. Similarly, build-up area exposure to coastal floods has also gone down. Here is the link to EQ Technologic’s Show Floor Showdown video. Data Management Show Floor Showdown Vendors’ Key Findings and Videos These vendors provided findings related to managing the data in their platforms. Cloudera Hybrid analytics and transaction functions can coexist on the same data management platform—but require different storage, processing, memory and channels management from each other. This reinforces the demand for workload management and separate optimization techniques. Orchestration is the key to data management across intercloud, multi-cloud and hybrid on-premises systems. Pervasive and persistent monitoring of use cases can result in automation of pattern-based or repetitive tasks. Pattern-basis can also consist of gradual drift and shift of workload patterns and this can be built into a predictive efficiency model. Metadata management goes beyond technical metadata and even combining that with business metadata when it infers or anticipates new users of recently introduced data assets. Effectively the “pull” analytics data search can be converted into a “push and verify” model in which new data is pushed to existing use cases. Cloudera comes from being built for the “cloud era”. Here is the link to Cloudera’s Show Floor Showdown video. Cockroach Labs Key Findings: The danger from natural or human-made disasters is amplified by loss of connectivity to or availability of data in geographically isolated data sets. Redundancy of data is not the same as “redundant copies of data”. Petulant maintenance of isolated copies of data increase cost. Whereas functional copies of data for the purpose of resilience and continuity of operations is a value-based investment. Distributed data assets requires the eventual synchronization of independent collection, and a distributed management system enables differing tempos between data capture, data reuse and data integration flows that better match business needs or demands. Cockroaches can go up to two weeks without food or water, and the protein receptors in their antennae help them determine if a new digestible compound has been discovered within three days—they can eat anything that has embedded protein no matter how scarce or difficult to get. Here is the link to Cockroach Lab’s Show Floor Showdown video. Exasol Embedding standardized data processes that are used by multiple analytic platforms and tools provides use-case consistency of the data—even when the analytics intent, design and outcomes are different. A mix of in-memory, physical storage, network-based data access management and automated redistribution of data based upon use cases permits broader acceptance of data assets as speeds, costs and access points are tuned to actual user requirements. Analytics data use cases continue to require very different statistics and physical resource management than transactional/operational systems in 2023 as evidence by speed tests and performance analysis across transactional systems and analytics focused data management approaches. Exasol’s name comes from Exabytes of Data Solved for Data management. Here is the link to Exasol’s Show Floor Showdown video. Thank you to all our amazing Bake-Off and Show Floor Showdown panelists!! Thank you to my cookie monster and script compliance judge at the BI Bake-Off, Aura Popa, my co host for the Data Science and Machine Learning Bake-Off, Georgia O'Callahan and to the rest of the team, Mark Beyer, Prasad Pore, Zain Khan  and JC Martel! A special thank you to our project managers without whom these sessions would not be possible, Sharlynn Sarmiento, Leah Ciavardini and Nellie Taratorin!! Happy Baking! Regards, Rita Sallam, Georgia O'Callahan, Aura Popa, Mark Beyer, Prasad Pore, Zain Khan  and JC Martel