One of the most rewarding aspects of being a marketer is the ability to both hear, cultivate and share customer success stories. So much of any marketer’s time is spent sharing messages that are aimed at capturing attention, and when the balance kicks in and you can candidly share how the solutions you market are really making a difference, the reward is great!
That said, today’s blog features the success of our customer, Equifax, and their journey to simplified and more powerful operational analytics with ChaosSearch. Thanks for tuning in!
As Equifax collects, aggregates, and delivers financial data on over 800 million individual consumers to support business decision-making for employers, government agencies, and financial institutions in 24 countries.
Global data, analytics and technology companies such as Equifax, and their Engineering teams, depend on log analytics for a variety of operational analytics use cases, from application troubleshooting to streamlining cloud operations and regulatory compliance management. ChaosSearch is uniquely positioned to help companies like Equifax significantly reduce the time, cost, and complexity of log analytics.
We explore this in detail in our latest case study which features our longtime ChaosSearch customer, Equifax.
Equifax Simplifies Cloud Operations Management Globally with Cloud Data Platform
This case study details the fintech log analytics challenges Equifax faced as it moved its operations into the cloud and scaled to 50TB of log data ingested per day. You’ll discover why Equifax chose to re-imagine its log analytics program, and how adopting ChaosSearch helped Equifax achieve a more cost-effective, secure, and democratized data analytics solution.
As a sneak preview of our Equifax case study, this blog highlights five fintech log analytics challenges the company faced while massively scaling its operations in the cloud:
For each challenge, we’ll explore its impact on the business, why traditional solutions were falling short for Equifax, and how adopting ChaosSearch helped Equifax find a cost-effective solution.
When Equifax recently implemented a best-in-class multi-cloud environment, one of its core objectives was to democratize access to its data. Data was spread out in many locations and difficult to access.
With many separate systems aggregating log data for analytics, the Equifax data engineering team struggled with the complexity and time demands of supporting so many deployments.
As Equifax continued to scale its daily data ingestion, it became clear that its current system of siloed deployments wasn’t working. Storing data across multiple systems was getting more expensive, and the lack of a centralized data repository meant that:
By adopting ChaosSearch, Equifax was able to employ a single, centralized cloud data repository in Amazon S3, taking full advantage of its modern multi-cloud environment.
This significantly relieved the pressure on the engineering team, improved economies of scale in data storage, and prevented Equifax from losing insights because of siloed and disconnected data sources.
Read: How to Set Up a Data Analytics Practice That Works for Your People
Before adopting ChaosSearch, Equifax was dependent on legacy analytics solutions. As Equifax scaled up its log data ingest to 50 TB/day and beyond, the costs associated with running these solutions (i.e. pipeline maintenance, data egress costs, storage costs, etc.) across multiple deployments were beginning to grow exponentially.
Equifax found that its decentralized, piecemeal approach to log analytics was leading to out-of-control costs that significantly reduced the efficiency and ROI of doing analytics.
In fact, legacy logging solutions were never designed for the scale of log data that Equifax was ingesting each day. Siloed legacy stack tool deployments became more costly and unstable as their indices grew in size, resulting in duplicate data, high data storage costs, and high management overhead.
By adopting ChaosSearch, Equifax was able to index, search, and analyze log data directly in Amazon S3 buckets, the most cost-effective repository for data in the cloud. With no data movement, no duplication, and low storage fees, ChaosSearch offered Equifax a cost-efficient solution for consolidating its log analytics operations.
Other log analytics solutions follow a schema-on-write approach that requires data engineers to clean, prepare, and transform data before it is available for analytics applications. When Equifax was running multiple siloed legacy stack deployments, data engineers were spread too thin, which meant long waits to repair broken pipelines or make new data available for business units.
Across the business, Equifax found that the time and effort needed to analyze log data at scale was becoming unmanageable. The slow pace of data meant that critical insights were unavailable for SREs and product teams who really needed them.
Equifax could have tried to meet this challenge by hiring more data engineers, but only at significant expense. The real problem was Equifax had too many deployments that were too expensive to keep operating separately, but also couldn’t be consolidated.
“Taking a piecemeal approach to data analytics got expensive quickly, and it became difficult to pull insights together to paint a broader picture that spanned knowledge silos. That’s where I saw the ChaosSearch opportunity. It’s a much more ubiquitous platform we can use to access data across disparate systems, at scale.”
— Jeff Kinsherf, SVP Engineering Services and SRE, Equifax
ChaosSearch follows a schema-on-read approach that eliminates the need for complex and fragile data pipelines. By adopting ChaosSearch, Equifax can now analyze log data much sooner after it is generated. Accelerated time to insights gives Equifax compliance teams, SREs, product managers, and DevOps teams the information they need to resolve issues faster and make better decisions.
Read: Enhancing Cloud Security and Ensuring Regulatory Compliance for FinTech Companies
Complex data analytics workflows are characteristic of organizations that depend on legacy analytics solutions. That’s because these solutions require organizations to transform data and move it around prior to analysis. The processes around these steps can be labor-intensive and fragile, leading to further resource consumption and complexity.
Complex workflows are a barrier to data democratization. For Equifax, the time-consuming and labor-intensive process of maintaining data pipelines for multiple legacy stack tool deployment meant that only some data was available for some analysis by some people.
By adopting ChaosSearch, Equifax was able to significantly simplify its data analytics workflows. Log data is centralized and stored in Amazon S3, where it can be indexed, searched, and analyzed using ChaosSearch with no data movement and no ETL process. This makes it easy for end users to start accessing and analyzing log data soon after it’s generated, expediting insights and maximizing the value of log data.
Other legacy analytics solutions are complex and time-consuming to manage at scale. At Equifax, this challenge was multiplied by the number of siloed deployments that existed in various business units within the company. Managing Elastic clusters and data pipelines was consuming a lot of valuable IT resources.
The high management burden of running other legacy analytics solutions was translating into high costs for Equifax. As other deployments grew in size, Equifax would soon need more data engineers to keep up with the demand for data.
Traditional log analytics solutions simply weren’t designed to cope with the massive volumes of log data generated and collected in modern organizations.
“With ChaosSearch, we no longer have to move or transform our data. That means we don’t have to think about how we’re going to use the data before we access it. It’s just there for us to query. That helps us deliver new opportunities to the business, or discover usage patterns in our systems.”
— Jeff Kinsherf, SVP Engineering Services and SRE, Equifax
ChaosSearch removes the need for data engineers to transform and prepare data before consumption, reducing the management burden of log analytics and accelerating time to insights. Log data is ingested into Amazon S3, automatically indexed by ChaosSearch, and made available for analysis via an intuitive web-based interface.
In addition, a single ChaosSearch deployment replaced several deployments of another legacy analytics solution and eliminated hours per month of duplicated efforts. Partnering with ChaosSearch allowed Equifax to greatly simplify the management of its log analytics program.
ChaosSearch helps modern organizations Know Better® by activating the data lake for analytics at scale. Equifax is just one great example of a FinTech company that modernized and future-proofed its log analytics program by adopting the ChaosSearch cloud data platform.
To learn more about our success story with Equifax, and how ChaosSearch can help FinTech companies access analyze logs more efficiently, while significantly reducing their costs. Click the link below and check out the full case study.
Read the Blog: Logging Blindspots: Top 7 Mistakes that are Hindering Your Log Management Strategy
Check out Our: Other Customer Stories
Read the eBook: Beyond Observability - The Hidden Value of Log Analytics