Last week I had the pleasure of attending DevOpsDays NYC, located at the beautiful Viacom office in midtown Manhattan. This was the 7th year that NYC hosted DevOpsDays, and I have to say it was a big success.
I’ve been lucky to attend several of the NYC-based events and am always impressed with the organizers' ability to create an inclusive and engaging environment for all attendees. This time, my new company — CHAOSSEARCH — was a first-time DevOpsDays sponsor. We had an extremely busy yet productive couple of days talking with numerous individuals, organizations, and sponsors about the challenges they face managing their rapidly growing log and event data.
The speaker lineup was impressive and everyone did a fantastic job, from the Keynotes to the Lightning talks in the afternoon. Dominica DeGrandis, author of “Making Work Visible,” kicked off DevOpsDays with a fascinating talk about “time theft” across organizations and how to discover what is really happening when you have many disparate systems and work groups. I loved the focus on making the value stream visible in order to get the big picture of how your organization gets work done.
Kubernetes still remains the technology of the moment. I noticed that many of the hallway track conversations (my personal favorite track at DevOpsDays) were centered around companies in various stages of their Kubernetes journey. Rob Boll, a software engineer at Datadog, gave an excellent talk called “Kubernetes: The Very Hard Way.” This talk gave great insight into the rationalization and design choices a company makes as they investigate a new and complex technology to help them scale their platform.
Open Spaces are always a highlight at DevOpsDays events. The ability to join a group of individuals all talking about similar challenges they have or solutions they’re building is a great way to stay connected to the real world of DevOps problems. The Open Spaces covered a wide range of topics, including Onboarding Engineers, Mental Health in Technology, and Monitoring and Observability at Scale. Even though many companies are adding to their Observability story by leveraging Time-Series metrics and Tracing, the death of centralized logging has yet to happen; if anything, logging continues to grow in usage for many businesses.
To me, one of the most important aspects of DevOpsDays was the conversations with attendees at our CHAOSSEARCH table. We discussed their growing log volume as they’ve moved to microservices. It was a common thread for anyone running Kubernetes. In the past, we used to have single applications per “server” or Amazon EC2 Instance, and that application would essentially be a single log stream from that one server. Now, we’ve entered a new world where Kubernetes enables you to run MANY microservice applications on your individual systems. We’ve gone to a place where people now have dozens or even hundreds of log streams per individual system. All this data streaming to Fluentd, and eventually into their hot Elasticsearch clusters. Another common complaint from these same engineers and operators was the competing challenge between the needs of the business to keep data searchable for analytics and compliance, and the ever-growing cost of keeping that data hot in Elasticsearch — which leads to an ever-increasing spend to simply run the ELK Stack on Amazon.
It’s always fun to attend these events and share the story of what we are building at CHAOSSEARCH. Our ability to demonstrate for engineers, operators, and anyone who gets paged in the middle of the night for logging pain that they can finally reduce the size of their hot Elasticsearch clusters is great validation of the platform. They can finally complement ELK by waterfalling data to their Amazon S3 and leverage CHAOSSEARCH to reduce the size and complexity of their logging systems — and significantly reduce their expenses...
Congrats to all the organizers, volunteers, and speakers who help make these events so wonderful to attend. See you all next year!