Era Software announced the private beta version of EraStreams, a no-code data pipeline that lets users integrate, transform, and route observability data to EraSearch, the company's petabyte-scale log management platform, and third-party monitoring tools.

Era Software is broadening the scope of its log management solution and fitting into more of the observability stack. With a time series database and object storage under the hood, the company's approach to observability data management resolves scale, performance, and cost issues associated with running applications on modern architectures, including cloud, containers, and microservices.

EraStreams complements EraSearch to optimize cost and performance and integrates into existing DevOps workflows and tools to help teams manage observability costs and improve troubleshooting effectiveness. As a result, IT and security teams can continue to use monitoring tools they rely on while controlling the volume of data that gets routed to these tools to optimize data usage and cost efficiency. Teams also have the option to route any logs to EraSearch for low-cost storage and fast, petabyte-scale query.

"Our vision is to be the observability data management choice for organizations dealing with massive volumes of observability data," said Todd Persen, Co-founder and CEO, Era Software. "The unveiling of EraStreams today advances this vision with a data pipeline to help you manage observability costs and improve troubleshooting effectiveness. In addition, it gives you real-time insights into application and system performance and adds another component to our scalable, cost-effective observability data management."

For security teams with high log management costs and performance challenges using expensive SIEM solutions, EraStreams transforms and routes optimized datasets to a SIEM for security analytics and offers an option to send raw data to EraSearch for cost-efficient log management. If personally identifiable information (PII) poses a security risk, teams can protect sensitive data by masking PII before writing it to data storage. Data managed in EraSearch can be efficiently retrieved through its observability data rehydration capability when needed for investigations or threat hunting.

EraStreams was designed with ease of use and reliability to help teams see how observability data flows through pipelines and manage data at scale. Challenges with data flow stem from system failures and data intake variation. EraStreams better handles failure modes and pipeline changes to minimize data loss with dynamic backpressure management and reconfigurations. In addition, EraStreams provides a powerful set of features that offer multiple ways to reduce observability costs. When used with EraSearch, EraStreams reduces the total cost of ownership for existing log management solutions while preserving historical information in EraSearch for low-cost object storage and fast search and query.

"Today, some companies may generate over 100 terabytes of log data per day, and scale and pricing prevents many organizations from ingesting more data," added Persen. "With EraSearch and EraStreams, we can help you manage high volumes of data at a lower cost per GB ingested – we give you the ability to ingest and make the data queryable in real time. You should be able to find a needle in the haystack. With EraSearch and EraStreams, you can economically ingest a petabyte of log data daily with an average response time of less than 500 milliseconds."

Some of the common use cases EraStreams supports include:

- Log management cost reduction

- Troubleshooting

- Compliance and risk management

- Data integration