Sentar has been using Artificial Intelligence and Machine Learning technology for years to provide advanced malware detection and classification. Our technology was derived from concepts in the Human Genome Bioinformatics efforts and it was the genesis of DARPA’s Cyber Genome project. Now, there is an open source project, Apache Spot that provides access to similar technology that you can explore yourself. Apache Spot uses Big Data Analytics and Machine Learning that can be applied to improve or create new cybersecurity applications.
The Apache Spot software can analyze billions of events in order to detect unknown or insider threats and provide new network visibility. It uses machine learning as a filter to separate bad traffic from benign and to characterize network traffic behavior. It makes it easier to integrate cross-application data for better enterprise visibility and new analytic functionality. The open data models make it easier for organizations to share analytics as new threats are discovered.
Computerworld published an article this week describing the project in more detail:
Originally created by Intel and launched as the Open Network Insight (ONI) project in February, the effort is now called Apache Spot and has been accepted into the ASF Incubator.
“The idea is, let’s create a common data model that any application developer can take advantage of to bring new analytic capabilities to bear on cybersecurity problems,” Mike Olson, Cloudera co-founder and chief strategy officer, told an audience at the Strata+Hadoop World show in New York. “This is a big deal, and could have a huge impact around the world.”
Based on Cloudera’s big data platform, Spot taps Apache Hadoop for infinite log management and data storage scale along with Apache Spark for machine learning and near real-time anomaly detection. The software can analyze billions of events in order to detect unknown and insider threats and provide new network visibility.