aware  0.11.1.src
About: aware-.* Asynchronous Event Framework for Responsive Applications, System Control and Monitoring (let system administrators tune system variables, set monitoring/security alarms, build adaptive distributed systems and more). Beta version.
  Fossies Dox: aware-0.11.1.src.tgz  ("inofficial" and yet experimental doxygen-generated source code documentation)  

Aware

Overview

The Aware project is an effort to create a software framework to measure, monitor, and control computer system resources. Aware is intended to enable system administrators tune system variables, set monitoring/security alarms and build adaptive distributed systems. Aware modules may be linked into applications making them 'aware' and able to participate in the larger managed system.

Ultimately, multiple systems will monitor themselves and others, cooperating to make decisions to optimally tune performance, proactively enhance security and compensate for faults.

The Aware software is a high performance distributed event processing framework built for systems management. It comes with probes for the common network services and system resources. Additionally, Aware allows the cross-correllation of many different streams of information.

The development goals are:

  • High performance with small footprint to minimize the impact on host and allow deployment on small systems.
  • Dynamically extensible such that 3rd parties can supply modules outside the Aware core software base.
  • Linkable libraries such that applications may incorporate Aware features and cooperate in system control and monitoring.
  • Portable, clean and simple code.

The development has 3 major phases:

  1. Resource measurement and monitoring tools
    • High speed efficient event processor
    • Linkable library APIs
    • Rich set of monitoring probes and notification/action handlers
    • High level probe/alarm/handler configuration language
    • Standalone applications for system monitoring/analysis
  2. Network wide adaptive control of resources
    • Secure messaging between Aware enabled applications
  3. Awareness
    • Smart/Adaptive distributed cooperative system control

Why? (Long Term Ivory Tower Vision)

It is not uncommon today to have a system with more than 100 or even a 1000 networked computers. Since the complexity of managing such systems rises exponentially with the number of computers connected (i.e., consider multiple ways each computer can interact with itself and others) we will rapidly reach a point in the next few years where it will not be possible to manage our vast networks of computers using the human intensive methods we generally employ today.

The solution to this problem lies within the systems themselves. In order to manage the next generation of computer systems that connect millions of computers, the computers themselves must take on the burden of managing such issues as security and availability. That's not to say the goal is to eliminate the human element in the decision making processes, rather, the systems need to be self monitoring and adaptive such that they only present the humans a manageable amount of information. IBM has a high level overview of the elements required to create such a system, they call these the 8 Elements.

The goal of this project is to build a working framework on which such systems can be built.

Why? (Short Term Practical Vision)

In the real world, conceptual systems are of little value. A goal of the Aware project is to make the software immediately useful while building on a framework that can extend to reach the loftly goals outlined in the previous section.

In order to build smart/adaptive feedback controlled distributed systems 4 major areas of functionality are required:

  1. Sensors:A comprehensive set of sensors that gather relevant information
  2. Analyzers: Components that process data from the sensors and issue controller commands
  3. Controllers: Components that change system state (e.g., run programs, change system parameters, control devices)
  4. Stores: Components responsible for the persistent storage and retrieval of sensor data

Essentially, the above describes a system/network monitoring system (S/NMS). Therefore, the short term goal of the Aware project is to build and extensible system/network monitoring platform. However, this is being done in the context of the long term vision and implies an implementation framework very different from most current monitoring packages. This framework has the following characteristics:

  • Open source implementation allows for robust code base and customization
  • Common core engine implements a model of event processing
  • A "plug in" style mechanism allows dynamic addition of handlers
  • Agents are composed of a set of running event handlers
  • Agents can get their configuration from other agents (e.g., a centrally managed set of agent configurations)
  • Agents can communicate with other agents using connection oriented, connectionless and broadcast based methods
  • Agents have authentication/authorization mechanisms
  • High performance and low impact implementation allows agents to do their jobs very efficiently

Once Aware has a comprehensive set of system management functions it can be extened in a natural way to the less mundane issues such as proactive security, adaptive provisioning, etc.

Using Aware

Software Architecture

The Aware software architecture has 3 conceptual levels, each with a different target user base:

  1. API: C code.
    Intended User Base: Programmers needing a high performance async event engine
  2. API: Wire scripts and command line tools.
    Intended User Base: Sysadmins that don't mind scripting
  3. API: Web based UI w/database.
    Intended User Base: Sysadmins that don't want to script and/or want easy to use tool.

Level 1 allows a programmer to create programs that use the aware event engine for high performance async event oriented applications. The discover program that comes with the distribution is a port scanner written in C that uses the event engine. Programmers can write their own event handlers in C and (if they follow the standard API) these handlers will be available via wire scripts. Programmers can develop these event handlers outside the aware distribution because each handler is a dynamic object, loaded at run time. Other examples include a very fast and scaleable webserver (see xap ) and a mulitcast file copy (see mcctx and mccrx ).

Level 2 makes it very easy to string together event handlers in 'wire' scripts. Using the 'exec' handler you can integrate with command line tools (e.g., usb-snmp tools). The examplesPage directory in the distribution contains many working examples of 'wire' scripts.

Level 3 builds on the previous levels with a complex handler that implements a web based UI by sub-classing the built in web server handler and adding a Sqlite database whose schema is oriented towards systems monitoring. By default the Aware installation will create a service configured to run at boot that implements this web based UI. Through this web base UI can can do various kinds of network monitoring as well as receive events from agents on other machines.

Aware Agents for Network Monitoring

Agents may be trivially configured through a web based user interface or through scripts to monitor all standard IP based services. Typically, you would install an Aware agent on a small number of machines inside your network that monitor all servers, sending notifications when a service or server is unavailable. Agents running outside your network monitor your system's availability from other networks or the Internet.

Aware Agents for System Monitoring

Aware can monitor all standard system statistics (e.g., processes, load, swap, free memory) generating events when system administrators require notificaction or taking actions (e.g., restarting failed applications).

Aware Agents for Security Monitoring

Aware can monitor all login/logout by users and watch sensitive files for modification. Aware can work with any security application that generates text logs, generating alerts when there are indications of a security issue (e.g., watching the tail of /var/log/messages for security related messages like 'su root' or dropped packets by a packet filter).

Aware's high speed event processing make it ideal to work with IDS's like Snort.

Aware Agents Cooperating

In the future, Aware agents will be able to communicate securely allowing feedback based control of your system. For example, Aware agents may detect a drop in traffic on your web servers (e.g., at night EST) and send events to a central Aware agent that calculates the aggregate load and the optimal number of servers. This agent then sends shutdown messages to the uneeded servers (we assume that your load balancer will gracefully reroute traffic to the remaing servers), thus saving considerable money on power costs. Later, as traffic increases, the central Aware agent sends a message to a power sentry on servers that need to be brought online to service the traffic.

You can implement a dynamic security policy using Aware agents. Consider a system with Aware agents monitoring sensitive files and network traffic on many servers and that one of these agents detects an active network based security breach. This agent may send a notification to the Aware agent on the firewall to immediately add a firewall rule to notch out all traffic from the offending IP address.

Available Handlers

All available handlers are listed here: Handlers . Click on the header file names to see the documentation.

Web Based Management User Interface

The managment UI (similar to OpenNMS, Big Brother, NetSaint) allows basic network monitoring via a web based interface. Features include:

  • Automatic network discovery
  • Drill down style display of groups, nodes, interfaces and services
  • Performance graphs for interfaces (ping rtt) and services (probe rtt).
  • Availability reports
  • SNMP enabled traffic graphs

Screenshots

User Documentation

The following is the online help:

  • uiPagedoc-Home
  • uiPagedoc-Nodes
  • uiPagedoc-Alerts
  • uiPagedoc-Admin
  • uiPagedoc-Tools

Useful Documentation