GEM — Generic Enterprise Manager
GEM is an enterprise infrastructure monitoring and inventory management platform built for hedge funds and investment banks managing heterogeneous on-premises environments. It provides a unified view of all databases and servers from a single web console.
What GEM Does
| Capability | Description |
|---|---|
| Configuration Management | Full CMDB — hosts, databases, applications, and service dependencies inventoried and tracked. |
| Monitoring & Alerting | Continuous MDA-based collection with rule-driven alarm routing, escalation, and notification. |
| Operations Management | Scheduled batch jobs, on-demand operations, run history, and status tracking. |
| Reporting | Web-based reports covering space, performance, locking, blocking, and configuration drift. |
| Stored Procedure Library | Extended sp__ procedures for Sybase and SQL Server — installed and managed by GEM. |
Build & Version
Build 758 (Dec 4 2012). Legacy production codebase — Perl 5.8+, Sybase-backed, cross-platform
(Windows via Win32_perl_lib_5_8/, Unix native).
Architecture
GEM is a Perl application backed by a central Sybase database (GEMDB server,
gemalarms schema). All monitoring data, inventory records, alarm rules, and
configuration live in this schema.
Stack
| Layer | Technology |
|---|---|
| Language | Perl 5.8+ |
| Database | Sybase — GEMDB server, gemalarms schema |
| Web Server | IIS (Windows) or Apache (Unix) — serves Perl CGI console |
| DBI Abstraction | DBIFunc.pm — Sybase, SQL Server, Oracle, MySQL |
| Credentials | Encrypted password files under conf/ via Repository.pm |
Key Directories
| Directory | Contents |
|---|---|
lib/ | Core Perl modules — alarm engine, inventory, DBI, reports |
conf/ | Configuration and encrypted credential files |
scripts/ | Monitoring and data-collection scripts |
bin/ | Build tools, admin utilities, and test scripts |
console/ | Web console Perl CGI pages |
doc/ | Full documentation (frameset HTML) |
Supported Platforms
GEM is designed to monitor heterogeneous environments. The collector runs on any platform where Perl 5.8+ is available.
Monitored Database Types
| Database | DBD Driver |
|---|---|
| Sybase ASE | DBD::Sybase (Unix) / DBD::ODBC (Windows) |
| Microsoft SQL Server | DBD::ODBC |
| Oracle | DBD::Oracle |
| MySQL | DBD::mysql |
Monitored Host Types
Windows servers, Unix/Linux hosts. Platform capability checks are gated
by RunOn.pm — controls which monitors execute on which platform/database combination.
Monitoring & Alerting
The alarm engine (MlpAlarm.pm, 287 KB) is the largest and most critical module.
It evaluates collected data against configured alarm rules, routes alerts, manages escalation,
and sends notifications.
Data Collection
Monitoring scripts under scripts/ run on a schedule managed by
JobScheduler.pm. They collect MDA (Monitor and Diagnostic Architecture) metrics
and write results back to the gemalarms schema.
Alarm Processing
| Phase | Description |
|---|---|
| Evaluation | Collected metrics are compared against configured thresholds in gemalarms |
| Routing | Alarms are routed to configured notification targets (email, pager, etc.) |
| Escalation | Unacknowledged alarms escalate through a defined severity chain |
| Notification | Alerts dispatched via configured channels; acknowledgement tracked in the database |
Alarm Rule Storage
All alarm rules, thresholds, routing logic, and notification targets are stored in
the gemalarms Sybase schema — not in flat config files. Changes take effect
without restarting any process.
Inventory & CMDB
Inventory.pm manages system inventory and attribute data. GEM maintains
a CMDB that covers all monitored systems and their relationships.
Inventory Records
| Entity Type | Description |
|---|---|
| Hosts | Windows and Unix servers — OS, hardware, role, status |
| Databases | Sybase, SQL Server, Oracle, MySQL instances — version, space, configuration |
| Applications | Applications and services running on monitored hosts |
| Dependencies | Service dependency relationships between systems |
Discovery scripts (bin/discover_sybase.pl, bin/discover_sqlsvr.pl,
bin/unix_metadata.pl, bin/mysql_metadata.pl) populate inventory
automatically.
Scheduled Operations
JobScheduler.pm reads and writes conf/jobscheduler.dat,
which defines all scheduled monitoring and maintenance tasks.
Job Types
| Type | Description |
|---|---|
| Monitoring collectors | Periodic scripts that gather metrics from monitored systems |
| Maintenance jobs | Database maintenance, log rotation, configuration sync |
| Report generation | Scheduled report runs that populate the web console |
| Alarm sweeps | Periodic evaluation of alarm rules against collected data |
On Windows, jobs are registered as Scheduled Tasks via
bin/schedule_tasks_on_win32.pl. On Unix, they run via crontab
(managed by bin/create_crontab.pl).
Reports
GEM generates web-based reports covering all aspects of the monitored environment. Reports are rendered as HTML and accessed through the web console.
Report Categories
| Category | Examples |
|---|---|
| Space & Capacity | Database space, device utilisation, growth trends |
| Performance | I/O stats, lock/block analysis, tempdb usage |
| Configuration | Config drift reports, parameter comparison across servers |
| Security | Login auditing, permission review, role assignment |
| Batch Jobs | Job run history, success/failure, duration trends |
| Incidents | Alarm history, escalation log, incident tickets |
Core report modules: GemReport.pm (base framework),
MlpReports.pm (extended report library).
Console
The GEM web console is a Perl CGI application served by IIS (Windows) or Apache (Unix). It provides access to all monitoring data, reports, inventory, and alarm configuration through a browser interface.
Build & Deploy
| Script | Purpose |
|---|---|
bin/build_the_console.pl | Assemble console pages from templates |
bin/publish_the_console.pl | Deploy built console to web server root |
bin/rollout_web_server.pl | Full web server configuration rollout |
Configuration
GEM configuration is split between the conf/ directory (flat files and
encrypted credentials) and the gemalarms database (alarm rules, thresholds,
routing). Run-time configuration is managed by gem_conf_editor.pl.
Credential Management
Repository.pm / RepositoryRun.pm manage encrypted
password files stored under conf/. These files hold database login
credentials for all monitored systems. Credentials are never stored in plain text.
Initial Setup
| Step | Script |
|---|---|
| Configure monitoring targets | bin/mustedit_discover.pl → edit then run gem_conf_editor.pl |
| Set up GEM database | bin/setup_gem_database.pl |
| Install stored procs | bin/install_gemalarms.pl |
| Sync config to database | bin/sync_configfiles_and_db.pl |
| Test connectivity | bin/test_dbi_connectivity.pl |
Database Schema
All GEM data lives in a Sybase database server named GEMDB. The primary
schema is gemalarms. This is the single source of truth for all monitoring
state, inventory records, alarm rules, and configuration.
Key Tables (gemalarms)
| Table Area | Contents |
|---|---|
| Inventory | Hosts, databases, applications, and service dependencies |
| Alarm Rules | Thresholds, conditions, routing targets per monitored resource |
| Collected Data | Current and historical metrics from monitoring scripts |
| Alarm State | Active alarms, acknowledgement status, escalation chain state |
| Job Definitions | Scheduled job configuration (mirrored from jobscheduler.dat) |
| Incidents | Incident tickets raised from alarm escalations |
Schema creation: bin/gemalarms_create_actual.sql (production) /
bin/gemalarms_create_template.sql (template).
Module Reference
All core Perl modules live in lib/.
Core Modules
| Module | Purpose |
|---|---|
| MlpAlarm.pm | Master alarm routing engine (287 KB) — evaluation, routing, escalation, notification |
| Inventory.pm | System inventory and attribute management |
| DBIFunc.pm | Generic DBI abstraction across Sybase, SQL Server, Oracle, MySQL |
| Repository.pm | Encrypted credential file management |
| RepositoryRun.pm | Runtime credential access and caching |
| JobScheduler.pm | Reads/writes conf/jobscheduler.dat; controls scheduled jobs |
| RunOn.pm | Platform capability checks — gates which monitors run per platform/DB type |
| GemReport.pm | Base report generation framework |
| MlpReports.pm | Extended report library |
| Incident.pm | Incident ticket management |
| CommonFunc.pm | Shared utility functions |
| LogFunc.pm | Logging framework |
Sybase-Specific
| Module | Purpose |
|---|---|
| lib/Sybase/SybFunc.pm | Sybase-specific helpers |
| lib/Sybase/DBISybFunc.pm | DBI+Sybase combination utilities |
| lib/Sybase/Login.pm | Sybase login and connection management |
| lib/Sybase/Release.pm | Sybase release/version detection |
Full Documentation
The complete GEM documentation — installation guide, configuration reference,
console user guide, alarm configuration, and troubleshooting — is in the
legacy frameset documentation under doc/.
| Document | Link |
|---|---|
| Full Documentation (frameset) | doc/index.htm |
| Change Log | CHANGE_LOG.html |
| FAQ | FAQ.html |
| Troubleshooting Guide | Troubleshooting_Guide.htm |
| GEM Database Schema | GemDatabase.htm |
| Stored Procedure Library | EdBarlowsStoredProcs |