这是一个补丁修补版本。我的名字也第一次出现，感谢Michael的帮忙，没有你我可能迟迟发不上去这个翻译。完整的release notes见 http://otrs.org/releases/3.0.11

Related posts:

1. OTRS FAQ 1.5.3 : you have six reasons to use it
2. OTRS::ITSM期待中的开源ITIL工具
3. OTRS.ORG，it is time to check it out；不得不：）

用tideway用户登录ADDM服务器，运行下面的命令

[tideway@localhost .ssh]$ ls
[tideway@localhost .ssh]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/usr/tideway/.ssh/id_rsa):  “敲回车”
Enter passphrase (empty for no passphrase):  “敲回车”
Enter same passphrase again:  “敲回车”
Your identification has been saved in /usr/tideway/.ssh/id_rsa.
Your public key has been saved in /usr/tideway/.ssh/id_rsa.pub.
The key fingerprint is:
27:20:55:fc:f9:9a:41:35:11:ef:6d:b7:6c:4e:78:00 tideway@localhost.localdomain
[tideway@localhost .ssh]$ ls
id_rsa  id_rsa.pub
[tideway@localhost .ssh]$

下载和安装key文件部署工具

http://code.google.com/p/keploy/downloads/list

下载 keploy-0.5.tgz
上传到ADDM服务器的/usr/tideway/.ssh目录中

[tideway@localhost .ssh]$ ls
id_rsa  id_rsa.pub  keploy-0.5.tgz
[tideway@localhost .ssh]$ tar zxvf keploy-0.5.tgz
keploy-0.5/
keploy-0.5/keploy
keploy-0.5/LICENSE
keploy-0.5/README
keploy-0.5/INSTALL
keploy-0.5/CHANGELOG
[tideway@localhost .ssh]$ ls
id_rsa  id_rsa.pub  keploy-0.5  keploy-0.5.tgz
[tideway@localhost .ssh]$ cd keploy-0.5
[tideway@localhost keploy-0.5]$ ls
CHANGELOG  INSTALL  keploy  LICENSE  README
[tideway@localhost keploy-0.5]$ ./keploy -v

NOTICE: You may be prompted for you password,
NOTICE: this is directly from the ssh client, not keploy

Preparing to deploy ssh key...
ERROR: No targets defined.
[tideway@localhost keploy-0.5]$ ./keploy -h
Usage: keploy [options] [hosts]

Options:
  -i ID_FILE            specifies identity file
  -l USERID, --login=USERID
                        define userid for remote connections
  -f TARGET_FILE, --file=TARGET_FILE
                        read list of targets from file
  -k, --use-known       read list of targets from known_hosts files
  -r, --remove          remove primary identity from remote(s)
  -c OLD_ID_FILE, --change=OLD_ID_FILE
                        replace old identity with new one on remote(s)
  -A                    enable/disable agent forwarding on remote
  -v                    give verbose output
  -q, --quiet           quiet the output
  --version             show program's version number and exit
  -h, --help            show this help message and exit
[tideway@localhost keploy-0.5]$

在keploy-0.5生成一个文件hostlist，里面每一行放一台机器的IP，回头让工具自动读取该文件

Linux被采集帐号准备

建议直接把ADDM采集机的key文件部署到root用户上，如果用户觉得这个做法不安全的话，我们可以建立一个不普通用户，把特权命令用sudo处理一下，下面是具体的做法

运行ssh的证书认证

用root登录被采集机，修改/etc/ssh/sshd_config文件，将如下参数的注释符号（#）去掉，以启用这些参数：
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
重启sshd服务
#/etc/init.d/sshd restart

配置sudo

Linux平台对采集账户需要配置sudo。以root用户登录被管服务器，修改/etc/sudoers文件，在文件末尾追加一行：
tideway ALL=(root) NOPASSWD:/usr/sbin/dmidecode,/usr/sbin/hwinfo, /usr/sbin/hbanyware/hbacmd,/usr/sbin/lsof,/sbin/ethtool,/sbin/mii-tool

增加采集帐号tideway

操作步骤如下（以root用户登录服务器）：
[root@ars home]# useradd tideway
[root@ars home]# passwd tideway
Changing password for user tideway.
New UNIX password: “使用tideway做密码，采集成功后可以修改掉”
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password: “使用tideway做密码，采集成功后可以修改掉”
passwd: all authentication tokens updated successfully.
[root@ars home]#

部署ADDM ssh 密钥到被采集机

部署

在采集机部署前，查看/home/tideway 目录

[root@ars home]# cd tideway/
[root@ars tideway]# pwd
/home/tideway
[root@ars tideway]# ls -la
total 36
drwx------ 3 tideway tideway 4096 Jun 17 00:55 .
drwxr-xr-x 4 root    root    4096 Jun 17 00:55 ..
-rw-r--r-- 1 tideway tideway   33 Jun 17 00:55 .bash_logout
-rw-r--r-- 1 tideway tideway  176 Jun 17 00:55 .bash_profile
-rw-r--r-- 1 tideway tideway  124 Jun 17 00:55 .bashrc
-rw-r--r-- 1 tideway tideway  515 Jun 17 00:55 .emacs
drwxr-xr-x 4 tideway tideway 4096 Jun 17 00:55 .mozilla
-rw-r--r-- 1 tideway tideway  658 Jun 17 00:55 .zshrc
[root@ars tideway]#

注意这里没有.ssh目录
开始使用我们下载的工具部署，登录到ADDM采集机上

[tideway@localhost keploy-0.5]$ vi hostlist
[tideway@localhost keploy-0.5]$ cat hostlist
192.168.189.154
[tideway@localhost keploy-0.5]$ ./keploy -f hostlist

NOTICE: You may be prompted for you password,
NOTICE: this is directly from the ssh client, not keploy

Preparing to deploy ssh key...
Using user-defined host list from file: hostlist
./keploy:164: DeprecationWarning: os.popen3 is deprecated.  Use the subprocess module.
  si, so, se = os.popen3(execute)
        Found host(s):
                192.168.189.154
        Found identity:
                /usr/tideway/.ssh/id_rsa.pub
        Working on host: 192.168.189.154
The authenticity of host '192.168.189.154 (192.168.189.154)' can't be established.
RSA key fingerprint is de:4e:f0:c8:9b:86:ee:25:af:a9:1c:19:8f:20:90:ad.
Are you sure you want to continue connecting (yes/no)? yes
tideway@192.168.189.154's password:  "输入目标机上tideway用户的密码，我用的是tideway"
                Public Identity Key: deployed
[tideway@localhost keploy-0.5]$

可以到被采集机的/home/tideway/.ssh下面看看传过去的正式文件，这种方式极大的避免了，手工创建.ssh目录，设置它和key文件权限的工作量

验证

在采集机上，使用tideway用户登录被采集服务器。

[tideway@localhost keploy-0.5]$ ssh 192.168.189.154
Last login: Fri Jun 17 01:08:21 2011 from 192.168.189.147
[tideway@ars ~]$

Related posts:

1. openssh 5.2 Aix5.3 bug
2. BMC Atrium CMDB 类定义迁移
3. CMDB中存什么？

众所周知Remedy ARS上运行的所有程序代码都是在数据库中存储的，几乎所有的程序代码和相关数据都是存储在后端的数据库里的。基于这种特性，我们可以很方便地从数据库层面实现对ARS系统的备份和恢复。所谓基于数据库的备份和恢复就是指数据库的全库导入和导出。在这方面Oracle做的很好，他所提供的imp和exp命令行工具能很好的完成此项任务。

When you may backup BMC Remedy ARS system Database

Remedy ARS系统数据库全备份可能会发生在下面的几个时间点：

1. 在安装过程中，Remedy ITSM套件的安装都是从ARS开始一个模块一个模块增加安装的，可以在安装每个新模块之前，做一个数据库备份，用于安装失败情况下的现场恢复

2. 在系统上线前，当系统需要做升级/代码更新等等大规模操作之前可以考虑做一次数据库的导出备份

3. 在做系统迁移的情况下，我们甚至可以生产库迁移到开发机上进行问题的重现，或者代码的修改，或者测试，完成之后再导回生产系统（如果生产系统再次期间处于维护状态的话）

How-to

下面是我总结的相关操作步骤参考。

备份ARSystem数据库

exp aradmin/ar#admin@itil file=c:\before-patch7.dmp log=c:\exp.log compress=n statistics=none

参数解释：
aradmin/ar#admin  是ARS系统默认的用户名和口令
@itil 是ARSystem数据库所在的数据库实例名
file=c:\before-patch7.dmp log=c:\exp.log  是导出的数据库文件和日志文件路径

删除ARAdmin用户数据

drop user aradmin cascade;

创建和授权ARAdmin用户

CREATE USER ARADMIN
IDENTIFIED BY ar#admin#
DEFAULT TABLESPACE ARSYSTEM
TEMPORARY TABLESPACE ARTMPSPC
PROFILE DEFAULT
ACCOUNT UNLOCK;

GRANT ARADMINROLE TO ARADMIN;
ALTER USER ARADMIN DEFAULT ROLE ALL;
ALTER USER ARADMIN QUOTA UNLIMITED ON ARSYSTEM;

导入备份的数据文件

imp ARAdmin/AR#Admin@ars file=c:\CMDB2010-12-13_13.dmp log=c:\imp.log  fromuser=ARAdmin touser=ARAdmin ignore=y

Credit to：以上操作步骤由神州太岳工程师李成旗协助编写和测试.

Related posts:

1. BMC课程通知: BMC Remedy AR 7.5 管理员培训
2. Remedy ARS 开源工具知多少
3. Remedy ITSM 7.6.04 套件快速安装

CI数据导出示例

Command: xexpdt
XEXPORT DATA
Export instance data from all classes? (F):
Classes to export data from:
Class and its subclasses, From namespace (1,2) (1):
Namespace (): BMC.CORE
Class name (): BMC_ComputerSystem
Dataset ID (): BMC.ASSET
Filename for exported data: C:\DEMO\cs-ci
Exporting Class BMC_ComputerSystem
i=0: namespaceName = BMC.CORE, className = BMC_ComputerSystemExported without errorsExporting Class BMC_MainframeExporti
ng Class BMC_PrinterExporting Class BMC_StorageSubsystem
CMDBExportData  results
ReturnCode:  OK
Status List : 0 items

到相应目录下面找到导出的文件，导出的结果至少是两个文件一个是META_FILE其他的才是CI数据文件。

关系数据导出示例

Command: xexpdt
XEXPORT DATA
Export instance data from all classes? (F):
Classes to export data from:
Class and its subclasses, From namespace (1,2) (1):
Namespace (): BMC.CORE
Class name (): BMC_Dependency
Dataset ID (): BMC.ASSET
Filename for exported data: C:\DEMO\REL-DEP.XML
Exporting Class BMC_Dependency
i=0: namespaceName = BMC.CORE, className = BMC_DependencyExported without errorsExporting Class BMC_AccountOnSystemExpor
ting Class BMC_HostedAccessPointExporting Class BMC_HostedServiceExporting Class BMC_InventoryBulkItemsExporting Class B
MC_InventoryComputerSystemsExporting Class BMC_InventoryEquipmentExporting Class BMC_InventorySystemComponents
CMDBExportData  results
ReturnCode:  OK
Status List : 0 items

到相应目录下面找到导出的文件，导出的结果至少是两个文件一个是META_FILE其他的才是CI数据文件。

导入数据示例

Command: impdt
IMPORT INSTANCE DATA
Handle duplicate Instance Ids:
(Error/New ID for Dup/Merge/New ID for All) (1-4) (1): 3
Filename containing import data: c:\demo\cs-ci_0_0  (导入包含CI数据的文件)

CMDBImportData  results
ReturnCode:  OK
Status List : 0 items
Total items Processed             :  2
Items Imported successfully       :  2
Items for which Import failed     :  0

Command: impdt
IMPORT INSTANCE DATA
Handle duplicate Instance Ids:
(Error/New ID for Dup/Merge/New ID for All) (1-4) (1): 3
Filename containing import data: c:\demo\REL-DEP.XML_0_0 (导入包含关系数据的文件)

CMDBImportData  results
ReturnCode:  OK
Status List : 0 items
Total items Processed             :  1
Items Imported successfully       :  1
Items for which Import failed     :  0

导入完毕之后使用Remedy User到相应的表格中查找，确认导入的数据是否正确。

Related posts:

1. BMC Atrium CMDB 类定义迁移
2. Stop to build CMDB for your IT – CMS是怎样炼成的？
3. Remedy ARS 7.6.03 overlay

cmdbdriver.exe能支持的参数

Options:
-u<USER>
-a<AUTHORSTRING>
-p<PASSWORD>
-l<LOCALE>
-s<SERVER>
-x<INPUTFILE_FULLPATH>
-t<PORT_NUMBER>
-q<QUIET_MODE_NUMBER>

登录CMDBdriver 实例：   cmdbdriver -u Demo -p bmc@XXX -s atriumcmdb -t 9988

登录之后在运行另外用于初始化的两个命令 init 和log

Command: init
INITIALIZATION

CMDBInitialization  results
ReturnCode:  OK
Status List : 0 items

Command: log
Control record:
Authentication string () :
User name () : Demo
Password () : **********
Locale[.charSet] () :
TimeZone () :
Server () : atriumcmdb

导出类定义的方法实例
Command: xexpdf
XEXPORT DEFINITION
Export all classes? (F):
Classes to export:
Class and its subclasses, From namespace (1,2) (1):
Namespace (): BMC.CORE
Class name (): BMC_ComputerSystem
Export all attributes with classes? (T):
Filename for exported data: c:\demo\BMC_ComputerSystem.xml

CMDBXExportDef  results
ReturnCode:  OK
Status List : 0 items

使用文本编辑器查看c:\demo\BMC_ComputerSystem.xml的导出结果

导入类定义的方法：
Command: impdf
DEFINITION IMPORT
Import Item List:
Number of import items (0):
Import option : Create/Overwrite (1-2) (1): 2
Filename containing import data: c:\demo\cs.xml

CMDBImportDef  results
ReturnCode:  OK
Status List : 0 items

使用Remedy User，用Demo登录，把语言切换到英文，然后在查BMC_ComputerSystem表达，在custom 2 ～n的tab页面里面查看哪些导入的属性。或者使用class manager查看确认导入是否成功。

Related posts:

1. BMC Atrium CMDB CI数据迁移
2. Remedy ARS 7.6.03 overlay
3. Stop to build CMDB for your IT – CMS是怎样炼成的？

二、配置管理的目的：
1. 确定、控制、记录、报告、审计、验证服务资产和配置项包括版本、基线、组成成分、属性和相关关系。
2.通过服务生命周期管理保护资产完整、配置项等账户。确保只有已授权的组件被使用和已授权变更被执行。
3.通过服务生命周期保护服务资产、配置项的完整性。为了建立和维持一个准确和完整的配置管理系统，确保资产和控制服务、IT基础设施的配置需求的完整性。

三、资产、配置管理的活动

1、规划
2、识别
配置项识别过程：
A) 定义和制定标准文件来选择配置项和他们的组件构成
B) 依据标准选择配置项及其组件并记录他们
C) 给配置项分配唯一的标识符
D) 指定每个配置项相关属性
E) 确认每个配置项是受配置项管理来管理
F) 确定每个配置项的责任人
3、控制
必须有效控制信息以维持配置管理数据库(CMDB)的及时更新。一旦某项活动改变了配置项已记录的特征或配置项之间的关系，则必须在配置管理数据库 (CMDB)中记录该项变动。需注意的是：只有变更管理才有权批准对配置项的特征进行变动，事件管理只能改变某个现有的配置项的状态来反映现实状况。
配置管理负责控制组织接收到的所有IT 组件并需确保这些组件被记录在系统中。硬件可在其已订购或已交付时进行记录，而软件则通常在其被纳入DML时进行记录。
4、记录
组件的生命周期可被划分成多个阶段，每个阶段都可以分配一个状态代码，但具体分成几个阶段则取决于公希望记录IT 基础设施的哪些特征。保持对每次状态变化日期的记录可以提供关于一个产品的生命周期的有用信息，如订购时间、安装时间以及所需的维护和支持。组件的状态决 定了可以对其进行操作的余地。
5、审核和报告
执行审计是为了核实配置管理数据库(CMDB)中记录的信息是否仍然反映了当前的现实状况。这些信息可用来检查和更新配置管理数据库(CMDB)。
在下列情形下需要执行审计：
A) 在建立了新的配置管理数据库(CMDB)之后；
B) 建立配置管理数据库(CMDB)一段时间之后；
C) 重大变更之前或之后；
D) 灾难恢复之后；
E) 其他任何方便的时候。
审计也可以随机地或在配置经理认为配置管理数据库(CMDB)中的信息不正确时进行。如果配置管理系统与审计工具之间存在关联，则可以每天生成针对某个相 关领域的审计报告。在发现差异时，不应该允许审计工具自动更新配置管理数据库(CMDB)。所有的差异都表明变更管理流程可能被忽视了，所以应该对这些差 异进行调查并通过变更管理对这些差异进行追溯性处理。
三、配置管理管理报告
配置管理流程的管理报告可能包括下列信息：
a) 有关流程质量的信息；
b) 在审计中发现的记录与实际情形不符的次数；
c) 发现未经批准的配置的次数；
d) 出现已记录的配置不能被找到情形的次数；
e) 审计中发现的有关配置项属性详细程度方面的差异；
f) 处理信息记录请求所需的时间；
g) 超过给定事件或变更次数的配置项的列表；
h) 有关 IT 基础设施的结构和组成的统计信息；
i)  有关 IT 基础设施发展情况及其他方面的信息；
j) 有关改进措施的总结、报告和建议，例如，针对由于业务、技术、市场价格和其他相关变化所导致的配置管理追踪记录的配置项的范围和详细程度的变更所提出建议；
k) 有关实施流程所耗费的人力成本的清单。

四、配置管理的绩效指标指标
a) 闲置许可数量;
b) CMDB错误导致变更失败的数量;
c) 未授权配置数量;
d) 配置项文档错误导致变更失败所引起的事故数量;
e) 因为CMDB错误导致违反SLA的数量;
f) 无相应配置项刷新的变更请求数量;
g) 不精确配置项百分比;
h) 客户满意度 ;

五、与其他流程的关系

转帖学习一下。

From http://blog.amteam.org/standard/A1159174.html

Related posts:

1. MartinMarks for 29th August
2. 规划CMDB数据填充-002
3. ITIL v3 术语表

版本的确认：所有操作系统，JVM，JDK，数据库，数据库客户端，和以上软件的位数都需要查明，都需要在Remedy的兼容文档中一一确认。

AIX操作系统的字符集：需要同时安装中英文的UTF8字符集。安装方法smitty，需要有操作系统的安装盘，安装完成之后，locale -a ，需要能看到 ZH_CN, ZH_CN.UTF-8,EN_US, EN_US.UTF-8 这四种字符集缺一不可。否则无法以中文方式安装成功，也无法让操作系统已中文utf8来启动AR服务。

XManager需要安装在于AR同一网段的Windows服务器上，例如Mid-tier可能安装在Windows上，并与AR在同以网段，那么安装程序最好在该服务器的Xmanager的客户端中安装，从哪里连接AIX服务器并运行安装程序。如果不在同一个网段，安装程序很有可能安装了一半或者无名的消失，安装进程会立刻中断，需要把系统推到重来。

AIX系统上font server的安装：使用netstat -na|grep 7100， 如果没有结果则需要从AIX的光盘中在AIX的服务器上安装font server和相关的包，并且启动它。如果不装的话，remedy安装程序可能界面显示不正常，或者不显示。

Oracle上众多僵尸进程：如果在安装CMDB或者ITSM套件的时候，安装界面挂死或者消失，这会是Oracle服务器上产生上百个僵尸进程，这些进程会吃光Oracle服务器的所有物理内存。我们的oracle服务器的16GB内存被吃光之后，ITSM安装界面几乎停止，20~30分钟才能安装完一个定义文件。解决方法是：随时监控Oracle服务器的内存使用情况，当出现内存被僵尸进程吃光的情况，可以考虑使用IT行里无人不知的无敌重启方案，重启Oracle服务器的操作系统，从启动Oracle数据库实例，重启ARS服务，然后再次安装Remedy ITSM。

Remedy安装之备份：如果是在AIX+Oracle的数据库上安装。Remedy还是非常绿色的软件，需要在安装完AR，CMDB，ITSM套件后各做一个Remedy安装文件目录的备份，在AIX上可以把这个文件夹tar起来备用。Oracle数据库还是非常容易备份的，使用imp和exp命了就可以搞定，具体语句咨询DBA。切忌在安装CMDB和ITSM之前一定要对Remedy系统的安装文件和数据库做备份，一旦安装失败，可以恢复文件，恢复数据库，排错后重新再来。

Related posts:

1. Remedy ITSM 7.6.04 套件快速安装
2. BMC提供ITSM洋快餐
3. Remedy ITSM 7.6 installation Tips

For as long as there have been computers, operating systems and business critical applications, there has been monitoring of these items to make sure they were working (available) and performing well for their constituents. In the early days of the computer industry (through about 1982) computer systems were vertical monoliths where a customer would buy “an accounting system” and that purchase would include hardware, systems software, and applications software. All of this usually came from one vendor so there was one throat to choke when it did not work.

Starting with the delivery of the PC, then Ethernet LAN’s and Novel Netware, and finally Windows Servers and the Internet, the industry reorganized along horizontal dimensions. A computer system was now a multi-layer cake and you could buy each cake from one of many vendors whose products were largely interchangeable with each other, and compatible with adjacent layers. So you could buy an Intel based server from one of N vendors, put either Linux or Windows on it, but a Java applications server from one of N vendors on that and then buy applications from thousands of different vendors.

This horizontal layering of the industry was heavily promoted by Microsoft and Intel (who “owned” two key layers of the cake), and also ushered in the tremendous innovation and price competition that continues to drive our industry today. Moore’s Law says that microprocessor performance doubles roughly every 18 months or so, and with those increases in price/performance come the more for less mentality that we have all become accustomed to.

With this freedom of choice at every layer of the cake however came problems. The first problem was complexity. There were now too many cooks in the kitchen which meant that there were both too many and not one single throat for the CIO to choke when something went wrong. The management software industry reacted to this by inventing a marketecture, Business Service Management (BSM). The idea behind BSM was to identify the key business services that applications or combinations of applications delivered to users (the ability to enter and order and ship the resulting product is a business service as is the ability of a consumer to transfer funds from one banking account to another), and to then map all of the software and hardware infrastructure that this business service depended upon and to manage that collection of linked resources as an entity.

While Business Service Management sounded like a great idea, it in fact ended up as a miserable failure. The reasons were:

• There were many applications for which it was just too hard to measure their performance (in response time terms). Web applications turned out to be pretty easy, but fat client Win32 applications written to proprietary client/server protocols turned out to be very hard. It turned out that for most enterprises, more than half of the business critical applications were in the hard pile (fat client/Win32) and less than half were the easy web applications.
• Many BSM frameworks relied upon scripts and synthetic transactions to measure response time and end user experience. However this approach failed for many enterprises due to the large number of applications that comprised a business service and how rapidly these applications were changing. For example if synthetic transactions were used to measure the performance of a set of transactions, and a company had 500 business critical applications (not a very high number – some companies have thousands), and each application was updated once a year, then that would translate into 10 updates a week for the monitoring scripts.
• The same is true for most passive monitoring approaches that rely upon templates to identify transactions in applications protocols. There are many products that can measure the response time of transactions from the perspective of the web server by attaching physical or virtual appliance to a span port on the switch that services the web server. However all such products require configuration to understand what set of granular http request/responses combine to create a transaction of interest. Maintaining these configurations across hundreds of applications and thousands of transactions proved to be a major impediment to instrumenting their service level in any kind of a broad and systematic manner.
• It turned out to be a nightmare to identify the hardware and software that supported each business service. This gave rise to the need for Configuration Management Databases (CMDB’s) that were supposed to get populated with the hardware and software assets and their relationships to each other. The effort to put in place a CMDB and to maintain it turned out to doom the CMDB projects and the associated BSM projects at many companies.
• The BSM vendors were unable to evolve their products at the same rate as the innovation of the vendors who contributed products to the layers in the cake. New devices (laptops that were not always connected), new protocols (ICA, VOIP), new operating systems (Linux), new languages (Java, C#), and new user interfaces (first the browser and then rich Internet applications) all occurred at a pace that no one vendor could keep up with.
• When the BSM vendors got overwhelmed by innovation, vendors of point monitoring solutions stepped in to monitor the newest layer or item in a layer.  This lead to a proliferation of monitoring tools which were not integrated with each other.
• When a problem occurred even if one had a BSM product and a CMDB it still horribly difficult to know exactly what path the failed transaction took through the entire hardware and software infrastructure. The BSM tools were rarely aware of every element of the stack and the tactical monitors that had been bought to fill in the cracks were not integrated with the BSM tools nor each other.

In summary we entered the age of virtualization and the cloud with both BSM and their supporting CMDB’s having failed at monitoring and managing a static infrastructure where applications and services largely stayed on dedicated hardware. In other words, before virtualization and the dynamic data center it was impossible for anyone in IT to see a problem and get told in a deterministic manner exactly where the problem was and how to fix it.

The Impact of Virtualization and the Cloud

Since holistic end-to-end monitoring of business services was essentially broken (or not attainable) before virtualization, it is reasonable to assume that virtualization and the cloud will only make this problem worse. This will occur for the following reasons:

1. Virtualization increases workload density and the dynamic operation of workloads. This will require both continuous mapping of application to infrastructure dependencies and more more frequent (near real time) collection of performance data. Just the requirements for real time mapping and real time data completely overwhelm existing monitoring systems. Hyperic has a good blog on this point here.
2. Inferring application performance from resource utilization statistics becomes impossible. This was possible on physical hardware, but on hardware that is shared via virtualization it no longers works. Hence the need for Infrastructure Performance Management solutions that measure Infrastructure Response Time. We expect Infrastructure Performance Management solutions from vendors like Akorri, CA Technologies (CA Virtual Assurance), Virtual Instruments, and Xangati to form the foundation layer of whatever will replace BSM.
3. Application will now get moved from cluster to cluster and ultimately from data center to data center (private cloud – hybrid cloud – public cloud). APM solutions will need to track the applications no matter where they go, and seamlessly work across different IP networks. Leading virtualization aware APM solutions like those from AppDynamics, BlueStripe, and New Relic meet these needs today and will likely form the APM layer of whatever replaces BSM.

All of the above together combine to create one new result for Systems Management. That new result is that in the general case it will be impossible to deterministically do root cause in a dynamic environment. This was explored in detail in this post.

The Significance of the Integrien Acquisition

The Integrien acquisition by VMware is significant because it means that VMware has recognized that only a dynamic, statistical, self-configuring,  and self-learning approach can keep up with the rate of change in these new dynamic IT environments. The self-learning approach simply means that you feed the system the metrics that get collected about the system and it figures out which ones are important, how the metrics are related to each other, and lets you know when anomalies have occurred.

This acquisition is all the more significant because this is not garden variety technology. There have only ever been three companies this this space. ProactiveNet was acquired by BMC a few years ago. Netuitive has been around since the late 1990′s and it took the company until the mid 2000′s before the product had matured into something that really just worked when you plugged it in. Integrien is a fairly recent entry in this field and is now part of VMware – which leaves Netuitive as the only remaining independent player.

The New Dynamic BSM – Service Assurance

Since the old BSM is dead due to a brittle and difficult to update technology approach it is reasonable to ask what will replace it. The answer is most likely a set of Infrastructure Performance Management tools (Akorri, CA Virtual Assurance, Virtual Instruments, Xangati) integrated with a set of next generation APM tools (AppDynamics, BlueStripe, New Relic) vis these self learning technologies. When this occurs, we will have a system that adapts on its own to changing conditions in the environment leaving IT staff available to interpret results (and not raw monitoring data). We will also have taken an important step towards dynamic service assurance which was discussed in detail in this post.

Understanding and Evaluating these Technologies

For most IT professionals either they or someone on their staff can digg in and understand how the technologies that they use work. However unless you have an advanced degree in statistics and/or mathematics you are not going to be able to dig an and decide for yourself based upon how these products work which one you should choose. Rather what you should do is apply the following criteria in making your selection:

1. Decide exactly what you want the product to do for you. These products are extremely flexible. You can feed them every alarm that is generated by all of your monitoring solutions and let them sort out the good ones from the bad ones. Or you can feed them revenue per minute for one key business application and let them figure what causes degradations in revenue per minute.
2. Make sure that the product has connectors to what you already use to collect metrics from your systems. These products are not in the data collection business (with some exceptions). They rely upon other products to collect data from them. They must therefore be interfaced with your existing monitoring solutions.
3. How hands off and plug and play will the product be in practice? This is the key criteria to the long term value of such a solution to your enterprise. Previous attempts at statistical approaches (neural nets) failed because the product had to be “retrained” every time conditions changed. Make sure that the product you select can automatically select and weight the inputs that it bases decisions off of and these these decisions are automatically updated over time. Self-learning needs to be a continuous thing, not just a one time or periodic thing.
4. Make sure that the product can handle time based (time series) as well as event based data. Performance metrics tend to be time based, but many performance problems are caused by configuration changes which are events. Make sure that the product can cross-correlate configuration change events with performance degradations.
5. Carefully assess the scale of the solution. This means how many inputs can the solution take per unit of time. Right now most of these solutions operate at 15 minute or 5 minute intervals. Monitoring a dynamic system may require intervals of 10 or even 5 seconds (or perhaps even real time continuous streams of monitoring data).

Summary

Self-learning performance management solutions like Integrien and Netuitive are going to be absolutely an essential part of the migration to dynamic data centers and IT as a Service. Once these dynamic data centers scale out to the thousands of applications in a typical enterprise, and scale up to address the most performance critical applications, the rate of change in the environment will be too high for legacy tools and manual administration to be able to keep. up. These automated self-learning approaches will be the only way in which IT Operations will be able to stay on top of these new environments while staying within staffing and budget constraints.

Related posts:

1. 图书推荐《服务建模：原理与应用》
2. 让Remedy飞一会
3. 老美真的很扯，一个职位的帖子被顶了22次

第一步 安装前的准备工作。
确定Remedy ARS的服务别名，例如“AtriumCMDB”。在所有的Web服务器（Mid-tier所安装的服务器）的host文件中加入一条Ip地址解析，例如：

192.168.10.11   AtriumCMDB

此ARS服务别名指向的是该Web服务器所对应的ARS服务器，例如：Web1中AtriumCMDB对应的ip为ARS1，Web2对应ARS2，Web3对应ARS3，以此类推。

第二步安装第一台ARS服务器
默认所有的ARS都安装了数据库客户端程序，如果是Oracle数据库，ARS上的客户端程序的大小版本号必须和远程数据库的大小版本号完全一致。Windows平台的Oracle客户端只支持32位的程序。在所有ARS服务器的host文件中加入一条Ip地址解析，例如：

192.168.10.11   AtriumCMDB

此IP地址为每台ARS自己的对外提供ARS服务的IP。ARS上安装完JDK之后，开始安装ARS，安装过程中服务器别名输入AtriumCMDB，其他的选项都按需要配置，所有有关服务器端组件、服务端口、密码、安装路径的信息都要做详细记录，用来安装Server Group中其他成员使用。安装完第一台服务器的ARS之后，申请Remedy License，打License，包括其他所有CMDB、ITSM相关应用模块的License，打完License后导出成文件备用。ARS安装成功之后，顺序安装其他应用，顺序时CMDB 》ITSM  其他。安装完毕后，通过Remedy User来确认所有应用功能是否正常。

第三步 配置第一台ARS服务器为Server Group中的管理服务器
配置方法参照，ARS Configuration Guide中的Server Group的相关章节。配置完毕之后打开Server Group的Log，从启动ARS服务之后，查看该Log看Server Group工作是否正常。

第四步 安装Server Group中的成员ARS服务器
准备工作参考第一台ARS服务器。运行ARS安装程序，选择Server Group，选择输入AtriumCMDB别名，选择共享的数据库，其他参数与第一台保持一致。安装完毕之后。使用ARS自带的Sample应用新增一个city，在ARS1上查询ARS2上新增的记录。同样参考的Server Group的相关章节，对ARS2进行配置。在ARS2上查看Server Group的日志，确认该ARS已经加入了以第一台ARS为管理服务器的群集中。为第二台ARS服务器打License。在确认第二台ARS服务器成功加入之后，安装CMDB应用。安装完毕之后，在第二台ARS服务器上，使用Remedy User客户端，打开CMDB的相关表单进行新增和查询操作；然后在ARS1上检查操作结果，保证两边一致。安装ITSM：直接把第一台ARS服务器的ar.cfg文件覆盖到第二台ARS的ar.cfg上，一定要修改第一台ARS服务器主机名的哪一行，把它修改为第二台ARS的主机名。复制第一台ARS的ITSM安装目录到第二台ARS的相同路径中，重启ARS服务。查看arerror.log文件看看ARS启动的是否正常。在第二台ARS上使用Remedy User确认ITSM应用是否工作正常，如果一切工作正常，则第二台ARS服务器安装完毕。按照相同的方式安装其他的ARS服务器。

第五步 配置每台ARS的ranking
按照ARS Configuration Guide中的Server Group的相关章节配置每台ARS服务器处理不同后台工作流的ranking。

第六步 安装配置所有Web服务器的Remedy Mid-tier
安装Remedy Mid-tier软件，都指向相同的ARS服务别名AtriumCMDB，当然该别名被解析为它所对应的ARS服务器的IP地址。使用浏览器测试每台Web服务器，保证Remedy Mid-tier都能正常工作。

第七步 配置F5负载均衡
配置F5的分发策略，按不同ARS服务器的用途，来分别不同的用户请求。考虑管理和接口功能的ARS负担少量的用户交互。开发一个jsp的程序部署在Mid-tier的shared目录中，用它来判断Web所对应的ARS的可用性，以此作为唯一判断条件来分发用户请求给可用的web服务器。

Related posts:

1. BMC课程通知: BMC Remedy AR 7.5 管理员培训
2. BMC Atrium CMDB 类定义迁移
3. Remedy ITSM 7.6.04 套件快速安装