什么是云计算？这还用问么？当然…..

Cloud computing is an entirely new paradigm shift in how IT services are built, managed and consumed. Cloud computing delivers higher efficiency, limitless scale and faster and easier deployment of new services and systems to the end users. Cloud computing also changes the economics of the datacenter by shifting the delivery of IT resources to an on-demand model. In the process, cloud computing is opening up new business models and opportunities for enterprise and service providers.  From: cloud.com

Related posts:

1. Open Source and Cloud Computing(开源与云计算)
2. Cloud Computing Infrastructure and Architecture Guide-From SUN
3. BMC and Cloud computing

For as long as there have been computers, operating systems and business critical applications, there has been monitoring of these items to make sure they were working (available) and performing well for their constituents. In the early days of the computer industry (through about 1982) computer systems were vertical monoliths where a customer would buy “an accounting system” and that purchase would include hardware, systems software, and applications software. All of this usually came from one vendor so there was one throat to choke when it did not work.

Starting with the delivery of the PC, then Ethernet LAN’s and Novel Netware, and finally Windows Servers and the Internet, the industry reorganized along horizontal dimensions. A computer system was now a multi-layer cake and you could buy each cake from one of many vendors whose products were largely interchangeable with each other, and compatible with adjacent layers. So you could buy an Intel based server from one of N vendors, put either Linux or Windows on it, but a Java applications server from one of N vendors on that and then buy applications from thousands of different vendors.

This horizontal layering of the industry was heavily promoted by Microsoft and Intel (who “owned” two key layers of the cake), and also ushered in the tremendous innovation and price competition that continues to drive our industry today. Moore’s Law says that microprocessor performance doubles roughly every 18 months or so, and with those increases in price/performance come the more for less mentality that we have all become accustomed to.

With this freedom of choice at every layer of the cake however came problems. The first problem was complexity. There were now too many cooks in the kitchen which meant that there were both too many and not one single throat for the CIO to choke when something went wrong. The management software industry reacted to this by inventing a marketecture, Business Service Management (BSM). The idea behind BSM was to identify the key business services that applications or combinations of applications delivered to users (the ability to enter and order and ship the resulting product is a business service as is the ability of a consumer to transfer funds from one banking account to another), and to then map all of the software and hardware infrastructure that this business service depended upon and to manage that collection of linked resources as an entity.

While Business Service Management sounded like a great idea, it in fact ended up as a miserable failure. The reasons were:

• There were many applications for which it was just too hard to measure their performance (in response time terms). Web applications turned out to be pretty easy, but fat client Win32 applications written to proprietary client/server protocols turned out to be very hard. It turned out that for most enterprises, more than half of the business critical applications were in the hard pile (fat client/Win32) and less than half were the easy web applications.
• Many BSM frameworks relied upon scripts and synthetic transactions to measure response time and end user experience. However this approach failed for many enterprises due to the large number of applications that comprised a business service and how rapidly these applications were changing. For example if synthetic transactions were used to measure the performance of a set of transactions, and a company had 500 business critical applications (not a very high number – some companies have thousands), and each application was updated once a year, then that would translate into 10 updates a week for the monitoring scripts.
• The same is true for most passive monitoring approaches that rely upon templates to identify transactions in applications protocols. There are many products that can measure the response time of transactions from the perspective of the web server by attaching physical or virtual appliance to a span port on the switch that services the web server. However all such products require configuration to understand what set of granular http request/responses combine to create a transaction of interest. Maintaining these configurations across hundreds of applications and thousands of transactions proved to be a major impediment to instrumenting their service level in any kind of a broad and systematic manner.
• It turned out to be a nightmare to identify the hardware and software that supported each business service. This gave rise to the need for Configuration Management Databases (CMDB’s) that were supposed to get populated with the hardware and software assets and their relationships to each other. The effort to put in place a CMDB and to maintain it turned out to doom the CMDB projects and the associated BSM projects at many companies.
• The BSM vendors were unable to evolve their products at the same rate as the innovation of the vendors who contributed products to the layers in the cake. New devices (laptops that were not always connected), new protocols (ICA, VOIP), new operating systems (Linux), new languages (Java, C#), and new user interfaces (first the browser and then rich Internet applications) all occurred at a pace that no one vendor could keep up with.
• When the BSM vendors got overwhelmed by innovation, vendors of point monitoring solutions stepped in to monitor the newest layer or item in a layer.  This lead to a proliferation of monitoring tools which were not integrated with each other.
• When a problem occurred even if one had a BSM product and a CMDB it still horribly difficult to know exactly what path the failed transaction took through the entire hardware and software infrastructure. The BSM tools were rarely aware of every element of the stack and the tactical monitors that had been bought to fill in the cracks were not integrated with the BSM tools nor each other.

In summary we entered the age of virtualization and the cloud with both BSM and their supporting CMDB’s having failed at monitoring and managing a static infrastructure where applications and services largely stayed on dedicated hardware. In other words, before virtualization and the dynamic data center it was impossible for anyone in IT to see a problem and get told in a deterministic manner exactly where the problem was and how to fix it.

The Impact of Virtualization and the Cloud

Since holistic end-to-end monitoring of business services was essentially broken (or not attainable) before virtualization, it is reasonable to assume that virtualization and the cloud will only make this problem worse. This will occur for the following reasons:

1. Virtualization increases workload density and the dynamic operation of workloads. This will require both continuous mapping of application to infrastructure dependencies and more more frequent (near real time) collection of performance data. Just the requirements for real time mapping and real time data completely overwhelm existing monitoring systems. Hyperic has a good blog on this point here.
2. Inferring application performance from resource utilization statistics becomes impossible. This was possible on physical hardware, but on hardware that is shared via virtualization it no longers works. Hence the need for Infrastructure Performance Management solutions that measure Infrastructure Response Time. We expect Infrastructure Performance Management solutions from vendors like Akorri, CA Technologies (CA Virtual Assurance), Virtual Instruments, and Xangati to form the foundation layer of whatever will replace BSM.
3. Application will now get moved from cluster to cluster and ultimately from data center to data center (private cloud – hybrid cloud – public cloud). APM solutions will need to track the applications no matter where they go, and seamlessly work across different IP networks. Leading virtualization aware APM solutions like those from AppDynamics, BlueStripe, and New Relic meet these needs today and will likely form the APM layer of whatever replaces BSM.

All of the above together combine to create one new result for Systems Management. That new result is that in the general case it will be impossible to deterministically do root cause in a dynamic environment. This was explored in detail in this post.

The Significance of the Integrien Acquisition

The Integrien acquisition by VMware is significant because it means that VMware has recognized that only a dynamic, statistical, self-configuring,  and self-learning approach can keep up with the rate of change in these new dynamic IT environments. The self-learning approach simply means that you feed the system the metrics that get collected about the system and it figures out which ones are important, how the metrics are related to each other, and lets you know when anomalies have occurred.

This acquisition is all the more significant because this is not garden variety technology. There have only ever been three companies this this space. ProactiveNet was acquired by BMC a few years ago. Netuitive has been around since the late 1990′s and it took the company until the mid 2000′s before the product had matured into something that really just worked when you plugged it in. Integrien is a fairly recent entry in this field and is now part of VMware – which leaves Netuitive as the only remaining independent player.

The New Dynamic BSM – Service Assurance

Since the old BSM is dead due to a brittle and difficult to update technology approach it is reasonable to ask what will replace it. The answer is most likely a set of Infrastructure Performance Management tools (Akorri, CA Virtual Assurance, Virtual Instruments, Xangati) integrated with a set of next generation APM tools (AppDynamics, BlueStripe, New Relic) vis these self learning technologies. When this occurs, we will have a system that adapts on its own to changing conditions in the environment leaving IT staff available to interpret results (and not raw monitoring data). We will also have taken an important step towards dynamic service assurance which was discussed in detail in this post.

Understanding and Evaluating these Technologies

For most IT professionals either they or someone on their staff can digg in and understand how the technologies that they use work. However unless you have an advanced degree in statistics and/or mathematics you are not going to be able to dig an and decide for yourself based upon how these products work which one you should choose. Rather what you should do is apply the following criteria in making your selection:

1. Decide exactly what you want the product to do for you. These products are extremely flexible. You can feed them every alarm that is generated by all of your monitoring solutions and let them sort out the good ones from the bad ones. Or you can feed them revenue per minute for one key business application and let them figure what causes degradations in revenue per minute.
2. Make sure that the product has connectors to what you already use to collect metrics from your systems. These products are not in the data collection business (with some exceptions). They rely upon other products to collect data from them. They must therefore be interfaced with your existing monitoring solutions.
3. How hands off and plug and play will the product be in practice? This is the key criteria to the long term value of such a solution to your enterprise. Previous attempts at statistical approaches (neural nets) failed because the product had to be “retrained” every time conditions changed. Make sure that the product you select can automatically select and weight the inputs that it bases decisions off of and these these decisions are automatically updated over time. Self-learning needs to be a continuous thing, not just a one time or periodic thing.
4. Make sure that the product can handle time based (time series) as well as event based data. Performance metrics tend to be time based, but many performance problems are caused by configuration changes which are events. Make sure that the product can cross-correlate configuration change events with performance degradations.
5. Carefully assess the scale of the solution. This means how many inputs can the solution take per unit of time. Right now most of these solutions operate at 15 minute or 5 minute intervals. Monitoring a dynamic system may require intervals of 10 or even 5 seconds (or perhaps even real time continuous streams of monitoring data).

Summary

Self-learning performance management solutions like Integrien and Netuitive are going to be absolutely an essential part of the migration to dynamic data centers and IT as a Service. Once these dynamic data centers scale out to the thousands of applications in a typical enterprise, and scale up to address the most performance critical applications, the rate of change in the environment will be too high for legacy tools and manual administration to be able to keep. up. These automated self-learning approaches will be the only way in which IT Operations will be able to stay on top of these new environments while staying within staffing and budget constraints.

Related posts:

1. 图书推荐《服务建模：原理与应用》
2. 让Remedy飞一会
3. 老美真的很扯，一个职位的帖子被顶了22次

View Full Screen

Related posts:

1. Remedy ARS 开源工具知多少
2. OTRS.ORG，it is time to check it out；不得不：）
3. MartinMarks for 29th August

http://www.information-management.com/news/amazon_bmc_software_cloud_computing-10015748-1.html

BTW: it seems the IT management is as important as infrastructure.

Related posts:

1. Open Source and Cloud Computing(开源与云计算)
2. Enterprise cloud computing
3. 云定义

From above post:

开源的成功有几个关键元素：

1.许可证要允许和鼓励再发布、修改乃至发展分支；
2.一个体系结构要使程序能被作为组件在任何可能的地方重用，以及可以被扩展，而不是被替换来提供新功能；
3.低门槛，让新用户轻松上手一试。
4.低门槛，让开发人员构建新的应用与大家分享。

所以我给出我的第一个建议：如果你关心云计算的开源，请在那些设计为联合而不是集中控制的服务上构建项目。体系结构从来都是战胜许可证的。

个人认为：使用云计算的用户并不一定是租用土地的佃户；佃户把自己的收成都给了地主，自己剩下的少的可怜。云计算的用户会愿意付出多少成本给地主是可以计算的，计算的原则就是‘保证自己的利益、安全等’。

Related posts:

1. Cloud Computing Infrastructure and Architecture Guide-From SUN
2. BMC and Cloud computing
3. 云定义

http://www.datacenterknowledge.com/

这个网站还是不错，有很多关于数据中心的知识。它应该主要是收集业内在该领域里新咨询，先收录一下，回头慢慢学习。

这个vido我觉得也可以学习一下。IBM和Google结成联盟开展企业云计算方面的研究，可能会建立“IBM-Google cloud ” 这个云会运行在Linux上，使用Xen 做系统虚拟和Apache Hadoop。同时Google和IBM都有自己巨大的数据中心，还不知道它们将来的研究成果会运行在谁家的设备上，让我们拭目以待。

Tips：

有些网络视频比较大，在线看如果网速比较慢的话可能不能正常浏览。很多下载视频的工具好像都是要收费注册的，否则只能下载60％。我发现的一个方法是，把视频网址输入到这里 http://www.techcrunch.com/get-youtube-movie/ 它会给你生成一个http的下载连接，点击后一般的http下载工具就能下整个视频了，之后用视频浏览的软件查看，正在缓慢下载上边的视频。不知道你有什么好方法或者是工具可以推荐呢？

http://youtubedownload.altervista.org/ 是我现在所使用的，不过下载的太慢了，不能支持多线程下载。

Related posts:

1. BMC and Cloud computing
2. Open Source and Cloud Computing(开源与云计算)
3. DO you have a google App Engine?

最近两周最吸引我的TAG是这个词“cloud computing”。在我前一个文章中稍微展望了一下云计算的前景“云计算吧电能转化成计算能力，然后向用电一样的来使用，我们甚至不需要使用PC这样的个人信息话设备来管理我们的信息，所有的信息和信息的处理都在云上发生”。在了解了更多的信息之后，这些可能实现，但是这种巨变需要时间，而且可能是很长时间；清醒之后不得不佩服Google在这方面的宣传。

理性认识云计算的简单方式，扫扫盲：

• 阅读一下wikipedia对‘cloud computing’的解释
• 听听John Willi的两个Podcast
• 多看看非google来源的信息：）
• 读读这个GRIDtoday的文章 — “Clouds are Looming and We Love It”

很多厂商都在参与这个热点，open source也要加油，期待开源在这个领域的发展。

Related posts:

1. 云定义
2. Open Source and Cloud Computing(开源与云计算)
3. Cloud Computing Infrastructure and Architecture Guide-From SUN