Introduction

As those with IT responsibilities attempt to balance increased demands with increased costs, the topic of open source software (OSS) usually enters the conversation.  However, this very familiar term is still wrapped in a high degree of misunderstanding and vendor hype.  This paper will cover the best practices necessary for identifying the need for open source, evaluating open source options and integrating solutions based on standards like Storage Management Initiative Specification (SMI-S) and Cloud Data Management Interface (CDMI). It will examine the risks and benefits companies face when developing these products, and why open source and standards are a marriage made in IT heaven. The paper will also provide assistance to end users in determining the importance of standards by their vendors by asking them the tough question.

Who’s Switching to Open Source

As more governments and businesses are turning to open source applications to meet their IT requirements one needs to examine what are the benefits for implementing open source. Also if by adding products compliant to standards how much does it all save to the bottom line.

Reasons for Governments

First let’s start by examining why the White House and governments globally have chosen to move some of their applications to open source such as the White House moving its Web platform. The governments have realized key benefits for implementing open source by observing the benefits the enterprise sector has enjoyed for years by using open source. The six main reasons governments are switching to open source are:

• Security – it has been shown statistically that security increases by the number of users and developers of the OSS. As usage of the software increases more security issues and flaws in the software are discovered.
• Procurement time – government procurement of proprietary software can take up to 3 years for approval, but OSS provides the ability for immediate download and usage by government agencies due to no cost.
• No vendor lock-in or lock-out – since OSS is in the public domain it eliminates the vendor lock-in and also provides multiple vendor support i.e. Linux
• Reduced cost – using OSS increases the competition for professional services which results in lower cost for support contracts.
• Increased quality – with increased usage of the OSS more reviews of the source code are conducted. Also the community users of the software have a tendency to be more critical in the review of the software.  Besides the review by the community the software also goes through additional reviews by the companies supporting the software. The result is more quality control for most OSS than proprietary products.
• Collaborative environment – by participating in OSS communities it allows governments to input their requirements for the software. In other words they have the ability to determine the strategic roadmap for the OSS.

Over 80% of governments and businesses globally have adopted open source solutions for their IT infrastructures, but yet the U.S. government is not in the forefront like other countries such as the U.K. The U.S. government is starting to increase the use of OSS under the Obama administration and also due to the economy times. The U.K. government has made is very clear on its policy “Open Source, Open Standards and Re–Use: Government Action Plan” (http://webarchive.nationalarchives.gov.uk/+/http://www.cabinetoffice.gov.uk/cio/transformational_government/open_source.aspx) of the use of open source and standards by the below commitments it has made to the U.K. citizens and Free Software Developers:

• Ensure that the Government adopts open standards and uses these to communicate with the citizens and businesses that have adopted open source solutions
• Ensure that open source solutions are considered properly and, where they deliver best value for money (taking into account other advantages, such as re-use and flexibility) are selected for Government business solutions. Strengthen the skills, experience and capabilities within Government and in its suppliers to use open source to greatest advantage.
• Embed an open source culture of sharing, re-use and collaborative development across Government and its suppliers, building on the re-use policies and processes already agreed within the CIO Council, and in doing so seek to stimulate innovation, reduce cost and risk, and improve speed to market.
• Ensure that there are no procedural barriers to the adoption of open source products within government, paying particular regard to the different business models and supply chain relationships involved.
• Ensure that systems integrators and proprietary software suppliers demonstrate the same flexibility and ability to re-use their solutions and products as is inherent in open source.

The migration from proprietary solutions to open source solutions will not be easy, but in the long run the benefits to governments outweigh the risks for not migrating to open source solutions. It is never good for any government, business or individual to be controlled by another. Hopefully our government has learned its lesson by allowing the oil industry for some many years to control the source of energy to the America people and will not take us as long to find alternate open source solutions to the dictatorially proprietary solutions.

Reasons for Businesses

As open source evolves more and more businesses are realizing the benefits and cost savings for using open source solutions. The main reason most of the CIOs and CTOs are making the change to open source is the total cost of ownership is reduced drastically with OSS. Even though open source solutions usually do come with the cost for services and additional product layers they are still cheaper than purchasing the proprietary solutions and also eliminates the vendor lock-in.  Companies are also realizing the cost of their proprietary solutions is becoming too much in comparison to what they are losing in innovation to their open source competitors.

In a study (http://news.cnet.com/8301-13505_3-10223005-16.htm) conducted by Gartner and reported by Matt Assay at CNET, CIOs reported they have increased investment in open-source software and decreased investment in proprietary software. CIOs reported that by investing in open source they were able to do the following:

• Reduce costs by 87% (while meeting or exceeding expectations)
• Improve quality by 92%
• Ease integration and customization by 86%
• Quicken pace of innovation by 82%
• Improve support by 84%
• Increase standards compliance by 91%
• Decrease time to market by 82%

The basic benefit in using open source is the freedom to control one’s experience when using software. For instance if an end user has a problem with the software and doesn’t feel they received an timely fix to the problem they can easily switch to another open source application that meets their requirements.

Benefits of Using Standards with Open Source

One of the early drawbacks for governments and companies in not adopting open source was the fear of intellectual property (IP) infringements by open source solutions. With the use of standards in the development of open source software IP fears have almost totally diminished by governments and businesses. Standards takes away the fear of IP infringement due to them being developed by multiple vendors and going through IP reviews by both the standards organizations and the public. In addition to totally reducing IP issues developing open source based on standards also provides the following benefits:

• Interoperability – no vendor lock-in
• Enables more choices and competition for end users when selecting products
• Decreases time to market for products
• Increases of Ease of Use
  • Establishes common terms for  common capabilities across multiple vendors
• Reduces costs for developing products
  • By reducing number of development teams for each proprietary interface
• Increases innovation
• Vendor Extensions

The benefits of combining the use of open source and standards together in developing IT solutions for government  and businesses is too great to be ignored anymore. It is never easy to change the way things have always been done, but when one weighs the pros for making the change it is obvious that open source combined with standards is the way to move forward.

A Perspective on Standards

“It is relatively easy to sit alone and develop a proprietary solution that fits exactly what a single company wants to achieve. It’s much harder to develop a standard that suits every voting member. Arguments, blockages, concessions and complications are all part of this process.

Does that mean the multi-vendor standardization process is flawed? Not at all! In fact, I would argue just the opposite. A standard that has the approval of all interested parties in that space has a far greater chance of acceptance and survival than does the rogue standard created by a single, uncooperative vendor.”

By Michi Henning, Managing Director, Triodia Technologies

Sample Solution

With all of the recent hype around cloud computing and storage let’s examine how the standards Open Cloud Computing Interface (OCCI) by the Open Grid Forum (OGF) and Cloud Data Management Interface (CDMI) Storage Network Industry Association (SNIA) can be used to develop a open source standards based cloud storage solution. The OGF and SNIA cloud technical working groups have been collaborating together since the beginning to work on each of their standards to ensure the two standards integrate. By using both of these standards together it provides the capabilities to control both the management and data paths for cloud services and products. For more information on how OCCI and CDMI integrate together please see the whitepaper on SNIA’s Cloud Storage Initiative webpage at http://www.snia.org/cloud/. The diagram below illustrates how the OCCI and CDMI integrate together.

Not only can a cloud product or service implement both OCCI and CDMI it can also use open source software and along with other standards such as SMI-S and eXtensible access method (XAM). For instance one could develop a product using JAX-RS for the web services to accessing the cloud storage resources through CDMI, control the data management path with XAM and use SMI-S for configuring and controlling the backend storage on the storage devices.  The below diagram is an overview of a product that could be developed using JAX-RS an open source software) along with the SNIA standards CDMI, XAM and SMI-S to manage cloud storage services.

Of course there are a number of combinations of open source solutions and standards based products to chose to meet the end user’s IT requirements so the end user must do diligence in choosing the best solution for their organization.  The next section will provide some helpful hints when making the choices between the many solutions available between open source, standards and proprietary products.

Questions to Ask Vendors

With all of the benefits for using open source and standards one wonders why many companies have not opted to migrating to them and also why some industries still are not fully adopting standards in their company product lines. For example the auto industry has been using standards for years in order to ensure that one could operate a vehicle with one pane of glass, but yet in the storage industry it is almost totally impossible for a system administrator to have one pane of glass to monitor and manage his storage network. Each device in his network has its own element manager that would be like having a separate manager for valves, spark plugs, radiator, tires, etc. Can you imagine how big one’s vehicle console would be, well this what system administrators’ of storage networks face today.

Now how can an end user determine if a company is really implementing standards or just using the marketing hype of the standard to sale his proprietary product. The below are some questions and things to look for in determining if your feed a bunch of hot air or vendor is for real when comes to using standards.

• Ask your vendor why they are still developing to multiple proprietary interfaces when they could be saving money and time by developing to a standard interface
• Ask why they belong to standard organizations but don’t implement or promote standard based product lines
• Members of the standard organizations and actively engaged in the technical working groups working on the standards
  • Such as in the SNIA they should also be authors of the standards being developed i.e. CDMI
    • Can verify by looking at the authors listed in the specifications at http://www.snia.org/tech_activities/publicreview/
• Actively participate in interoperability testing for the standards
  • Vendors equipment in interoperability labs is available for testing more than 75% of the time
    • Ask for uptime report of vendor’s equipment in labs i.e. SNIA SMI-Lab
• Do conformance testing and test more than minimum capabilities for passing conformance testing for compliance to standards such as SMI-S
  • Compare testing done by storage vendors for SMI-S at http://www.snia.org/forums/smi/tech_programs/ctp/conformingproviders

Conclusion

As an end user how can you cause the acceleration of the movement from proprietary to open source standard based solutions? The fact is as the end user you have the power to force the change and below are some of the ways to do it.

• Require standards in RFQs and RFPs
• Verify vendors are implementing standards (don’t take the vendors word for it)
  • Don’t allow your company to be locked in by one vendor with proprietary solutions
• Get involved in standard organizations and open source projects
• Provide feedback to companies implementing open source/standard based products.
  • The result will be you driving the product roadmaps for the companies and the standard organizations.

• The number one way to change the situation and give the power back to you as the customer is to vote with your wallet.

• Only buy products compliant to standards and using open source.

Olocity has been doing quite a bit of end-user listening, research and development in the past few months with our StorageIM product, specifically performance monitoring capabilities. We found that end-users did not have the performance visibility they required to make suitable decisions regarding their hardware and fabric.

The challenge from our perspective is determining from a heterogeneous point of view the following:

• What vendors support standards and to what extent
  • SNIA SMI-S
  • Web Services
• If SMI-S is supported, what version of SMI-S does that vendor and hardware models support?
• If SMI-S is supported, what vendors support statistics and in what SMI-S profiles do they reside?
  • What vendors support the SMI-S Block Server Performance (BSP)?
  • What ‘specific’ vendor’s hardware supports the BSP capabilities?
  • What ‘specific’ BSP metrics are available for each vendor’s hardware model (e.g. Top Level Computer System, Component Computer Systems, Ports, Disks and Volumes)?
• How does one determine the above and test ones development against each and every hardware platform?

What has become clearly apparent after years of working with standards is that unfortunately most end-users do not recognize the SNIA SMI-S standards and how it should assist them in determining their hardware and aid in avoiding vendor lock-in. From our perspective during the development of these core metrics it also became apparent what vendors are invested in SMI-S and what vendors really have not sunk their teeth into standards or have rejected standards completely.

Below list what we have chosen as our 1.0 performance metric based on our test customer requirements:

Storage Array

• Support for component computer system port statistics
  • IOPS
  • Kbytes Transferred / sec.
• Support for volume statistics
  • IOPS
  • Kbytes Transferred / sec.
• Support for disk statistics
  • IOPS
  • Kbytes Transferred / sec.

Switch

• FC Port Statistics
  • Bytes Transmitted
  • Bytes Received
  • Packets Transmitted
  • Packets Received
  • Link Failures

Results

The below screenshots are sample StorageIM performance charts of a Symmetrix array and QLogic Switch utilizing the SNIA SMI-S. In my opinion EMC has done a superb job constantly extending their SMI-S implementations and ensuring that as their product lines grow their implementation of standards grow alongside. The primary switch manufacturers have also made sure their product lines implement standards as their product lines grow.

We’ve also had the opportunity to implement these new features at sites around the globe. We were actually quite amazed at some of the feedback.  Below is the feedback we received from one of our test customers. We’ve taken out the name of the storage vendor and their product to protect the innocent.

“I really like the improvements on the hardware stats.  The numerical values for the IOPs and KB/sec is exactly what we’ve been looking for.  I’ve already been able to identify our busiest LUN, which we were not able to do before given the tools we had.”

What is becoming quite obvious is the lack of visibility of performance metrics for storage administrators.

I will also note that quote above also mentioned lack of SMI-S performance statistics for the storage vendors Disk Drives. This is a definite issue with the user citing a need and use case, “Last week, we had so many disks fail that it didn’t rebuild one of the arrays, and while we were noticing degraded performance,  we had no indication of why until the storage vendor’s technician showed up with disks in hand…”

If you are interested in testing StorageIM we would be more than happy to give you a 60 day evaluation.

Screenshots

StorageIM Storage Processor Port Statistics

StorageIM Disk and Volume Statistics

StorageIM Switch Statistics

The first Storage Networking Industry Association (SNIA) Joint Cloud Lab with the Open Grid Forum (OGF) was held from April 28 to 30, 2010 at the SNIA Technology Center in Colorado Springs, CO. The SNIA and OGF are working on the development of the reference implementations for the SNIA Architecture Cloud Data Management Interface (CDMI) and OGF Open Cloud Computing Interface (OCCI) specifications in preparation for their upcoming joint cloud demonstration at the OGF 29/HPDC conference in Chicago, IL.  

During the Joint Cloud Lab Olocity and Xiotech demonstrated the use of CDMI with Representational State Transfer (REST) web services for accessing and controlling storage resources through the cloud with Olocity’s StorageIM and Xiotech’s Intelligent Storage Element (ISE) and CorteX (RESTful Web Services) products.  The demonstration illustrated how products using industry standards and open source can easily interoperate together to provide cloud services. The below diagram is an overview of the demonstration.

Upcoming Demonstration Events:

Don’t miss seeing Olocity and Xiotech demonstrating real product implementations of  CDMI and REST for providing cloud storage resources and services at the upcoming events below:

• OGF 29 http://www.ogf.org/OGF29 and HPDC http://hpdc2010.eecs.northwestern.edu conferences in Chicago, IL from June 20 to 24, 2010
• FLGISA 2010 Annual Conference http://www.flgisa.org/Conferences.aspx in Clearwater, FL from July 20 to 23, 2010
• SNIA Storage Developers Conference http://www.snia.org/events/storage-developer2010 in Santa Clara, CA from September 20 to 23, 2010

Our team has been dealing with storage and storage standards for quite a while and took on the Cortex challenge. Could we actually develop, manage and monitor a Xiotech ISE using a proprietary API and do it quickly? The challenge was incorporating Xiotech’s cortex REST implementation within our systems management application, StorageIM. Could we perform this development with efficiency, a rapid application development paradigm?  How would Cortex perform?

For the books, I’m a advocate in storage standards. I’m also an advocate for vendors thinking outside the box and that is exactly what Cortex is about. Cortex, Xiotech’s ‘Embedded’ RESTful storage API for their Intelligent Storage Elements.

For starters Xiotech chose a RESTful web service that has implemented XML as its current mime type and utilizes HTTP methods for monitoring, provisioning and indications. The URI’s implemented for capturing information about the ISE are similar to the SMI-S view concept.  As an example, the cortex ‘query’ URI call (localhost/query) looks familiar to the SMI-S Multiple Computer System Profile. The difference is you don’t have to create any association traversals.

Cortex also has a set of XSD’s that we used to dynamically create our client object model. These definitions made it a snap to create a normalized DB schema that performs and scales nicely for trending and utilization.

All of the above being said I also never touched the ISE element manager during development. I provisioned everything through simple http URI posts and would check to ensure endpoints were created, volumes were created through simple http URI gets.

During and after implementing our initial ISE aggregated monitoring solution a question kept creeping up, why are storage standards so complex?  Xiotech has completely removed any complexity.

In order to better utilize cloud computing and its resources, the SNIA Cloud Storage TWG is in the process of defining a Cloud Data Management Interface Specification (CDMI) in which the objective is to expose standardized methodologies using a RESTful architecture and leverage existing standards such as SMI-S and XAM to manage capabilities of storage, objects and metadata. SNIA has also announced the public review of the draft CDMI specification, http://www.snia.org/events/storage-developer2009.

The overall goal of Olocity is to initially create a Cloud Data Management Interface (CDMI) reference implementation that will coexist with the open source StorageIM™ storage management application and leverage existing standards such as SMI-S. This reference implementation will utilize the SMI-S standards to assemble containers for primordial pools, concrete pools and storage volumes as well as define reference CDMI capabilities.

Technology and Documentation

Cloud Data Management Interface Specification: http://www.snia.org/tech_activities/publicreview

StorageIM: Open source monitoring and reporting tool for storage systems and storage networks built around SMI-S, DASH and SMASH standards.
http://www.olocity.com

SMI-S: SMI is a membership-based initiative created by the SNIA to develop and standardize interoperable storage management technologies and promote them to the storage, networking and end-user communities.
http://www.snia.org

Jersey (JAX-RS): Jersey is an open source, JAX-RS(JSR 311) production quality implementation for building RESTful web services.
https://jersey.dev.java.net/

Read the document and results

Download /View the StorageIM CDMI Reference Implmentation Code

Building Adobe Flex applications using Ant would seem like a no brainer.  It appears the Ant builds using Flex has issues with understanding image paths. My CSS and any image embeds for our flex application StorageIM needed to have the image paths modifed in order for our Ant flex build to work appropriately. Below are modified image paths from a CSS and mxml file.

1. CSS Example: icon:  Embed(source=”../../assets/images/icon_close_up.png”);

2. mxml image: folderOpenIcon=”@Embed(source=’/assets/images/icons/folder-with-file-16×16.png’)”

Hope this helps with your flex ant builds.

I’ve been getting some end-user emails lately regarding StorageIM and the inclusion of performance monitoring. Because our road map is based on end-user feedback, ask and receive. These emails however were quite telling as to where storage monitoring is in general.

So the bad news is some end-users have serious issues with their storage vendors from a performance monitoring perspective. The key issue I’ve been hearing about lately is performance metrics. These issues consisted of what do the performance metrics truly mean and how are the performance metrics implemented such that you can can find an ‘immediate’ solution.

The good news is SMI-S supports performance metrics via their Block Server Performance Profile. We are now in the process of implementing these metrics. I will write a follow-up, Part 2 on post development and whether the SMI-S vendor implementations create a valuable solution based on end-user feedback.

Welcome to the Olocity Blog. We are an open source software and professional services company that is primarily involved in open source software and standards primarily pertaining to SMI-S and XAM. We started by developing an open source product StorageIM back in late 2005. During the inception of this project, the founders of Olocity were brainstorming on ways to collectively bring product and services that would be beneficial to our clients and software end-users.