All Articles

Summary:

Purpose: The purpose of this concept paper is to explain the requirement of a framework which is not only easy to interpret but also dynamic in understanding customer requirements when it comes to analyzing vendors. It also explains the concept of CPI (CISO Perception Index) and CRI (CISO Recommendation Index) and offer insights regarding their use as a framework for analyzing the IT Security sector.

Research: Research for this framework included a review of most of the existing frameworks along with an in-depth analysis of the IT Security sector.

Methodology: Business research methods such as Cluster analysis, graphical analysis and surveys using Likert scale have been used for collecting and analyzing the data mathematically.

Major Findings:

The major findings include -:

1. CPI (CISO Perception Index) and CRI (CISO Recommendation Index) for a vendor.
2. An overall analysis of the vendors by using CPI and CRI.
3. Carrying out in-depth analysis of the vendors based on the questions asked to the test sample.

 (Read more:  Top 5 Big Data Vulnerability Classes )

Do we need a new framework?

Yes, for following reasons:

• Existing frameworks are difficult to interpret.
• No exclusive framework to assess vendor on Product - Features, Pricing/ROI, Ease of Implementation/Use, Integration, Support, overall recommendation by CISOs and perception of CISOs.
• Some of existing frameworks take customer references from vendors to evaluate the product,at times this may lead to bias in vendor analysis.
• Not very useful in helping CISOs make buying decisions.

How can we overcome this gap?

CISO Platform took initiative to develop a framework that overcomes the gap with following attributes:

• Simple, Dynamic and Easy to interpret.
• Exclusively assess vendor on Product - Features, Pricing/ROI, Ease of Implementation/Use,Integration, Support and overall recommendation by CISOs.
• Along with customer’s references taken from vendors, other CISOs evaluation of the product has to be taken into consideration for methodology.

We call it “CISO Platform Index”, CISOs are the backbone for this framework and play very important role in evaluating the vendors.

(Read more:  How to write a great article in less than 30 mins )

Brief description about CISO Platform Index and Methodology

The CISO Platform Index uses a transparent methodology to compare the players in software,hardware, or services market so that the CISOs can make well-informed decisions. The CISO Platform Index offers two indexes to compare:

1. CISO Perception Index - Index developed based on the perception of CISOs about the vendor /Product on different evaluation metrics.
2. CISO Recommendation Index - Index developed based on the recommendation by CISOs who used the vendor product on different evaluation metrics.

Who are Participants in CISO Platform Index Process

CISO Index builds upon participation of 3 key players:

• CISO Platform Analyst - Manages the CISO Index process.
• Vendor - To provide detailed product /service information and customer references.
• CISOs - Evaluate vendor products by survey questionnaire.

Watch more : How MIT website got hacked despite having any vulnerability ?

Evaluation Method:

The following steps will be executed for evaluating the vendors-:

Step 1: Following 6 questions will be floated among CISOs and customer references provided by the vendors.

• How satisfied are you with features of this product?
• How satisfied are you with Return on Investment of this product?
• How easy is to implement this product without disturbing existing system?
• How well this product can integrate with existing system?
• How do you rate product support?
• Would you recommend this product to CISOs?

Step 2: CISOs and vendor referred customers will rate the vendor –product on likert scale of 7 for each question.

Sample Likert Scale:

Step 3: Initially, likert scale scores collected from CISO’s and vendor’s customer references will be divided into two buckets based on whether the participants have used the vendor product or not.

• The average likert score for each vendor collected from participants who haven’t used vendor product but are aware of its pros and cons will be known as CPI (CISO Perception Index).
• The average likert score for each vendor collected from participants who have a current/prior usage experience of the vendor product will be known as CRI (CISO Recommendation Index).

Vendors are encouraged to come up with more references of their customers as this will help in analyzing them with more confidence. Also, we will be providing an additional index points to the vendors based on the number of references they provide.

Mathematical Analysis

We will use clustering analysis (Agglomerative and K-mean method) for dividing out the vendors into different clusters based on: 1. CPI 2. CRI.

Following steps will be executed as part of mathematical analysis -:

1. We will carry out two kinds of analysis over them -:

• An overall analysis of the vendors by using CPI and CRI
• Carrying out in-depth analysis of the vendors based on each question asked.

2. The collected data will be initially analyzed using Agglomerative clustering .This method is promptly used for identifying the possible number of clusters present within the sample.

3. Once we have the number of clusters present within the sample, we will use this number as an input to another very popular clustering method called “K-Mean”.

4. K-Mean method helps in identifying the initial and final cluster centers within the sample based upon the questions asked.

5. K-Mean, further, helps in calculating inter-cluster distances and developing a correlation matrix between the available clusters to analyze how close these clusters are to each other.This will help in ranking these clusters.

6. The ANOVA table, which also comes as an output, tells us which of the 6 questions (variables) is/are significantly different across the clusters. Thus, it is used as a measure to identify which question bears more significance with respect to the end users.

7. In the end, we will pick the top two clusters and present it in our report. The top two clusterswill be termed as

• Champions
• Challengers

CPI Vs. CRI

Cluster Membership Table

CISO Recommendation Index for Companies  

More:  Want to become a speaker and address the security community?  Click here