Key Figures for Information Security

The aim of a key performance indicator (KPI) system is to make information security measurable. KPIs are therefore an important aspect of measuring the efficiency and effectiveness of IT security measures. To do this, data from various areas of information security must be collected and aggregated. This collection can be carried out using a SIEM, for example, or processed using specialized tools such as Ampeg Security Lighthouse or simple Excel spreadsheets.

Another topic is the informative value of key figures. For example, is the number of emails with malicious code blocked on the mail gateway a meaningful KPI for virus protection or primarily an indicator of the number of employees or the activities of cyber criminals? Good KPIs are meaningful but easy to determine. Bad KPIs are expensive and difficult to obtain and are not very meaningful or even misleading.

Standards for Security Key Figures

In recent years, several standards have therefore emerged that deal with security metrics in IT. In practice, three standards have proven to be relevant.

ISO/IEC 27004

ISO/IEC 27004 is a standard of the ISO/IEC 27000 family. The aim of the standard is to help institutions measure the effectiveness of their ISMS and the associated controls and thus systematically improve them. ISO/IEC 27004 requires a sound understanding and correct assessment of information security risks for the development of KPIs. The KPI system should provide reliable information about the risks in information security and the status of the implemented ISMS in order to be able to assess the risks. By collecting security KPIs, the current level of information security should be made tangible and presentable. Furthermore, compliance with and achievement of information security objectives should be checked and any need for action identified. The measurement process developed for this purpose and described by ISO/IEC 27004 includes defining the key figures and collecting the data required to assess the effectiveness of the ISMS and the implemented controls.

ISO 27001:2013 specifies some example indicators in Annex B:

B.1.1: ISMS-trained Personnel
B.1.2: Information Security Training
B.1.3: Information Security Awareness Policy Compliance
B.2.1: Password Quality – manual
B.2.2: Password Quality – automated
B.3: ISMS Review Process
B.4.1: Information Security Incident Management Effectiveness
B.4.2: Corrective Action Implementation
B.5: Frequency of Management Review
B.6: Protection Against Malicious Software
B.7: Physical Entry Control with Access Cards
B.8: Log Files Review
B.9: Management of Periodic Maintenance
B.10: Security in Third Party Agreements

Cobit

The use of a suitable control instrument for IT activities is becoming increasingly important, as IT support for business processes is becoming ever more significant and critical. For this reason, Cobit ("Control Objectives for Information and Related Technology") was developed as a control model for the entire IT. Cobit is developed by the IT Governance Institute (ITGI) and was originally developed by the Information Systems Audit and Control Association (ISACA). In contrast to ISO 27004, Cobit considers all IT processes.

Objectives and KPIs are defined on three levels in Cobit

IT goals and KPIs, which define what a company expects from IT
Process objectives and KPIs that define what a process must deliver to support IT objectives
Activity objectives and KPIs, which define what should happen within a process to achieve the required performance and how this performance is measured

Two measures are defined for this purpose

Outcome Measure (formerly KPI, Key Performance Indicator): Define metrics that show management whether IT functions, processes, or activities have met their objectives
Performance Indicator (formerly KGI, Key Goal Indicator): Define metrics that determine how well IT functions or processes are performing in terms of supporting goal achievement

According to Cobit, effective key figures should have the following characteristics:

There should be a balance of effort and informative value
The key figures should be internally comparable (numerical values)
The key figures should be externally comparable (regardless of company size or industry)
It is better to have a few good key figures than a long list of bad key figures
The key figures should be easy to measure

Compared to ISO 27004, Cobit is more complex as it covers a larger area. However, Cobit is also less suitable as a KPI system for evaluating IT security. However, the Cobit system could be helpful in defining security KPIs.

NIST Special Publication 800-55

The National Institute of Standards and Technology (NIST), or more precisely its Computer Security Division, has published SP 800-55 "Performance Measurement Guide for Information Security" as part of its "Measurements for Information Security" initiative.

NIST SP 800-55 is a guide for the development, selection and implementation of measures in IT. Chapter three "Information Security Measures Background" describes general basics about measures. Chapter five "Measures Development Process" describes the development of measures in seven phases. The sixth chapter deals with the implementation of the measures. The implementation consists of several phases to ensure the continuous application of these measures, which are used to monitor and improve IT security. The appendix to the standard contains a series of sample KPIs in the form of KPI profiles.

The following seven phases are defined for the development of KPIs:

Stakeholder Interest Identification - In the first phase, the relevant stakeholders are identified. Stakeholders should be involved in all subsequent phases to gain support across the organization.
Goals and Objectives Definition - In this phase, the security objectives to be measured are identified and documented. These objectives could be determined by legal and regulatory requirements or by the organization's objectives.
Information Security Policies, Guidelines, and Procedures Review - The third phase focuses on organization-specific information security practices. The information security policy is used to determine what metrics and key performance indicators should be established for compliance.
Information Security Program Implementation Review - In the fourth phase, all existing metrics and data sources are reviewed to determine if they are suitable for use or derivation of metrics. Data sources may change over time.
Measures Development and Selection - The fifth to seventh phase deals with the development of the metrics. NIST SP 800-53 can be used here.
- Measures Development Approach - The aim is to achieve a broad approach. The entire safety program should be measured as well as individual safety targets.
- Measures Prioritization and Selection - Selection and evaluation of suitable measured values.
- Establishing Performance Targets - Definition of targets to be achieved.
Program Results - Measurement of efficiency and effectiveness. The objectives defined in the first phase should be achieved in the best possible way.
Business / Mission Impact - This check is intended to ensure that company values are secured and losses are limited.

NIST KPI, © NESEC GmbH — Figure: Illustration of the seven phases of key figure development according to NIST

In order to be able to measure at all, security policies and procedures must exist that can then be evaluated. Another prerequisite is meaningful key figures that are as easy as possible to collect and that are integrated into an improvement process.

Quality of Key Figures

Key figures can be good or bad. Good key figures are meaningful and easy to determine. Bad key figures are expensive and difficult to obtain and are not very meaningful or even misleading. According to Sabine Kammerhofer, the following criteria can be used to evaluate key figures.

Simple: The key figure should be easy and quick to determine. A lot of information can be obtained automatically by a SIEM or from reports, for example. In principle, the simpler the data for a key figure is determined, the more efficient these key figures are for an institution.
Meaningful: The meaningfulness describes whether useful information can be obtained from the key figure. Information that serves to improve IT security is important.
Easy to understand: The KPI should have a recognizable meaning and be clearly understandable. The target group of the KPI must be taken into account, as they must understand and correctly interpret the KPI.
Repeatable: It should be possible to repeat the data collection at regular intervals. This is the only way to detect deviations over longer periods of time and to track the impact of measures.
Verifiable: A regular review of the key figures must be carried out to ensure that they are up to date.
Comparable: Many key figures can only be meaningful in relation to each other. For this reason, it is important that the same numerical values or data source are used when collecting the key figures. Within an organization, it is primarily the comparability over time and not the comparability with other companies that is relevant.
Timely: For a meaningful KPI, only current data should be determined so that any deviations can be recognized at an early stage and responded to as quickly as possible.
Reliable: No erroneous or incorrect data should be determined for the key figures. For this reason, it is advantageous if data is collected automatically.
Concretizable: It should be possible to adapt the KPIs to the business objectives, the size of the company, the maturity level of the IT and the type of company so that an individual KPI system can be developed.
Objective: It should be possible to collect key figures objectively, as subjective aspects of a key figure carry the risk of being influenced. This may falsify the statement of a key figure.
Documentable: The results of the measurement as well as general information on the measurement and presentation of a key figure, such as the calculation method or the form of presentation, should be documented.

These parameters can be presented in a so-called key figure profile, for example. This key figure profile contains all the information required to determine, calculate and evaluate a specific key figure.

Key Figure Profile

A key figure profile can have the following content:

Example of a Key Figure Pofile, © NESEC GmbH — Figure: Graphical representation of a key figure profile with various fields

Our Service

We have experience in identifying meaningful and useful key figures for determining information security. We analyze your infrastructure and identify useful key figures for you. We also help you to automate the necessary data collection and correlation of required information.

We are also happy to help you with the graphical preparation and presentation of your key figures for your management.