Manual vs Automated Controls

Overview

The difference between manual and automated controls completed by the service organization is pretty self-explanatory. Manual controls rely on a human action (e.g., review, signature), while automated controls are system-driven (e.g., configurations, rules), with a human only getting involved if/when there’s an error notification.

Using control 1.02 as an example, a manual control would be the hiring manager and application/data owner approval signature. Evidence of approval can be stored in a system (e.g., electronic signature in a service desk ticket), but it still requires a person to perform the action of approving access.

A simple to understand automated control would be user password settings. The settings are configured at the network or application level and apply to all users. Using control 1.03 as a more complicated example, an automated control would be disabling employee access on their last day of employment through integrations built into the Human Resources Information System (HRIS) and relevant applications/systems.

It’s also possible for a control to be both automated and manual if there are multiple aspects to the control. Continuing with control 1.03, additional steps might be needed to disable access for systems not integrated with the HRIS (e.g., facility access system).

Aside: I try to use separation instead of termination nowadays, but they are interchangeable.

Example SOC 1 control matrices for an access control objective. This is not a fully fleshed out objective and is just being used for illustrative purposes.

Testing

The type and extent of testing performed can be drastically different depending on if the control is automated or manual. Prior to testing, the service auditor will inquire with the control owner(s) to obtain an overview of the control, determine how the control is performed, and ascertain what evidence is available for inspection.

Continuing with control 1.02 as a manual control example, the service auditor will request a population of personnel hired during the period and select a sample from the population for testing. The larger the population, the larger the sample size selected for testing. There are upper limits to samples sizes, but that’s a topic for another day. The service auditor will rely on spreadsheets, exports from the ticketing system, and/or screenshots as evidence for testing to ensure approvals were obtained prior to granting system access.

Continuing with control 1.03 as an automated control example, the service auditor will still request a population of personnel separated during the period. However, since the control is automated, the service auditor will inspect automation criteria (system configuration, rules) and system logs/records for a sample of 1-2 transactions during the period. A full sample from the population based on sample selection guidelines isn’t generally required.

It’s important to keep in mind when implementing AI into the internal control environment, the system should still log how and/or why decisions were made. From an internal controls and audit perspective, there needs to be evidence available to prove a control was working as expected and was effective. If there’s no proof, then it didn’t happen.

SOC Planning – Control Design

System and Organization Controls (SOC) control design involves documenting policies, procedures, and technical safeguards to align with the specific framework (e.g., SOC 1 control objectives, SOC 2 Trust Service Criteria). Last week I posted an example SOC 1 control objective with 4 supporting controls (below) with a caption indicating it’s not a fully fleshed out control objective and is just being used for illustrative purposes. With that in mind, let’s take a look at some topics to enhance the controls supporting the objective with some example controls.

Topic #1: Hiring Process

The hiring process typically includes various controls that need to be completed prior to an employee being granted access. These controls can be included to provide a beginning-to-end employee lifecycle, but would most likely be assessed moderate or low control significance during the controls risk assessment.

Employees are required to pass a general background check prior to their first day of employment.
Employees are required to complete security awareness training on their first day of employment and annually thereafter.
Employees are provided the employee handbook upon hire, which documents the company code of conduct and authorized use of computer resources, that they are required to sign as an acknowledgment they read the handbook.

Topic 2: Split Access Controls

The process for granting an employee general network access at hire is different than the access required for specific job functions. For example, a payroll analyst would require role-based access to the ERP (Enterprise Resource Planning) system while a system administrator would require privileged access to 1 or more systems. These controls would be assessed high control significance during the controls risk assessment.

Application-level permissions are approved by the application/data owner and assigned based on custom configured user groups based on employee role to ensure permissions promote appropriate separation of duties.
Administrator-level permissions for network infrastructure (e.g, servers, firewalls, etc.) is approved by the CISO and is restricted to personnel who require elevated access to perform administrative job functions.
Access for administrative roles requires multi-factor authentication (MFA).

Topic 3: Authentication Controls

Authentication controls include minimum password length, password complexity, and account lock-out mechanisms for when unsuccessful attempts to access an account reach certain thresholds. The account lockout would include maximum attempts before getting locked out and a lockout duration, which lowers the risk of attackers guessing passwords to access the system. Authentication controls would be assessed a moderate or high control significance during the controls risk assessment depending on how the control objective is phrased.

Operating system, application, and database passwords are configured for strong user authentication parameters, including password length, password complexity, user account lockout on invalid attempts, and password change intervals.
User sessions are automatically logged out after 30 minutes of inactivity.

The option exists to be more specific regarding password controls. For example, the minimum password length can be listed (e.g., 10). However, by leaving it a little more generic, the service organization can point to the policy for the auditor to use as a baseline for testing. Also, depending on how systems are integrated, password controls might be different for all systems/applications or the same when integrated with single sign-on tools.

Topic 4: Monitoring Controls

Controls to monitor user access can be split between general network, applications (with role/permission review), and privileged access (administrative) permissions, with different frequencies that align with their risk. For example, a general review of access to the network to ensure no separated employees continue to have access might be completed annually while an administrator access review might be completed monthly. Monitoring controls would be assessed a moderate control significance during the controls risk assessment.

System administrators review network user access reports annually to verify access for separated employees was disabled and/or removed.
ERP module owners review user access reports monthly to verify access is current and appropriate based on the users role within the system.

As you can see, this can result in 10+ controls supporting a single control objective, which opens up the option of splitting the control objective so it’s more focused. For example, a control objective can be added that focuses on the hiring process or the control objective can be split to focus between general network access and administrative access to network infrastructure (e.g. servers, firewalls, IDS).

The decision to split the control objective or include more controls in a single objective can depend on the size and complexity of the environment, contractual requirements, or industry standards (e.g., NIST). And, as noted in the previous post, there’s a lot of grey area that goes into these decisions that requires professional judgement on the part of the service auditor when making recommendations during a readiness assessment or planning for the examination.

SOC Planning – Controls Risk Assessment

There are various planning steps that occur at the onset of a System and Organization Controls (SOC) 1 and/or 2 engagement, most of them the client isn’t aware is happening in the background. These steps are generally included in the audit program with templates available to complete some of the steps.

The controls risk assessment is a planning step that can begin once the control matrices are received from the client. This is when the fun begins. The SOC controls risk assessment is different from what would normally be included in a NIST-based risk assessment, which looks at risks from a confidentiality, integrity, and availability perspective, and assesses those risks based on impact and likelihood.

The SOC control risk assessment analyzes the design of controls to ensure they adequately meet the control objective or trust service criteria (TSC), the significance of the controls as they related to the objective/criteria, and the risk of non-compliance for each control.

Control Design

The primary goal when reviewing the controls is to determine whether the sum of the controls adequately support the control objective or TSC.

This is a wholesale review of the controls to ensure they meet the minimum requirements of the control objective. Although not ideal, there are situations in which a single control is enough to meet the controls objective. It’s also possible for there to be 10+ controls and still miss a key aspect of the control objective.

Using the example control matrices above, if this were a very large organization, I’d inquire about controls to update access for employees that transfer from one business unit (BU) or department to another BU or department. This is a potential area for there to be a design deficiency.

There’s a lot of grey area in this review that requires professional judgement that comes from working with clients both big and small, more and less complex. The audit firm should be cautious about starting the engagement before clearing up any potential design deficiencies.

Control Significance

The control significance assessment determines the significance of the control as it relates to the control objective. Each control is assessed a rating of high, moderate, or low based on its significance. A high control significance would indicate the control has a direct correlation to the control objective. A moderate control significance would indicate a supporting control. A low control significance would indicate limited or no correlation to the control.

At a minimum, there needs to be at least 1 control with high control significance per objective/criteria. Otherwise, there would be a design deficiency.

In my experience, policies and procedures would be assessed as low control significance, assuming the control objective doesn’t directly call out for policies and procedures. This isn’t to say policies and procedures aren’t an important aspect of the overall control environment, just that they’re not directly related to meeting the objective/criteria as stated. Monitoring controls (e.g. periodic review) would be assessed as moderate control significance. These controls are detective controls rather than preventive controls.

Risk of Non-Compliance

Each control is then assessed a rating of high or low based on the risk that testing would result in an exception or finding. This assessment is pretty straightforward. Controls are typically assessed as low risk of non-compliance unless one of the following criteria are met:

First Year Engagement: All controls are assessed a high risk of non-compliance.
Subsequent Year Engagement: Control assessed a high risk of non-compliance if exceptions or findings were noted in the prior year.
Other: Although less frequent, if exceptions were noted during other internal control assessments, the control could also be assessed as a high risk of non-compliance. These might be found on a corrective action plan (CAP) reports provided during planning.

Example completed risk assessment using the access control matrices in the image above.

Why

Now, you might ask, what’s the purpose of going through this process? There are 2 primary reasons:

Completeness: Ensure the controls are complete and there are no design deficiencies that would result a qualified opinion.
Sampling: Control significance and risk of non-compliance are used as a basis for determining testing sample sizes.

The completed risk assessment is documented in the audit workpapers and isn’t usually provided to the client. However, in some situations, the client might request the results of the risk assessment in what would be referred to as a test plan. The test plan would include the risk assessment along with a tentative plan for how each control will be tested. This request is more common with government clients.

Not all firms use the same audit methodology as noted above. However, it’s important for service organizations to understand what goes into each of the steps of an engagement prior to contracting with a service auditor. The best time to do this is during the request for services and proposal process, at which point the service auditor should provide a general audit methodology used to complete the engagement.