MacOS CPU Performance Issues
Summary: This article goes over changes that can be implemented through the Cyberhaven console to help remedy the issue of CPU spiking in Mac devices. This is commonly reported where the users are developers and are conducting development related tasks. Question_md: What remediation steps can Cyberhaven and/or the customer implement to improve CPU performance on Mac OS devices? What steps do I need to take to implement these changes?
Answer_md: We have seen an uptick of cases indicating CPU performance issues with Mac OS devices. While there are some improvements that address the issue down the pipeline in regards to the performance issues with blocking policies on Mac devices, we have a couple things we can do in the meantime to help alleviate the issue. The two approaches we 've seen with successful results are removing hosts from the blocking scope as well as setting exclusions for directories with high event volume.
Removing hosts from blocking scope
This approach is the more straight forward approach of the two. The justification for this is as follows:
" Enabling blocking policies on the endpoint affects both the system-wide I/O performance (for all macOS services) and process-wide (for specific applications). This is a limitation of the technology used for the blocking, as starting from macOS 11, the only possible way to block something (without hacking) is to use the macOS built-in Endpoint Security framework.
The framework allows every file operation (like file-open, file-create, etc.) to be checked and blocked if necessary.
The I/O performance degradation may include the reduced rate of file I/O operations per second and the lag (delay) for every file-based operation.
Cyberhaven teams work constantly to reduce both the number of irrelevant processes that should not be blocked by default and the delay for blocking resolution, which improves the end-user experience in general. Still, it all comes down to policies in effect. We currently do not recommend blocking policies for developers ' endpoints because of that."
After it is verified that the hosts effected are in the blocking scope, we can set a condition in blocking policies such as:
or
Depending on the amount of hosts and blocking policies, a list might be the preferable method. This will effectively exempt the developers from the blocking policies. This decision will ultimately need to be decided on by the customer, but we can suggest this approach and walk them through implementing it. Customers will have to weigh the impact of removing the blocking policy and assess it to the impact of the performance issues for the users.
(Note: Excluding users from the blocking scope does not work. The method to excluding users should be accomplished by explicitly specifying hostnames)
Setting exclusions for directories with high event volume
In order to minimize the negative impact on the endpoint's performance, we can mute Endpoint Security events from macOS system processes (unless it's not required for some built-in functionality, like copy/move tracking/blocking). This approach will require us to be more thoughtful to ensure we mute events that are impacting performance.
One method to gather the loudest applications and file paths is through the 'High Event Volume Applications Exclusions Exploration' report. This report is not a pre-populated report so it may need to be loaded onto customer environments (see instructions at the bottom for importing this report). Once the data populates, you will be able to analyze what applications and paths are triggering the most events in a specified time range.
(Suggestion is to analyze the report and get a listing of loudest applications and paths. Relay this information back to the customer, and if the customer approves the list of applications/paths to exclude then add them into the remote configuration.)
Another method to gather exclusion paths is through the Cyberhaven console. Customers will typically report that a specific set of users are experiencing the CPU spike, and specific applications the users are working in where the users experience the performance issues. Try to gather as much info as you can such as users, applications, tasks, and time stamps to help narrow down the events. Once you narrow down the events, you will be able to pull a list of paths to exclude through investigating the events and corresponding destination paths.
For example, user states they are getting reports of CPU spikes when using the 'jcmd' tool. Try to get a list of users effected and filter by users and hostname and filter by application. You should see the the number of events in that timeframe for 'jcmd' such as:
We see 779k events in the specified time frame. After investigating the events and gathering information on the directory paths through the events tab, I see the destination path for the application are regularly found in '/User/$Username/.vscode/extensions/'. I can verify this is a path to exclude by setting a condition such as in the same search:
As we see, no events are populated when filtering events for that path. We
commonly exclude paths that are in users' home directories, so you will have
to verify the paths are correctly set under path_prefixes or
user_dir_path_prefixes in the remote configuration settings and both of
these would be under the ignored_apps property. So for the above example,
you would place the '/.vscode/extensions/' path under user_dir_path_prefixes.
To sum this up, if it is found that destination file paths are based out of
the users' home directories then they would be added under
user_dir_path_prefixes as this assumes the file path starts off as
'/User/$Username/....'
If the destination file paths are based out of other system directories such
as '/Application/...' or '/Library/...', they should be placed under
path_prefixes in the remote config.
Here are some commonly used paths for loud applications we have observed for developers:
"/Applications/Docker.app/"
"/Applications/Visual Studio Code.app/",
"/Applications/Xcode.app"/,
"/Applications/WebStorm.app/",
"/Applications/Sourcetree.app/",
"/Applications/IntelliJ IDEA.app/",
"/Applications/IntelliJ IDEA Ultimate.app/",
"/Applications/PyCharm CE.app/",
"/Applications/PyCharm.app/",
"/Applications/Qt Creator.app"
``
``
``
``
``
Importing the 'High Event Volume Applications Exclusions Exploration'
report
You will need the .zip file that will automatically build the report when utilizing the import tool. That file is attached in the 'Related' tab of this knowledge base. To import the report on the Cyberhaven console:
Navigate to 'Visual Analytics' > click on 'Import dashboards' button towards top right of screen > Select file > Navigate to file and select > Import
Report will be automatically built after this. If customer wants to view this report as well, you will have to publish the report by simply toggling the 'Draft' button to 'Published':