October 11 2025 Saturday
Type 2 Hypervisors and the evils of "E" cores
In this post I want to give some pointers on making VirtualBox (or any Type 2 hypervisor) run “better” on CPUs that use Big-Little CPU architecture.
First and foremost, this information is provided without warranty, guarantee of correctness, or applicability to your individual situation. As always with information, YMMV (your mileage may vary). Again, any attempt you make is at your own risk. I am simply trying to help.
OK, what is Big-Little? It’s an Intel technology (also found on recent AMD CPUs) that divides CPU cores into two tiers: Big (performance or “P” cores) and Little (efficiency or “E” cores). E cores are typically used for background tasks or processes that don’t require immediate attention, such as dormant browser tabs.
My regular desktop PC has an Intel i9-13900T CPU, which features eight P cores and 16 E cores. You can see this by checking the CPU specs from your CPU manufacturer:
https://www.intel.com/content/www/us/en/products/sku/230498/intel-core-i913900t-processor-36m-cache-up-to-5-30-ghz/specifications.html
Now, Windows doesn't really tell you which is which, so in Task Manager, you can’t distinguish. But for anything virtualization, E cores ARE SIMPLY ABYSMAL. If a VM is moved to an E core, it pretty much stops responding. The moral of the story is that VMs need to run on P cores.
“That’s great news, Darren, but how does one accomplish this?” I hear you ask. There are ways. Again, do these at your own risk and YMMV. Personally, I use a program called “Process Lasso”. This lets me tag specific EXE files and processes to ONLY run on P cores and keep them off the dreaded E cores. You can also disable E core in the BIOS settings of most PCs, but that’s overkill (unless no other way works for you).
Process Lasso is available here: https://bitsum.com/
Process Lasso will actually show you the type of cores (E cores have the leaf):
Process Lasso reports I have 16 P cores, but since they are hyperthreaded, I effectively have 8. I’ll let you Google it if you really want to blow your mind, should you really want to know about hyperthreading.
If you download, install, and run Process Lasso as an administrator, you can pin specific applications or processes to specific cores. This is called processor affinity and can be seen here, where some tasks are on CPU 0-15 (P cores) and some are on CPU 0-31, which is every core on the PC, P or E, whichever the OS decides to place them on:
As you can see from the above, I used Process Lasso to tie VirtualBox (and VBoxSVC, and VBoxHeadless) to 0-15, meaning only ever run them on P cores. Once you do this, the performance difference of the running VMs is DRASTIC. I created a profile called “P-cores-only”, but you can at least test it by deselecting “E” cores like this:
Hopefully, after pinning the VirtualBox processes to P cores, your VMs now respond and freeze or crash less often. Again, YMMV.
First and foremost, this information is provided without warranty, guarantee of correctness, or applicability to your individual situation. As always with information, YMMV (your mileage may vary). Again, any attempt you make is at your own risk. I am simply trying to help.
OK, what is Big-Little? It’s an Intel technology (also found on recent AMD CPUs) that divides CPU cores into two tiers: Big (performance or “P” cores) and Little (efficiency or “E” cores). E cores are typically used for background tasks or processes that don’t require immediate attention, such as dormant browser tabs.
My regular desktop PC has an Intel i9-13900T CPU, which features eight P cores and 16 E cores. You can see this by checking the CPU specs from your CPU manufacturer:
https://www.intel.com/content/www/us/en/products/sku/230498/intel-core-i913900t-processor-36m-cache-up-to-5-30-ghz/specifications.html
Now, Windows doesn't really tell you which is which, so in Task Manager, you can’t distinguish. But for anything virtualization, E cores ARE SIMPLY ABYSMAL. If a VM is moved to an E core, it pretty much stops responding. The moral of the story is that VMs need to run on P cores.
“That’s great news, Darren, but how does one accomplish this?” I hear you ask. There are ways. Again, do these at your own risk and YMMV. Personally, I use a program called “Process Lasso”. This lets me tag specific EXE files and processes to ONLY run on P cores and keep them off the dreaded E cores. You can also disable E core in the BIOS settings of most PCs, but that’s overkill (unless no other way works for you).
Process Lasso is available here: https://bitsum.com/
Process Lasso will actually show you the type of cores (E cores have the leaf):
Process Lasso reports I have 16 P cores, but since they are hyperthreaded, I effectively have 8. I’ll let you Google it if you really want to blow your mind, should you really want to know about hyperthreading.
If you download, install, and run Process Lasso as an administrator, you can pin specific applications or processes to specific cores. This is called processor affinity and can be seen here, where some tasks are on CPU 0-15 (P cores) and some are on CPU 0-31, which is every core on the PC, P or E, whichever the OS decides to place them on:
As you can see from the above, I used Process Lasso to tie VirtualBox (and VBoxSVC, and VBoxHeadless) to 0-15, meaning only ever run them on P cores. Once you do this, the performance difference of the running VMs is DRASTIC. I created a profile called “P-cores-only”, but you can at least test it by deselecting “E” cores like this:
Hopefully, after pinning the VirtualBox processes to P cores, your VMs now respond and freeze or crash less often. Again, YMMV.
January 3 2025 Friday
Domino Router bug - seems to also affect server availability index
I was waiting for the other shoe to drop for this bug. Surely the errant code wasn't only in the router task. Well, it seems that it's NOT only the router.
After working with a customer on fail-over issues in a cluster we came across this interesting availability index "issue". On a server patched for the router bug (or that is un-patched server that has not been rebooted) the "show ai" command behaves as expected, the XF, Hits and AI min and max are populated:
However, on a rebooted, un-patched server AI is completely and utterly blank:
So, if you're clustering and using any type load balancing, or restricting of server access using AI, you may see issues on un-patched Domino servers.
After working with a customer on fail-over issues in a cluster we came across this interesting availability index "issue". On a server patched for the router bug (or that is un-patched server that has not been rebooted) the "show ai" command behaves as expected, the XF, Hits and AI min and max are populated:
However, on a rebooted, un-patched server AI is completely and utterly blank:
So, if you're clustering and using any type load balancing, or restricting of server access using AI, you may see issues on un-patched Domino servers.
March 21 2024 Thursday
OpenNTF March 2024 Webinar - Me, today, presenting Domino security
Today (March 21, 2024), although it will be available on the YouTube after, I am presenting as part of the monthly OpenNTF webinar series.
It's at 11am eastern, and details are here:
https://openntf.org/webinars
It's at 11am eastern, and details are here:
https://openntf.org/webinars
September 4 2023 Monday
Collabsphere 2023 - Great new Domino features since 9.0.1 FP8
The first in-person event for a number of years was held at the stunning Chicago Botanic Gardens. Yet another fantastic event by Richard Moy and team (and LeeAnn).
HCL's openness is also still kind of strange to me after all the barren IBM years. Now, if only they can provide public notice of customers having to run DLUA in order to get a renewal.....still, the product side is knocking the ball out the park.
So here the SlideShare link to the pres....
https://www.slideshare.net/darrenduke/great-new-domino-features-since-901fp8-2023-edpptx
HCL's openness is also still kind of strange to me after all the barren IBM years. Now, if only they can provide public notice of customers having to run DLUA in order to get a renewal.....still, the product side is knocking the ball out the park.
So here the SlideShare link to the pres....
https://www.slideshare.net/darrenduke/great-new-domino-features-since-901fp8-2023-edpptx
Darren Duke
|
September 4 2023 03:35:56 PM
|
collabsphere conference domino notes presentations
|
Comments [1]
April 16 2023 Sunday
Ransomware Prevention Part 11 - Let’s talk about Service Accounts
Part 11 - Let's talk about Service Accounts
See here for the entire series of posts, if you are just stumbling onto these posts.
As I said all the way back in part one, these post are supposed to be helpful in giving you meaningful useful advice to prevent ransomware.
Service Accounts. They exist everywhere. Most have common (and scary) attributes such as passwords that haven't changed in a few years, if not a decade or more. Service Account passwords are rarely, if ever, changed because of of the havoc the can be created when they are. And there are many organizations that have no earthly idea where or how many times these accounts are used. Finally many are local admins on server, or (queue scary music) are domain admin level accounts.
Given any of the above attributes, let alone several of those attributes being present in a single service account, it should come as no surprised that they represent an adversary hitting a gold mine if the can compromise one of them. So how does one protect these God level accounts? Read on.....
In all my years of doing this, I have yet to find an actual reason to have any service account listed as a domain admin. Zero. Nada. Ziltch. If you have a service account in Domain Admins you are simply doing your job wrong.
Managed Service Accounts
The preferred way to create, manage and use service accounts is utilize Managed Service Accounts (MSA). These MSA accounts come in two distinct flavors, stand-alone MSA accounts (sMSA aka MSA), and group MSA accounts (gMSA) and were first introduced in Windows 2008 R2 with gMSA accounts being added with 2012. The only significant difference between the two types is that a sMSA account can only ever be used on a named, single server. it cannot be "assigned for use" on two servers at the same time (note, I said servers, not services!). gMSA accounts on the other hand can get used across several names servers, so a shared account if you will.
For security reasons, sMSA accounts should always be your default choice. gMSA accounts have specific use cases, the one I see the most is using a single gMSA account on several ADFS servers in an ADFS farm.
MSA accounts in general address many, if not all, of the issues with traditional service accounts, namely:
"Darren, this sounds perfect!" Well, yes and no.
That being said, MSA accounts can and should be used anywhere and everywhere you can. For management of MSA accounts, ManageEngine have a helpful free tool available here: https://www.manageengine.com/products/free-windows-active-directory-tools/free-active-directory-service-account-management-reporting-tool.html. Microsoft has detailed MSA documentation and a quick Google will show you how to set them up.
Traditional Service Accounts
As mentioned above, you may locate services that simply won't work with MSA accounts. If you have those then you're left with the traditional service account way, which is simply a user account. These accounts, while nowhere near as secure as MSA accounts are, can have increased security.
Traditional service accounts should be prevented from interactive logins. While MSA accounts have this prevention enabled by default, traditional service accounts need to be set up for this via a GPO
Traditional service accounts need long, complex passwords. I'd look at a minimum of 32 characters.
See here for the entire series of posts, if you are just stumbling onto these posts.
As I said all the way back in part one, these post are supposed to be helpful in giving you meaningful useful advice to prevent ransomware.
Service Accounts. They exist everywhere. Most have common (and scary) attributes such as passwords that haven't changed in a few years, if not a decade or more. Service Account passwords are rarely, if ever, changed because of of the havoc the can be created when they are. And there are many organizations that have no earthly idea where or how many times these accounts are used. Finally many are local admins on server, or (queue scary music) are domain admin level accounts.
Service Account passwords are rarely, if ever, changed
Given any of the above attributes, let alone several of those attributes being present in a single service account, it should come as no surprised that they represent an adversary hitting a gold mine if the can compromise one of them. So how does one protect these God level accounts? Read on.....
In all my years of doing this, I have yet to find an actual reason to have any service account listed as a domain admin. Zero. Nada. Ziltch. If you have a service account in Domain Admins you are simply doing your job wrong.
Service Accounts should never, ever be a Domain Admin
Managed Service Accounts
The preferred way to create, manage and use service accounts is utilize Managed Service Accounts (MSA). These MSA accounts come in two distinct flavors, stand-alone MSA accounts (sMSA aka MSA), and group MSA accounts (gMSA) and were first introduced in Windows 2008 R2 with gMSA accounts being added with 2012. The only significant difference between the two types is that a sMSA account can only ever be used on a named, single server. it cannot be "assigned for use" on two servers at the same time (note, I said servers, not services!). gMSA accounts on the other hand can get used across several names servers, so a shared account if you will.
For security reasons, sMSA accounts should always be your default choice. gMSA accounts have specific use cases, the one I see the most is using a single gMSA account on several ADFS servers in an ADFS farm.
MSA accounts in general address many, if not all, of the issues with traditional service accounts, namely:
- Automatic changing of passwords by AD ever 30 days or whatever your AD machine password expiration is. NO manual intervention is needed.
- Password complexity is high, 240 bytes make brute forcing difficult
- MSA accounts have to be specifically assigned to a server before a server can use it.
- MSA accounts are prevented from interactive user logins.
"Darren, this sounds perfect!" Well, yes and no.
- Not every service you have running can use MSA accounts. It's gotten better over the years, but it's still trial and error.
- They are an absolute pain in the backside to create and manage the first time you try.
- You still need to reduce the MSA account to least privileged access.
That being said, MSA accounts can and should be used anywhere and everywhere you can. For management of MSA accounts, ManageEngine have a helpful free tool available here: https://www.manageengine.com/products/free-windows-active-directory-tools/free-active-directory-service-account-management-reporting-tool.html. Microsoft has detailed MSA documentation and a quick Google will show you how to set them up.
Traditional Service Accounts
As mentioned above, you may locate services that simply won't work with MSA accounts. If you have those then you're left with the traditional service account way, which is simply a user account. These accounts, while nowhere near as secure as MSA accounts are, can have increased security.
Traditional service accounts should be prevented from interactive logins. While MSA accounts have this prevention enabled by default, traditional service accounts need to be set up for this via a GPO
Traditional service accounts need long, complex passwords. I'd look at a minimum of 32 characters.
December 9 2022 Friday
Domino 12.0.2 - no support for Windows 2016? Really?
While talking with a customer today I was informed HCL told them Windows Server 2016 wasn't supported for Domino 12.0,2 (apparently due to some technical limitation). I thought there was no way this was correct, so off I go to HCL's support web site, and low and behold, no Windows 2016 listed as a supported OS!
From https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0101447
From https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0101447
October 21 2022 Friday
My Collabsphere 2022 presentation - Great new Domino features since 9.0.1FP8
It's available over on slideshare. Link is below.
https://www.slideshare.net/darrenduke/great-new-domino-features-since-901fp8pptx
https://www.slideshare.net/darrenduke/great-new-domino-features-since-901fp8pptx
June 30 2022 Thursday
Domino 12.0.2 adds VSS backup support
Since BackupExec ceased support for Domino backup APIs after v14 there have been very few backup utilities that integrate with Domino natively. The fix was always for IBM to add VSS support to Windows Domino installs (the vast, vast majority of installs I see *are* on Windows). But IBM (along with 1000's of other fixes they should have and could have done) choose not to.
HCL have finally fixed this oversight (and by oversight I mean complete dereliction of duty from IBM). I fully admit I was worried when HCL went all chips in and bought it all from IBM, but boy have they been adding stuff that has been sorely missing from the product. VSS support included.
The best part is that (for backup at least, restores are a tad more finicky so be sure to read the docs) there is no setup on your side once 12.0.2 ships and you install it. It is available today in FlexNet as a preview release, not gold code yet so you've been warned. Here is what happens on the Domino side (I have logging turned up) when Veeam backs up my 12.0.2 Windows server with Veeam "application aware processing" turned on:
HCL have finally fixed this oversight (and by oversight I mean complete dereliction of duty from IBM). I fully admit I was worried when HCL went all chips in and bought it all from IBM, but boy have they been adding stuff that has been sorely missing from the product. VSS support included.
The best part is that (for backup at least, restores are a tad more finicky so be sure to read the docs) there is no setup on your side once 12.0.2 ships and you install it. It is available today in FlexNet as a preview release, not gold code yet so you've been warned. Here is what happens on the Domino side (I have logging turned up) when Veeam backs up my 12.0.2 Windows server with Veeam "application aware processing" turned on:
June 9 2022 Thursday
Ransomware Prevention Part 10 - Credential Guard, the feature you didn’t know existed
Part 10 - Credential Guard, the feature you didn't know existed
See here for the entire series of posts, if you are just stumbling onto these posts.
As I said all the way back in part one, these post are supposed to be helpful in giving you meaningful useful advice to prevent ransomware.
This series is now over a year in the making.....I hope a reader or two still exists.
Certain versions of Windows have a special feature called Credential Guard. Due to Microsoft not being, well, particularly into security this feature is not present in Home nor Pro versionS of Windows. I view this a travesty, but hey Microsoft makes tons of money so why should they care. It does exist in Enterprise and Education desktop Windows and also in Windows Server since 2016. If you have looked at doing Windows 10 Enterprise before, but haven't found a killer feature, then this is it (and LTSC). If you have never looked at buying Windows 10 Enterprise, this is the feature that should make you look into it.
Not only is is not widely available, it's woefully marketed. Did you even know about this? One of the two most important tools in the hacker prevention tool chest? (the other being SRP, aka part 6 in this series)
Credential Guard does what it says on the box, it protects credentials. Specifically in-memory credentials. These are stored in such a way as to be accessible to hackers once they compromise the device ("pass the hash" is the usual name for this type of hack). If you have no idea what I'm talking about go watch this video that uses the MimiKatz tool to extract in-memory credentials (password hashes to be exact) out of thin air....
https://youtu.be/bTYR_xYSDIk
Scared yet? If you're not then you're in the wrong job. Go read a gardening blog or take up knitting.
What the above video shows is how easy it is to effectively harvest credentials from Windows OSes. Credential Guard addresses this Windows "feature". It also worth noting that some CPUs now also have this type of protection built in, specifically AMD Ryzen Pro CPUs can have a similar protection enabled in BIOS. But on the Pro line.
OK, so I have Windows 10 Enterprise, or 2019/2022 server. How do I get this level of protection? For starters, VMs are a bit different, so I'll cover those later. Second, laptops with VPN clients are different so read this all before you enable it on laptops. Even a standard desktop OS it's a lot of convoluted steps. Thankfully Microsoft do provide a PowerShell script to simplify enabling it. They way it works it also a bit convoluted. The setup even more so. The PowerShell (see two paragraphs down) is a God send.
See, the "fix" Microsoft came up with is to install a Hyper-V machine on the device in question, lock it down and encrypt it and store the credentials in that Hyper-V instance. So now you have two PCs. Kind of. If you really want to know more about how it works see here: https://docs.microsoft.com/en-us/windows/security/identity-protection/credential-guard/credential-guard-how-it-works
The PowerShell readiness/enablement script is here: https://docs.microsoft.com/en-us/windows/security/identity-protection/credential-guard/dg-readiness-tool
The above script needs to be ran in an admin PowerShell with the -capable switch, then you need to reboot and run it again with -capable. Be sure to check out the pre-reqs as well (UEFI, enable virtualization technologies in BIOS, etc.)
If the script says you can enable it, run it with -enable. A reboot and few auto reboots later, Credential Guard is installed. Note, if you use a VPN client on the device in question, chances are the VPN is not fully CG compatible, so be sure to check, If the VPN client is not CG compatible run with -enable -cg (so not just -enable, add the -cg).
Here I'm running the script with the -capable switch to see if my PC can indeed enable CG.....

I need to reboot, then I run the -capable again:

In the above screen shot I have highlighted an issue. In this case it is very likely a VPN shim driver as I'm running it on a laptop with a VPN client so I will run the "-enable -cg" flags to enable *only* CG (just "-enable", so without "-cg" does get me better security, but experience tells me it will stop my VPN client for working.)

Above, we can now see Hyper-V and IOMMU have been enabled. Time to reboot again....and then rerun the PowerShell with the -ready switch:

As you can see I now have Credential Guard enabled. The yellow warning are because I chose to *only* enable CG and not the other option as that would croak my VPN client. MimiKatz has now been taken to the vet and euthanized and the password hashes are no longer accessible to hackers. You can see this in action on this video:
https://youtu.be/urqXgBbVyWY
Once enabled my LSA credentials are not longer stored in-memory in plain text. This also adds another Windows process, LSALSO which is the new credential handler:

If this were a LAN PC, and hence no need for the -cg switch (I'm presuming a LAN connected PC doesn't need to VPN into the LAN.....) the -ready check should show this after I ran a straight -enable switch. Below is Windows 2022 Server after the -enable with all features green:

OK, servers. Physical servers are enabled the same as desktops. VMware Windows guests are different. These need to enabled in the VM options under Virtualization Based Security (VBS) and then the PowerShell ran as desktop. This feature is available in vCenter 6.7 and higher. At the time of writing I'm still not getting Windows Server 2016 to work even though it should, 2019 and 2022 are both fine. YMMV. As always take a snapshot for the VM before jacking with it. Checking the VBS box will enable IOMMU and UEFI (you should already be using UEFI anyway). Here's the check box in question for VMs (note, you only see this if you specifically set the guest OS version, i.e. Windows Server 2022 in the General Options section, if you leave VMWare Tools to figure it out this check box does not appear):

It is not lost on me the irony of a Windows VM running a Windows VM in order to secure it's credentials. Nested VMs like this used to be a big serious no-no but with the advent of CG/VBS most of the real-world arguments are around performance. I haven't seen an perceptible performance degradation on any VM, but again YMMV.
I'm a really big believer in doing Credential Guard whenever and wherever possible. If it's a 2019 or greater server and I've built it chances are it's CG protected. All of the STS desktops and laptops are CG enabled as well, although you do need Windows 10 Enterprise or Education to enable it. If you want to talk about getting on the Windows 10 Enterprise bus, drop Lisa a line and we can talk about it. It's a worth having if for no other reason than CG.
This is just the basics of Credential Guard so be sure to check out the additional mitigations you can also take to further secure your environment here: https://docs.microsoft.com/en-us/windows/security/identity-protection/credential-guard/additional-mitigations
See here for the entire series of posts, if you are just stumbling onto these posts.
As I said all the way back in part one, these post are supposed to be helpful in giving you meaningful useful advice to prevent ransomware.
This series is now over a year in the making.....I hope a reader or two still exists.
Certain versions of Windows have a special feature called Credential Guard. Due to Microsoft not being, well, particularly into security this feature is not present in Home nor Pro versionS of Windows. I view this a travesty, but hey Microsoft makes tons of money so why should they care. It does exist in Enterprise and Education desktop Windows and also in Windows Server since 2016. If you have looked at doing Windows 10 Enterprise before, but haven't found a killer feature, then this is it (and LTSC). If you have never looked at buying Windows 10 Enterprise, this is the feature that should make you look into it.
Not only is is not widely available, it's woefully marketed. Did you even know about this? One of the two most important tools in the hacker prevention tool chest? (the other being SRP, aka part 6 in this series)
Credential Guard does what it says on the box, it protects credentials. Specifically in-memory credentials. These are stored in such a way as to be accessible to hackers once they compromise the device ("pass the hash" is the usual name for this type of hack). If you have no idea what I'm talking about go watch this video that uses the MimiKatz tool to extract in-memory credentials (password hashes to be exact) out of thin air....
https://youtu.be/bTYR_xYSDIk
Scared yet? If you're not then you're in the wrong job. Go read a gardening blog or take up knitting.
What the above video shows is how easy it is to effectively harvest credentials from Windows OSes. Credential Guard addresses this Windows "feature". It also worth noting that some CPUs now also have this type of protection built in, specifically AMD Ryzen Pro CPUs can have a similar protection enabled in BIOS. But on the Pro line.
If a hacker can harvest a domain admin account you are toast. They have already won. Just take down your tent and go home. Find a good gardening blog or take up knitting. Your job is to prevent that from happening......
OK, so I have Windows 10 Enterprise, or 2019/2022 server. How do I get this level of protection? For starters, VMs are a bit different, so I'll cover those later. Second, laptops with VPN clients are different so read this all before you enable it on laptops. Even a standard desktop OS it's a lot of convoluted steps. Thankfully Microsoft do provide a PowerShell script to simplify enabling it. They way it works it also a bit convoluted. The setup even more so. The PowerShell (see two paragraphs down) is a God send.
See, the "fix" Microsoft came up with is to install a Hyper-V machine on the device in question, lock it down and encrypt it and store the credentials in that Hyper-V instance. So now you have two PCs. Kind of. If you really want to know more about how it works see here: https://docs.microsoft.com/en-us/windows/security/identity-protection/credential-guard/credential-guard-how-it-works
The PowerShell readiness/enablement script is here: https://docs.microsoft.com/en-us/windows/security/identity-protection/credential-guard/dg-readiness-tool
The above script needs to be ran in an admin PowerShell with the -capable switch, then you need to reboot and run it again with -capable. Be sure to check out the pre-reqs as well (UEFI, enable virtualization technologies in BIOS, etc.)
If the script says you can enable it, run it with -enable. A reboot and few auto reboots later, Credential Guard is installed. Note, if you use a VPN client on the device in question, chances are the VPN is not fully CG compatible, so be sure to check, If the VPN client is not CG compatible run with -enable -cg (so not just -enable, add the -cg).
Here I'm running the script with the -capable switch to see if my PC can indeed enable CG.....
I need to reboot, then I run the -capable again:
In the above screen shot I have highlighted an issue. In this case it is very likely a VPN shim driver as I'm running it on a laptop with a VPN client so I will run the "-enable -cg" flags to enable *only* CG (just "-enable", so without "-cg" does get me better security, but experience tells me it will stop my VPN client for working.)
Above, we can now see Hyper-V and IOMMU have been enabled. Time to reboot again....and then rerun the PowerShell with the -ready switch:
As you can see I now have Credential Guard enabled. The yellow warning are because I chose to *only* enable CG and not the other option as that would croak my VPN client. MimiKatz has now been taken to the vet and euthanized and the password hashes are no longer accessible to hackers. You can see this in action on this video:
https://youtu.be/urqXgBbVyWY
Once enabled my LSA credentials are not longer stored in-memory in plain text. This also adds another Windows process, LSALSO which is the new credential handler:
If this were a LAN PC, and hence no need for the -cg switch (I'm presuming a LAN connected PC doesn't need to VPN into the LAN.....) the -ready check should show this after I ran a straight -enable switch. Below is Windows 2022 Server after the -enable with all features green:
OK, servers. Physical servers are enabled the same as desktops. VMware Windows guests are different. These need to enabled in the VM options under Virtualization Based Security (VBS) and then the PowerShell ran as desktop. This feature is available in vCenter 6.7 and higher. At the time of writing I'm still not getting Windows Server 2016 to work even though it should, 2019 and 2022 are both fine. YMMV. As always take a snapshot for the VM before jacking with it. Checking the VBS box will enable IOMMU and UEFI (you should already be using UEFI anyway). Here's the check box in question for VMs (note, you only see this if you specifically set the guest OS version, i.e. Windows Server 2022 in the General Options section, if you leave VMWare Tools to figure it out this check box does not appear):
It is not lost on me the irony of a Windows VM running a Windows VM in order to secure it's credentials. Nested VMs like this used to be a big serious no-no but with the advent of CG/VBS most of the real-world arguments are around performance. I haven't seen an perceptible performance degradation on any VM, but again YMMV.
I'm a really big believer in doing Credential Guard whenever and wherever possible. If it's a 2019 or greater server and I've built it chances are it's CG protected. All of the STS desktops and laptops are CG enabled as well, although you do need Windows 10 Enterprise or Education to enable it. If you want to talk about getting on the Windows 10 Enterprise bus, drop Lisa a line and we can talk about it. It's a worth having if for no other reason than CG.
This is just the basics of Credential Guard so be sure to check out the additional mitigations you can also take to further secure your environment here: https://docs.microsoft.com/en-us/windows/security/identity-protection/credential-guard/additional-mitigations
March 24 2022 Thursday
Ransomware Prevention Part 9 - More semi-easy stuff
Part 9 - More semi-easy stuff
See here for the entire series of posts, if you are just stumbling onto these posts.
As I said all the way back in part one, these post are supposed to be helpful in giving you meaningful useful advice to prevent ransomware.
Well, it's been a while since part 8, but this series generated a lot of work for STS, so I'm not complaining. After doing some of this stuff for a while now I have some more suggestions for you to implement.....
HTA - the malware distribution engine Microsoft provides free of charge
I always forget organizations still have and use Internet Explorer. Thanks Microsoft! Even though it goes end of life in June 2022, it's still used in a lot of places. A lot. Not only did MS foist IE on us, but they also bundled on top a thing called HTML Application (HTA for short, see https://en.wikipedia.org/wiki/HTML_Application). And this thing keeps giving, and giving, and giving. In short it allows a bad actor to trick your users into possibly opening a bad application. See HTAs don't use the security model of the browser. No, HTA's are fully trusted applications that use the IE engine and allows VBScript and JScript to run with access to the file system and the registry. Without supervision. FYI, file system access is how ransomware encrypts shared drives, You've just connected the dots right? From the Wikipedia article above:
Then this gem....
Holy $%*# Batman.
In all my years doing this I don't recall seeing a single practical use for HTAs. The only time I've seen it used is to distribute malware via a script based attack. To block this super villain sized nastiness do as many of these things as you can:
1. Block all .HTA files from running. Period. Use a software restriction policy, add it to your endpoint protection. Add it everywhere.
2. Block mshta.exe from running. Again, use a software restriction policy, add it to your endpoint protection. Add it everywhere.
3. Block the MIME type application/hta.
4. Block files containing MTA:Application header.
5. Delete the .hta file association in Windows,
Do all of the above as HTAs are slippery little buggers. Oh, and just deleting mshta.exe is probably not sufficient as it can be dropped again using a different name. You really need as many of the above as you can figure out how to implement.
If you thought HTAs were bad.....two words: Velvet. Sweatshop. Read on.....
Somewhat harder....Password protected Office files
Look, I know it's hard. People like to think they are in the know by password protecting Excel files. It feeds their superiority complex. However, what you may not know is that if said password is "VelvetSweatshop" (no quotes) it is exceedingly special in Excel. And not in a good way.
Password protected files in Office are actually also encrypted files in Office (the encryption level is based on the Office version, https://en.wikipedia.org/wiki/Microsoft_Office_password_protection). So by being encrypted, these files essentially bypass any and all scanning by your security systems as the scanners cannot see inside the files to analyze them. Still with me? Because this is about to get very, very interesting....
When a user opens a password protected Excel file, Excel (all by itself) tries to use the password "VelvetSweatshop" to decrypt it. And if that is the password on the file, it does and it happily opens the file. Without any user intervention an Excel file protected and encrypted with the password "VelvetSweatshop" opens. Again, in big text.....
And again.....
Yeah, yeah...the daily thought of "what the eff are Microsoft thinking?" shouts inside your head yet again. Yes Excel has a default password. Kinda like crappy Wifi routers, right? Go try it, it's equal parts insane and, well, insane. It still works as of Excel 2201.
So if you can't scan a password protected Office file and Excel will happily open the file and (drum roll please.....), and lets say for kicks and giggles you also have Office macro support enabled (and you do, everyone does)..... then an unsuspecting user can open an Excel file which has neutered all of your high priced security systems, which then runs code (the Office macro), the bad actor could now have a foot hold in your environment. All because of a default password that exists in Excel.
Oh, and if you think the yellow bar at the top of Excel asking if the user wants to trust the macro, and that said user won't click "absolutely", I have some DarrenCoin to sell you.
Which brings us to....
Somewhat, somewhat harder.....Disable Office Macros from running
I their defence, Microsoft is adding extra security in a few months to Office to prevent internet downloaded Office macros from running. That's a step, but after reading about HTAs and Velvet Sweaters are you going to trust Microsoft?
You simply (ha, not that simple at all actually) need to disable Macro support in each and every Office product that has it. If your business processes are so complicated that they require Office Macro support, simplify your process by firing the idiot who designed it and then disable macro support.
Conclusion
Doing the above will seriously raise your security. You'd be better than most, if not all, of your peers. The older the version of Office you are on, the more vulnerable to these attacks you will be. Just saying.....
And if you want to kick your superiority complex into high gear, go ask your security people about HTA and Velvet Sweaters and when they look at you funny, send them this post. They won't sleep for days.
See here for the entire series of posts, if you are just stumbling onto these posts.
As I said all the way back in part one, these post are supposed to be helpful in giving you meaningful useful advice to prevent ransomware.
Well, it's been a while since part 8, but this series generated a lot of work for STS, so I'm not complaining. After doing some of this stuff for a while now I have some more suggestions for you to implement.....
HTA - the malware distribution engine Microsoft provides free of charge
I always forget organizations still have and use Internet Explorer. Thanks Microsoft! Even though it goes end of life in June 2022, it's still used in a lot of places. A lot. Not only did MS foist IE on us, but they also bundled on top a thing called HTML Application (HTA for short, see https://en.wikipedia.org/wiki/HTML_Application). And this thing keeps giving, and giving, and giving. In short it allows a bad actor to trick your users into possibly opening a bad application. See HTAs don't use the security model of the browser. No, HTA's are fully trusted applications that use the IE engine and allows VBScript and JScript to run with access to the file system and the registry. Without supervision. FYI, file system access is how ransomware encrypts shared drives, You've just connected the dots right? From the Wikipedia article above:
When a regular HTML file is executed, the execution is confined to the security model of the web browser. This means it is confined to communicating with the server, manipulating the page's object model (usually to validate forms and/or create interesting visual effects) and reading or writing cookies.
Then this gem....
On the other hand, an HTA runs as a fully trusted application and therefore has more privileges than a normal HTML file; for example, an HTA can create, edit and remove files and registry entries. Although HTAs run in this 'trusted' environment, querying Active Directory can be subject to Internet Explorer Zone logic and associated error messages.
Holy $%*# Batman.
In all my years doing this I don't recall seeing a single practical use for HTAs. The only time I've seen it used is to distribute malware via a script based attack. To block this super villain sized nastiness do as many of these things as you can:
1. Block all .HTA files from running. Period. Use a software restriction policy, add it to your endpoint protection. Add it everywhere.
2. Block mshta.exe from running. Again, use a software restriction policy, add it to your endpoint protection. Add it everywhere.
3. Block the MIME type application/hta.
4. Block files containing MTA:Application header.
5. Delete the .hta file association in Windows,
Do all of the above as HTAs are slippery little buggers. Oh, and just deleting mshta.exe is probably not sufficient as it can be dropped again using a different name. You really need as many of the above as you can figure out how to implement.
If you thought HTAs were bad.....two words: Velvet. Sweatshop. Read on.....
Somewhat harder....Password protected Office files
Look, I know it's hard. People like to think they are in the know by password protecting Excel files. It feeds their superiority complex. However, what you may not know is that if said password is "VelvetSweatshop" (no quotes) it is exceedingly special in Excel. And not in a good way.
Password protected files in Office are actually also encrypted files in Office (the encryption level is based on the Office version, https://en.wikipedia.org/wiki/Microsoft_Office_password_protection). So by being encrypted, these files essentially bypass any and all scanning by your security systems as the scanners cannot see inside the files to analyze them. Still with me? Because this is about to get very, very interesting....
When a user opens a password protected Excel file, Excel (all by itself) tries to use the password "VelvetSweatshop" to decrypt it. And if that is the password on the file, it does and it happily opens the file. Without any user intervention an Excel file protected and encrypted with the password "VelvetSweatshop" opens. Again, in big text.....
Without any user intervention an Excel file protected and encrypted with the password "VelvetSweatshop" opens.
And again.....
Without any user intervention an Excel file protected and encrypted with the password "VelvetSweatshop" opens.
Yeah, yeah...the daily thought of "what the eff are Microsoft thinking?" shouts inside your head yet again. Yes Excel has a default password. Kinda like crappy Wifi routers, right? Go try it, it's equal parts insane and, well, insane. It still works as of Excel 2201.
So if you can't scan a password protected Office file and Excel will happily open the file and (drum roll please.....), and lets say for kicks and giggles you also have Office macro support enabled (and you do, everyone does)..... then an unsuspecting user can open an Excel file which has neutered all of your high priced security systems, which then runs code (the Office macro), the bad actor could now have a foot hold in your environment. All because of a default password that exists in Excel.
Oh, and if you think the yellow bar at the top of Excel asking if the user wants to trust the macro, and that said user won't click "absolutely", I have some DarrenCoin to sell you.
Which brings us to....
Somewhat, somewhat harder.....Disable Office Macros from running
I their defence, Microsoft is adding extra security in a few months to Office to prevent internet downloaded Office macros from running. That's a step, but after reading about HTAs and Velvet Sweaters are you going to trust Microsoft?
You simply (ha, not that simple at all actually) need to disable Macro support in each and every Office product that has it. If your business processes are so complicated that they require Office Macro support, simplify your process by firing the idiot who designed it and then disable macro support.
Conclusion
Doing the above will seriously raise your security. You'd be better than most, if not all, of your peers. The older the version of Office you are on, the more vulnerable to these attacks you will be. Just saying.....
And if you want to kick your superiority complex into high gear, go ask your security people about HTA and Velvet Sweaters and when they look at you funny, send them this post. They won't sleep for days.