This is a big issue
and it has been there for a long time. Just a month ago I finally got
verification that the Microsoft Product Groups not only know about this but that
they have begun working on a fix. As I want to be known as a white hat I had to
wait for this to happen before I blog this.
There is a small but
CRAZY bug in the way the "Feature Update" (previously known as
"Upgrade") is installed. The installation of a new build is done by
reimaging the machine and the image installed by a small version of Windows
called Windows PE (Preinstallation Environment). This has a feature for
troubleshooting that allows you to press SHIFT+F10 to get a Command Prompt.
This sadly allows for access to the hard disk as during the upgrade Microsoft
disables BitLocker. I demonstrate this in the following video. This would take
place when you take the following update paths:
- Windows 10 RTM --> 1511 or 1607 release (November Update or Anniversary Update)
- Any build to a newer Insider Build (up to end of October 2016 at least)
The real issue here
is the Elevation of Privilege that takes a non-admin to SYSTEM (the root of
Windows) even on a BitLocker (Microsoft's hard disk encryption) protected
machine. And of course that this doesn't require any external hardware or
additional software. It's just a crazy bug I would say :(
Here's the video:
Why would a bad guy
do this:
- An internal threat who wants to get admin access just has to wait for the next upgrade or convince it's OK for him to be an insider
- An external threat having access to a computer waits for it to start an upgrade to get into the system
I sadly can't offer
solutions better than:
- Don't allow unattended upgrades
- Keep very tight watch on the Insiders
- Stick to LTSB version of Windows 10 for now
(Update 6.12.2016: Read the next blog as well: http://blog.win-fu.com/2016/12/every-windows-10-in-place-upgrade-even.html )
I am known to share
how I do things myself and I'm happy to say I have instructed my customers to
stay on the Long Time Servicing Branch for now. At least they can wait until
this is fixed and move to a more current branch then. I meet people all the time
who say that LTSB is a legacy way but when I say I'm going to wait a year or
two to get the worst bugs out of this new "Just upgrade" model - this
is what I meant…
Remember to subscribe to my newsletter as I will disclose more like this very soon! Subscribe here!
And you can learn how to find these by yourself by letting me teach you some Windows Internals!
- http://win-fu.com/ilt/
- http://dojo.win-fu.com/
- http://pluralsight.com/
- I also offer Security Auditing for companies just send me an email: sales@adminize.com
And as I kind of guessed many readers will be greatly from managed Enterprises with SCCM - for you SCCM can block this. I'm an OS guy and more worried about the ones that are not managed by SCCM as there are still a lot of millions of installation of such character out there ;)
ReplyDeleteCan this method be used on external drive as i have bitlocked it but does not have the password or recovery key of the same. Please tell as i am searching for a solution for this from a year and till yet nobody has given a proper answer t this problem.
DeleteThank You
Ajay Madrewar
No, it doesn not work on external drive. I'm sorry to say but you are not supposed to find a way to get around your problem without the key - otherwise BitLocker would be useless.
DeleteThanks Sam I posted this on my Facebook page "deployment Research"
ReplyDeleteRecommending LTSB is inappropriate.
ReplyDeleteEvery environment is different and that was only one of my solutions. Maybe the first one is yours. Out of curiosity would you be willing to elaborate on your reasons for not using LTSB? Real business reasons would come handy when I need to convince my customers in the future.
DeleteI think LTSB is appropriate in highly critical system. It is so stripped down (No edge, No Windows Store)that I would not recommend it for regular users. What's the point of using Windows 10 if it's not for the incremental features upgrades and the app store... ?
DeleteThat's not the case for my environment. I can't use Edge as it's not working for what I need although it's getting better all the time. LTSB supports universal apps so I'll sideload them if I need them but I don't need the store for anything. I might change my mind when I find something in the store that I need. I need Windows 10 for higher security and longer support. From my perspective "so stripped down" sounds a bit weird and I see it as it just misses a few minor things.
DeleteSami, Completely agree. For our company environment the "stripped down" version makes more sense. We simply look at it from a business need vs personal "need". Same issue with Edge. Not to mention if you are following standard CIS Level 1 or NIST requirements LTSB is pretty much what you end up with once on the security policies are in place.
DeleteThis would be the same for all Win versions, no? (Why title the article just for Win 10?)
ReplyDeleteAlso couldn't the issue could be applied when re-imaging a LTSB workstation?
When doing in-place upgrades on Windows 7 and up, yes this is an issue as they all support BitLocker and the installation works the same. But has someone done inplace upgrades before we got to Windows 10? I don't have a single customer.
DeleteTrue, Kieran. This has been an issue/feature of Windows at least as far back as Windows XP if not further back.....
DeleteXP did not have an imaging based in-place upgrade mechanism. This was added with Vista. If you wanted WinPE on XP you needed to boot from external media.
DeleteBut Windows XP did support the CMD shell in the setup procedure. It wasn't WinPE but the Shift+F10 feature was there. That time it was just a nice troubleshooting tool as there was no BitLocker.
DeleteDoes using VeryCrypt for personal use make a difference here? not in an enterprise environment with lots of centrally managed computers
DeleteDoes using VeryCrypt for personal use make a difference here? not in an enterprise environment with lots of centrally managed computers
DeleteVeraCrypt won't be disabled. There are other drawbacks on using that solution and I'm not changing BitLocker for that. But if you just have a personal computer then this will not work against VeraCrypt. THat's all I'm gonna say on that ;)
DeleteIf bad actors have such free physical access to your gear that they wait to pounce on an in place upgrade, I humbly suggest that you have much, much bigger problems.
ReplyDeleteThe LTSB isn't designed for use as a daily driver. Full stop. Users will encounter significant usability issues; $dayjob has already seen this happen, notably with users of Netbrain.
The impact of this issue to any organisation must be examined in the context of their threat model. Again: if bad actors have the freedom of access to wait for updates, then your organisation has much bigger issues.
Every company is different and LTSB was just one of my suggestions, maybe the first one is then yours. To battle your "Full stop", out of curiosity would you be willing to elaborate on your reasons for not using LTSB? Real business reasons would come handy when I need to convince my customers in the future.
DeleteAnd sad to say I travel more than 200 days a year and I have seen hundreds of computers doing upgrades at airports so I agree there is a bigger problem but I don't see how having a bigger problem would have prevented me from using this to access the machine rather than anything that is harder.
If bad actors have such free physical access to your gear that they wait to pounce on an in place upgrade, I humbly suggest that you have much, much bigger problems.
DeleteRogue sysadmin, disgruntled by company policy, who has the boss's laptop to rebuild? That's an easy scenario to visualise.
I'm in agreement that LTSB isn't good for standard enterprise PCs. They will likely result in a poor user experience in the long run since they won't have access to features like the App Store, agreed, minimal value now, but hopefully not in the future. For companies that want to provide a more consumer like experience, this also leaves you with the old model of doing things where users often have newer, better technology at home. Microsoft still hasn't fully articulated a solid roadmap for future chipsets on LTSB. Under their current stance, the next two chipsets after Kaby Lake may not be supported by LTSB until a new LTSB is released in 2018 or 2019. Finally, this will continue to perpetuate the need to support multiple operating system versions for a long period of time.
DeleteA rogue sysadmin in possession of the PC already has full access, they don't need exploits to do bad things to the PC. This bug doesn't offer them anything they can't already do.
I don't like the "likely" but rather have decisions made on facts. I stated my plan of moving to CBB in a few years. The current plan is 2018 after Microsoft has been able to fix silly issues like this. The future roadmap of CPU's plays no role in this schedule. I will not have the problem of supporting multiple versions. I will do one inplace upgrade in 2018 after you have done approx 2-4. Well, if MS hasn't fixed things by then to what I'm happy with I might reconsider.
DeleteHow is this a bug? I have to agree with the previous comments.
ReplyDeleteShift+F10 has been around for ages - You can do it in Windows 7! I also agree in that recommending LTSB is definitely inappropriate for non large enterprise users.
Finally, I quote grey_area's comment here, "If bad actors have such free physical access to your gear that they wait to pounce on an in place upgrade, I humbly suggest that you have much, much bigger problems."
Shift-F10 is not the issue here. The issue is switching off BitLocker while upgrading. Shift-F10 is just an example of hundreds of ways of using this. I can always just reboot the box in the correct phase and bypass BitLocker but I used this example as it could be (just tested) executed by a 7 year old. Shift-F10 I used in NT to play solitaire while installing a machine so that is WAY old.
DeleteAnd I repeat my previous comment:
When doing in-place upgrades on Windows 7 and up, yes this is an issue as they all support BitLocker and the installation works the same. But has someone done inplace upgrades before we got to Windows 10? I don't have a single customer. Or has it been the recommendation from Microsoft.
And now that we got started, out of curiosity would you be willing to elaborate on your reasons for not using LTSB? Real business reasons would come handy when I need to convince my customers in the future.
If you fail to turn off BitLocker before the upgrade you can't actually "upgrade". Windows PE would have no visibility into the files and would result in a full system re-image. So all of your programs would need to be re-installed, all profiles have to be re-established (not to mention loss off any items stored in those profiles), etc. So yes, BitLocker *can* remain on during the imaging process but the end result would be way worse than the the far flung physical-access-requiring scenario you imagine.
DeleteTotally disagree. You can "open" the BitLocker encrypted drive in Windows PE and upgrade, you don't have to SUSPEND the protection as it does now. Windows Recovery Environment does this when you recover a non operating Windows - it does not suspend like this does. So your comment is technically incorrect.
DeleteThe comments in this blog really shows what's wrong with 90% of enterprise IT departments. Information is considered secure while there are no security issues within company premises and network. However reality is a totally different thing. Company laptops are used everywhere. And end users tend to do everything to avoid security. Physical security is really an issue everywhere.
ReplyDelete+10
DeleteThis is 100% true. In a perfect world, this isn't much of an issue. The enterprise safeguards will protect us. Like someone else said - you can see PC's upgrading in some places. It'd be easy to utilize this "bug" and grab what you needed. It is something to be aware of, and possibly get a fix in the future.
DeleteThe LTSB... It's got it's pros and cons. There are some reasons where it would be appropriate, and in this case - if it's a problem for you, it is one of a few different options. It's going to be a case by case basis, though, obviously.
LTSB is inappropriate... That Microsoft marketing brainwash machine... Big up to you Sami for being the only one MVP saying publicly to use LTSB. People will beat you for that, but that will definitely make you a Hero and a trust full person. All my respect for that!
ReplyDeleteThanks! :)
DeleteAgreed,
DeleteI wish I would have installed LTSB across all our laptops in the field.
Honestly I can’t imagine why people knock it so much in a corp. environment. First I don’t see why I would ever want to let a user install their own apps let alone have free reign of the MS app store along with all the other “helpful additions” MS puts into the CCB versions”. My laptops are glorified kiosk where users get the apps they need and 0 access to any they don’t. The cry to not use LTSB seems more a marketing ploy then actual advice on how to keep security tight.
Just wondering. I have BitLocker on on my C-drive (the only drive), if my laptop got stolen (when it is shut off, not in sleeping mode), how would somebody use this "exploit"? Someone already need an user account and the laptop should be on for this to work right?
ReplyDeleteIf you have preboot authentication for BitLocker (PIN/USB) and the computer is in hibernate/power_off you have no issue with this. If your computer turns on and you have automatic upgrades enabled, and not controlled by WSUS/SCCM, then at some point the computer will install even if you are not logged on.
Deletejust out of curiosity, if i create an Image to Go on a USB stick and start the computer from USB, won't that bypass the a/m scenario?
Deleteand on another point - LTSB vs. CBB - i had the same strong opinion against CBB, but please consider the following:
1. EDGE cannot be installed (if you need a browser other than explorer / chrome / FF)
2. some kernel based updates will not be delivered until the next LTSB update (2-3 years)
3. new features which you may want (such as ATP or "Application Guard") will not be available
4. any feature you do not want (Cortana, store, widgets...) can be disabled as part of the installation
Thanks, I have considered these and many other things before I made up my mind. I'm sorry I don't understand the Image to Go point. Do you mean "Windows To Go"? That doesn't Support inplace upgrade AFAIK.
DeleteImage to Go - SCCM Deployment scenario, create a stand-alone ISO that you can put on a flash drive, so the process would be:
Delete1. boot from USB (SCCM image to go)
2. start install / upgrade process, suspends the encryption
3. press F10 (or F8) to get command prompt
4. have full access to the HD...
Bitlocker is not only used for protecting data. In enterprises it is used to prevent regular users to become Admin, more general it is used to keep the integrity of the machine. From an IT-Admin view the regular user is the attacker. If the user has the possibility to gain admin-rights the IT-Admin has lost the game. So: As Sami demonstrated the inplace-upgrade is a big issue. It disables bitlocker - the regular user sitting in front of the machine has several ways of easily using this timeframe while upgrading to get admin-rights. Currently no way of preventing this.
ReplyDeleteJust a small nomenclature issue because it's worth mentioning to keep this consistent with TechNet. Bitlocker is NOT disabled (that would imply the drive is decrypted). Bitlocker is SUSPENDED, which means that the TPM tampering checks and additional PIN/password is bypassed, thus decrypting the drive on the fly automatically.
ReplyDeleteI think the best course of action for most companies is to use WSUS/SCCM, which even small organizations can easily set up, and then not approve any upgrades to computers that are not in a controlled and trusted environment until this is fixed. That's probably the simplest course of action for many, but every environment is different, of course.
Good clarification on my terms, thanks :) Suspended would have been the correct term.
DeleteWhat is a "trusted environment"? How would you prevent that the User "owning" the machine gets his fingers onto the machine while the FeatureUpdate is running? Supervising the Machines by IT-Staff?
DeleteI'll just leave this here..
ReplyDeletehttps://www.youtube.com/watch?v=mU8vw4gRaGs
Not able to edit my comment.
DeleteTitle: WSUSpect - Compromising The Windows Enterprise Via Windows Update
How about deploying an in-place-upgrade that the local wsus admin have not yet deployed, instead of for example PsExec.
1) Once an attacker has physical access to your PC, all bets are off.
ReplyDelete2) This would occur only if you left an updating PC unattended.
Much ado about nothing.
The Attacker is the regular user who gains Admin-Rights. Regular Users have physical access (as long as we are talking about physical Machines and not VDI).
Deleteagree there, most upgrades are off-hours, so users are not interrupted.
DeleteI have to answer to this comment just By saying that I counted three updating, unattended PC's, in the Air Berlin lounge in Vienna last week...
Deleteyou have a point, but that scenario is for a 24x7 type of business (airports, hospitals, etc), i think most businesses are not 24x7 businesses.
DeleteSorry but NO, WSUS/SCCM won't be of any help to work around this bug. WSUS deliver upgrade in the same way as Windows update do, so there is no additional security layer that prevent this delivery method to act differently from what you showed in your Video. Upgrade within SCCM won't allow you to disable F10 Key because task sequence Upgrade don't use SCCM's WinPE. The upgrade process really works on is own and use the PE embbeded in the upgrade sources. So, not to say that there is no way to inject the text file in the PE before it launch, but there's nothing out of the box in both tools that is able to do it for now !
ReplyDeleteThe BitLocker issue of course is there but I have a lot of people and most of all Mr. Arwidmark whom I trust like god telling that is not just doable in SCCM but even easy?
DeleteWSUS only helps in deferring this, not in blocking this. And as said non of them will fix the big issue of a suspended BitLocker.
Yes, I've red the twitter conversation and agreed in the first place but after further thinking i came to the conclusion that what he told you does not apply to Inplace upgrade scenario. What Yohan told is about the WinPE image managed by SCCM. This PE image is required by every task sequence template... except for the inplace upgrade one (i explained why in my previous comment)! Apart form being completely out of topic i'm sure Yohan won't disagree with me.
DeleteAarghh... Pulling hair here... Well it seems you are correct :) I just asked Johan and he said you are so you must be.. It is sad of course to hear this... Thanks for correcting this!
DeleteSami, I agree with you on LTSB. We are a consulting company in DC and we are using LTSB 2015 and now 2016. We have an SCCM environment in place for the updates. Users here dont need the app store or edge as we have some legacy systems that function on IE only. Our company was saved from the many updates that caused major issues like the webcam update and anniversary updates.. We deploy security patches and updates every month and that has kept issues to a minimum. I love how people say stay away from LTSB but dont provide any substance in why we should. Windows 10 was plagued with problems since day 1 and issues are still occurring! Sure LTSB is a strip down version but in a workplace environment your computer should not be the same as what you would experience at home! In addition, Bitlocker has been vulnerable for quite some time. If you search youtube you can find various hacks.. So disabling Bitlocker while performing the upgrade is unreal.
ReplyDeleteAnyways, that is my two cents!
+10
DeleteNot if Bitlocker is managed with MBAM
Deleteso there is no mitigation normal user can apply?
ReplyDeleteMicrosoft is working on a patch as reported...
DeleteIn the future you might want to utilize Windows 10 2016 LTSB and/or WSUS, SCCM to manage updates. We did not perform upgrades from 1511 to 1607...
Every environment is different so its best to evaluate what is appropriate for your needs. I personally do not appreciate the "automatic" updates/features being installed on Windows 10 and like to control what our users experience, see, and function with!
Hope this helps...
You can add:
DeleteDisableCMDRequest.tag to the %windir%\Setup\Scripts\ folder of your Windows 10 image
Wait a few days and I'll tell you why that doesn't fix the issue ;)
DeleteThis comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteLet me tell you what is remarkable: It is remarkable that it takes people so long to realize. When did the first people test in-place upgrades from win7/8.1 to 10 preview? That must have been almost 2 years ago.
ReplyDeleteWhen did the first people uses WSUS to deploy win10? 1,5 years ago, maybe a little later, because the developers at MS are not able to develop a package for wsus while it's the same as the OS setup, basically.
And now they realize what they have done. Admins around that world now face millions of workstations where no one knows if the users have abused this (I am not calling it a bug) thanks to the great idea to utilize wsus to spread OS upgrades.
I started a thread about this in January 16, see https://www.administrator.de/wissen/admin-windows-10-upgrade-setups-293904.html and another on technet https://social.technet.microsoft.com/Forums/windows/en-US/61c9fa5b-6f3d-418c-b459-fedc1858cbc4/windows-10-upgrades-and-shift-f10-why-does-ms-still-offer-this?forum=win10itprosecurity
How will they try to fix this? So right after an upgrade is initiated, Bitlocker will move to a special new mode? This will not happen, I am afraid.
Computer security will never come. Too complex.
This comment has been removed by a blog administrator.
ReplyDeleteWhy (build 10240)ltsb 2015 doesnt run client ui.exe for mbam
ReplyDeleteToo old client of MBAM? Helped in my case. Although I have to say that I don't understand how to comment relates to this blog...
DeleteMicro$oft works with all intelligence agencies and OF COURSE has absolutely cool backdoors in the latest windows version. What better way to spy on THE WHOLE WORLD?
ReplyDeleteFUCK M$
I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post.
ReplyDeleteHow would EFS mitigate this issue?
ReplyDeleteIt would block access to certain files but not the system.
DeleteNo, this applies to the OS drive only.
ReplyDeleteHello,
ReplyDeleteis this still a thing?
I would love to do it without a pre-boot pin/password and use only the TPM and a good user (windows) password.
I wonder, are gonna windows upgrades (to a new redstone for example) installed at all when no user is logged on?
Thank you,
greetings
Martin
Yes, still a thing. Not with Shift+F10 but BitLocker still gets disabled. This is same with PIN or USB or anything so go ahead with TPM, that's what I use as well. At some point the computers will install RS4 or whatever, yes.
ReplyDeleteHi,
Delete> This is same with PIN or USB or anything (...)
Woow so even pre-boot auth does not help?
So a thief for example has to wait max 6 months to the next upgrade and then he has access to the data?
Nice "feature"... and Microsoft does not care at all?
I would like to have a fraction of this "looseness".... would make life a lot easier! ;-)
Why on earth is this not big in the news?
Thank you,
greetings from Austria!
Martin
And now I can officially say that progress is being made on this: https://twitter.com/miketerrill/status/991323332489326592
DeleteSorry für the dumb question, but this is a setup.exe command-line option - right?
DeleteHow do I set this when I install from a iso or media creation tool usb drive?
Can it be also set after the installation?
And whats the default value/behavior?
Thank you,
greetings
Martin
More info starts to appear: https://blogs.technet.microsoft.com/mniehaus/2018/05/02/new-upgrade-to-windows-10-1803-without-suspending-bitlocker/
ReplyDeleteThis is new to me as well so I'm currently learning as fast as you are ;)
Oh wait a moment, this is only a switch for future rollouts - right?
DeleteSo this not affects the behavior when the device updates through windows updates.
What can I do to prevent this issue with a SOHO machine which is not under control of any deployment services like WSUS?
Shouldn't prevent a Pre-boot auth the device from obtain an update?
Normal quality updates (monthly) don't require BitLocker to be suspended. This only applies to 1709-->1803 or future upgrades after that. I don't think you can prevent the suspend on a non-managed device :( Maybe the default will change in the future - I sure hope so.
DeleteAnd remember this requires you to use TPM-only, no PIN or USB. Well, I never use them anyway but just good to remember.
I know, only upgrades (I call them pseudo service packs *g*... 10 years support/service packs/updates.... were good times) does suspend Bitlocker.
DeleteBy the way - is then only the system partition "open" or every other partition or internal drive as well?
> I don't think you can prevent the suspend on a non-managed device
Any other idea for my problem?
I will just prevent that a thief gives the machine an internet connection,
waits until the next upgrade and has then meanwhile the upgrade) access to the data.
I wonder if a pre-boot auth prevents the machine not anyways from obtaining an upgrade?
And (yes I know i asked that before) are you really sure that a device can get and install
upgrade (not a update!) at all when no user is logged on?
I have to admit that I have presumably too few win10 devices around me to seen a machine doing that.