Gatekeeper in Mountain Lion: What You Need To Know

According to the numbers, Mountain Lion brings over 200 new features to your Mac. While many of those are under-the-hood enhancements and security features that most users will never even think about, one feature in particular can dramatically change the way you use your Mac, and knowing how it works will ensure that you’re using your computer in the safest way possible. Hit the jump to learn everything you need to know about Gatekeeper in Mountain Lion.

On The Origin of Gatekeeper

Gatekeeper (mostly) began in OS X 10.4 Tiger, where Apple introduced a relatively low-profile technology that they called File Quarantine. While you may not have heard that name, you’re certainly familiar with it’s purpose.

You know that dialog that shows up whenever you try to open a file downloaded from the Internet? The one that says “[AppName].app is an application downloaded from the Internet. Are you sure you want to run it?”. That’s File Quarantine.

File Quarantine in Lion reminding you that the app you’re trying to run came from the Internet.

Essentially, the File Quarantine validation system flags any files that are downloaded (via certain applications only, such as Safari, Mail, or iChat). The first time you try to open a flagged file, you’ll get a dialog box asking if you’re sure you want to run it. If you choose to open the file, the flag will be removed and File Quarantine won’t bother with that file anymore.

The intention here is to provide an extra layer of security against malware, forcing you to consciously open a downloaded file rather than letting things execute themselves behind your back.

The intention here is to provide an extra layer of security against malware, forcing you to consciously open a downloaded file rather than letting things execute themselves behind your back. However, File Quarantine doesn’t really have any customization settings, and is definitely prone to “mindless click throughs” after you’ve seen the dialog for the thousandth time.

The Next Logical Step

Enter Gatekeeper. With OS X 10.8 Mountain Lion, Apple has introduced what amounts to a much more customizable and useful version of File Quarantine.

On the surface, the most recognizable feature of Gatekeeper is the option to run software according to one of three different tiers of security. You can elect to have OS X permit software from:

• Mac App Store: This option is relatively straight forward, and would probably offer an experience quite similar to iOS. Software downloaded from the Mac App Store will execute seamlessly, while any applications downloaded through a browser or from an email, for example, can simply see themselves out via the Trash.
• Anywhere: This option is also pretty straight forward, as it is most similar to how OS X operates now (because you will still get the File Quarantine warning message, even if you select this setting). With Gatekeeper essentially turned off, you’re free to download and run applications from anywhere you please.
• Mac App Store and identified developers: I’ll go a little deeper on what it means to be an “identified developer” a little later, but basically, this option seems to offer the most optimistic marriage between security and freedom. It will allow applications downloaded from the MAS as well as from developers OK’d by Apple to run on your machine.

The radio buttons at the bottom of the window have been added to the Security preference pane in Mountain Lion

How Does It Work?

This is where things get a little tricky, but knowing how Gatekeeper operates behind the scenes will help you make the best decisions in protecting your Mac.

Steven, in the engineering department at Panic (one of my favorite developers), published a blog post back in February with a brilliant (and very well-written, in normal, human language) explanation of how code-signing works, but I’ll do my best to briefly explain it in a technically digestible way. Ready? Here we go.

Essentially, code-signing is a form of encryption applied to executable data and is primarily used to verify the origin of that application. Here’s how it pertains to Gatekeeper:

1. Signed code involves the use of a pair (“public” and “private”) of keys (a very large number). The owner of the private key, the developer, can “sign” the code, which is basically a stamp to prove it’s authenticity. The public key (which is most often held by the end user) can then be compared with the signature made by the private key. If the pair is a match, you can be reasonably sure that the software came from the developer, and wasn’t tampered with before it arrived on your computer. From the aforementioned Panic blog post:
   

   Anyone with that signature and my public key can then be almost 100% sure that data came from me, and that it was not modified by any third-party along the way. The data could’t have any virus or vulnerability injected into it, because then the signature would no longer match the data.

   Apps that don’t contain signed code (and are therefore prone to undetected malicious tinkering) will only be allowed to run on your Mac if you select Anywhere in Gatekeeper.

2. If the developer in question has registered with Apple for a developer ID, Apple has in essence approved the software. The app’s users can then trust the origin of the application, while still being able to obtain the software somewhere other than the App Store (usually, directly from the developer). Apps from identified developers with signed code (along with App Store downloads) are allowed to run on your Mac when the Mac App Store and identified developers option is selected in Gatekeeper. This is the default option in Mountain Lion.
3. If the developer is part of Apple’s developer program, has signed his code, and submitted it to the App Store for approval, and Apple has deemed his application safe to use, Apple will append an extra signing, or layer of encryption. These apps are the safest of all, and are the only apps allowed to run if Gatekeeper is set to run Mac App Store apps only.

Malicious code still has the potential to get through, since developers are in no way vetted before obtaining the ID.

I have plenty of opinions about what the default option means for Apple, users, and developers (many of which are reflected in Steven’s post on the Panic blog), but this isn’t an opinion article, so I’ll stick to the facts.

What is important for you to know is that obtaining a developer ID requires very little effort on the part of the developer. The implications of this little side note is that while Apple may have originally OK’d a developer, malicious code still has the potential to get through, since developers are in no way vetted before obtaining the ID.

The good news is that Apple can (and likely will, we hope with due swiftness) revoke a developer ID

The good news is that Apple can (and likely will, we hope with due swiftness) revoke a developer ID in the event that an application egregiously violates the trust of the end user.

And what happens to applications that try to run against the boundaries established by Gatekeeper? A warning dialog is displayed to let you know that it doesn’t comply with your security settings, and your only option is the “OK” button.

This may seem cold, but the feature is fully overridable. And with Mac-based malware becoming increasingly common, it can be a pretty useful tool for protecting your machine.

OS X Mountain Lion gives you the cold shoulder when it comes to running unauthorize software, but the feature is completely overridable.

Final Thoughts

I realize that Gatekeeper may be among the more boring or mundane updates in OS X Mountain Lion, but for me it’s a very interesting development in the Apple world. We have been given the tools to employ the level of security found on iOS devices on our Mac machines, but more importantly, the decision has been left up to us.

Ideally, you now know more than you previously did about Gatekeeper and the powerful security included in Mountain Lion, and you’re now equipped to use your system in the safest way possible that suits your needs. Let us know how you plan on utilizing Gatekeeper, or even your general thoughts on Apple’s decision to include it in this update.