Leopard+Firewall+iPhone Sync Issues

We have been really busy around here with the new iPhone/iPod touch version of 1Password. Overall things have gone as smooth as one could expect for a new product on a new device. Unfortunately, we have found some serious issues that need the be addressed.

The most serious issue involves the OS X Keychain Service and how it detects if an application is allowed to access an item in a keychain.

How Keychain Permissions Work

If an application tries to access an item in the keychain that it is not explicitly allowed to access, OS X will present a dialogue to the user asking for confirmation:

Normally you will never see this confirmation dialog as we explicitly add 1Password and all the browsers to the Access Control List (ACL) of each item when we create it. If any application is updated after the ACL is created, the OS X Keychain will detect this and prompt for confirmation that the updated application is still trusted:

Clicking Allow will cause the keychain to update every item in the keychain for this "new" application. Technically, the items in the keychain are not modified directly, but rather the Code Equivalence Database will be updated to say the updated application is "equivalent" to the original application, thereby making the existing keychain item ACLs valid.

Leopard + Firewall + iPhone Sync Issues

Since the iPhone/iPod touch version of 1Password was released, there have been several reports that after upgrading 1Password users were prompted to allow access for every item in the keychain.

At first we could not figure out why this was happening as the ACL of each item should not be affected by upgrades. Clicking Allow once should have been all that was required. After some digging we found the problem was related to how Leopard handles WiFi syncing with the iPhone/iPod touch application when firewalls are enabled.

On Leopard systems that have the firewall enabled, WiFi syncing will cause Leopard to implicitly codesign the 1Password application. This happens because the 1Password application found on our site is not signed (more on this later).

Once implicitly signed, the keychain treats 1Password as a completely new application and proceeds to prompt for confirmation on every single item in the keychain. Worst of all, after any update, Leopard will re-codesign 1Password, causing the cycle to repeat itself.

Planned Solution

We have discussed solutions for this at length, and think we found the best solution given the circumstances.

One plan was to sign the 1Password application. We have been dragging our feet on this as it would require every single user to click Always Allow on every item in their keychain. We almost decided to do this anyway, but elected not to as we are concerned there could be unforeseen consequences. We could end up with a new set of problems, and still not have an acceptable work around available.

Another plan was to release two versions: one signed and one not. This would be a maintenance nightmare and is easily broken if a signed version is accidently upgraded to an unsigned one. Because of this we really do not want to go this route either.

What we decided to do was to finish the new Agile Keychain format which has been under development for the past year. The Agile Keychain will be an alternative to the OS X Keychain and will provide several new features in the long term, including better synchronization with MobileMe, the ability to add file attachments to items in 1Password, and several other amazing things we have planned.

Because of the urgency of this Permissions Issue, we decided we will release the Agile Keychain format before all these new features are completed so that those experiencing this problem can use the Agile Keychain as a workaround. This new keychain format is undergoing internal testing at the moment and should be available in a Beta release soon.

Potential Workarounds

For those awaiting this fix, you have a few options, albeit each one is far from perfect:

• Disable all automatic updates. Only upgrade when we post the "go ahead" in a post here or from the Agile Newsletter.
• Disable Betas in the automatic update. The plan is to not release another "official" version until this is resolved.
• Turn off your Firewall. We know this sucks and are not advising you do this, but wanted to list all available options.
• (Unconfirmed) Stop using the iPhone sync. AFAIK the implicit code signing only happens during the sync, so if you skip the sync you should be in the clear.
• If you must have the updates and must have the firewall, you will need to bite the bullet and click Always Allow on every item after every update.
• Downgrade to Tiger. I'M JOKING!! :)

Again, we know that none of these are acceptable long term solutions but hopefully at least one of them should be able to keep you going until we have the final work around available.

We recommend workaround #2: turn off Betas in the automatic updates until we resolve this issue.

Conclusion

The plan is to not release another "official" version until this is resolved.

We will, however, be having many Beta releases over the coming weeks as we start rolling out the new keychain format. Therefore, if you have this issue, be sure to double check your Update Preferences and disable Beta updates.

Sorry for the trouble, folks! We will get this resolved as soon as we can.

Comments

[Robert Kosara]

Does that mean that if I say no when I get asked whether to allow incoming connections for 1Password, it will stop asking me for every item? Or do I need to reinstall the program? I can live without iPhone sync for now, if that makes working with the program a bit less painful. Turning off my firewall (or downgrading to Tiger ;) is really not an option for me.
Anyway, good to see that you're on top of this, and I'm looking forward to being able to sync again. Attachments sounds like a great idea, that was something I thought would be a very useful feature (to store tax forms, etc.).

[Roustem Karimov]

Robert, turning off Wi-Fi syncing and re-installing the application should help in your case.

Just remember that, with the firewall enabled, the moment you turn on the Wi-Fi syncing, Leopard will automatically sign the application. This will lead to the permission prompts for every keychain item.

[Ben Dalton]

I couldn't figure out what was wrong. I just kept saying to myself "wait, that's not right" while I was working and needing passwords.

Glad to see you guys are on top of this. Thanks for the excellent product and great support!

[M. Jordan]

"The Agile Keychain will be an alternative to the OS X Keychain"

Does that mean that after that "workaround" there will be two Keychains again ? No OS X-Keychain integration anymore ?

Then i can reenable my KeePassX again, which works (for me) that way and (cross-platform-) syncing is really easier that 1password.

Can't understand that trouble. It's simple: old versions work, so the way you "enhanced" the new versions is wrong. IMO i don't need "cool new features", i need some more cross-platform usability other than my1password (a PC version), rock-solid security and stability.

I am still a "didn't bought yet" user, and i am still happy about that. I don't think that the small advantages of 1password over KeePassX will win MY comparsion. That new issues and the solution of creating some own keychain format aside from the Leopard keychain will give some very bad minus-signs to 1Password.

M. Jordan.

[David A Teare]

@M. Jordan: You are absolutely right! We need to have much better cross-OS support, rock solid security, and stability. The new Agile Keychain is designed to provide exactly these things. Let me take each point in turn:

Cross-OS Support: The OS X Keychain is by definition OS X only. We looked at moving OS X Keychain existing code to Windows and Linux and it was not a simple task. The new Agile Keychain will work on any platform, as well as enable us to have a plugin for KeePass.

Rock solid security: We will be using OpenSSL AES for encryption. The OS X Keychain is great because it is Open Source, supported by Apple, and is robust as it used on every single Mac. OpenSSL is Open Source, used on 10-100 times more machines than there are Macs, and is actively supported by a huge community.

Stability: The encryption algorithms are very stable b/c of OpenSSL, but the main stability/robustness issues are usually related to syncing. .Mac syncing of keychains used to be very good but since Leopard and the MobileMe change over, it has had many issues, including duplication of data, data loss, etc. The Agile Keychain will be side stepping these issues by making the sync process very simple.

Hopefully that gives you a bit of insight on where we are headed and why. Of course, we need to give A LOT more information about the new format before anyone can judge if it is acceptable to them. The post above was not meant to introduce the new keychain format, but rather explain what we are doing to address these permissions issues. Once the new format is ready, we will have a detailed write up on the architecture of the new format, its benefits, how to sync, etc, etc.

[bassclef]

(As I'm sure you know) there's a lot more to security in these sorts of applications than just the encryption. Apple has had years to get the OS X Keychain right (no VM on the pages with decrypted keys, etc). I hate to see you rush a replacement to address this issue, and in a rush isn't really the best way to architect this sort of thing in the first place!

So, put me down for one who would prefer that you sign your application (which has other security advantages!) and continue to use the OS supplied Keychain.

Has Apple made any comment on the requirement to acknowledge every item separately (assuming you have a way to ask)?

[David A Teare]

@bassclef:You're right we don't want to do this in a rush. In fact, we have been working on this for over a year already, but even given that we will not be forcing anyone to use the new keychain. Roustem has things architected to allow you to switch back-and-forth between the keychain formats so you will be able to make the decision on which one is best for you.

This one single issue is not the only reason we're looking at an alternative; we simply decided to make the new keychain available earlier than planned b/c of this permissions issue. Over the next 6-12 months you will see many more features that would not be possible with the OS X Keychain.

[Chijo]

First of all, I'm glad you're working on this fix.

I thought I had it handled for now by using v.2.7.2. Everything was going OK and then one day it started asking me for permissions for all of my 800 entries when I do a search.

I do have beta updates turned off but could you please clarify which version is the "safe" version to be using? Should I upgrade beyond 2.7.2?

[Jesse]

Hi David, thanks for listing all the options you are considering and what the pros and cons of each are. I appreciate that insight into your thinking. I'm not sure if you wanted opinions or 'votes' but here's mine anyway. =)

I strongly prefer the signed application solution. I personally won't use a custom keychain storage format, even if it gives great new features and provides a workaround for a Leopard quirk.

Security is so hard to get right, with so many opportunities for gotchas that go unnoticed. I believe the best chance for good security comes when a product is used, analyzed, and tested by the largest number of people. I chose 1Password because it uses the Mac OS X Keychain, which is high profile enough that I expect security flaws to be reported widely and fixed quickly.

As bassclef said, securing a system requires a lot more than just using secure encryption algorithms; it's the whole system that needs to be secure.

Have you talked to Apple about this problem? It seems as though a newly signed application ought to be able to be added to the Code Equivalence Database as equivalent to the old, unsigned version of the application, just like any other new version. Maybe it's just a Leopard bug which can be fixed by Apple?

[Bob]

Quote: "One plan was to sign the 1Password application. We have been dragging our feet on this as it would require every single user to click Always Allow on every item in their keychain."

But wouldn't this be a one time issue for current users? New users from a fresh install would also have this one time requirement. After that it would be a non issue, assuming it is possible to work out what Jesse mentioned in terms of having new versions of an application "inherit" equivalence(which seems reasonable given the huge number of signed applications that are out there - doesn't Apple achieve this themselves somehow?).

Like Jessie I would strongly prefer a working signed app that leveraged the existing OS X Keychain and not a proprietary format. Maybe I'm not fully understanding here, but I see it like this - 1Password is the most popular tool of its kind on OS X. It houses very critical data(or at least the keys to that data). This puts a target on your back as developers. If someone can compromise 1 Password, via malicious code that exploits something you may not have thought of in your proprietary system, look what they get! I can't imagine the fallout of such a breach becoming publicly known. It is from that vantage point that I believe my preference would be to stay behind the "shield" of Apple(such as it is) and keep yourselves tied to the Keychain.

[Dave Teare]

@Bob, @Jesse: Thank you both for the frank feedback. I appreciate it! You both make very good points and I agree that switching from the OS X keychain is a big undertaking and should not be taken lightly.

It is up to us to prove we have gotten things right, and there will be a *lot* more on this in the coming weeks. With that said, it will be your choice on whether you want to stick with the OS X keychain or use the Agile Keychain. We will outline the Pros and Cons on each format and let you make the choice that best suits your needs.

Cheers!

--Dave Teare
Co-author of 1Password

[Tommy]

Ugh. IMO, integration with the OS X Keychain was a strong point of 1password. It allows the users to feel good about not being locked into another product, the code is well vetted, and it allowed the 1password devs to focus on bringing more value-add to the app itself, rather than reinventing the wheel. Yet, reinventing the wheel seems to be the path that has been chosen. It seems unlikely that these issues will be issues long enough to justify developing an alternative storage format. At this point, I'm still on Tiger, and I don't own an iPhone or iPod touch. Maybe I'm not representative of the target market, but I don't feel this is a change for the better.

[Dave Teare]

@Tommy: I understand your point of view. One could certainly argue that the OS X keychain enabled us to add more value rather than reinventing the wheel, but after the last few years I can unequivocally say that things are now reversed. Something as simple as attachments is not practical with the OS X Keychain, not to mention the bigger issues of syncing and cross platform support, such as Linux, Windows, and iPhone.

1Password uses the OS X Keychain more than any other application and has pushed it to its limits. We have not shared all the reasoning for the switch yet and I think you will be pleasantly surprised by what the change will enable. We will be detailing all these things soon.

Other than new features, we need more flexibility. Let me explain. You see, the Leopard upgrade was a very rough ride for us, and the iPhone was even more so. Many hidden assumptions in OS X and Apple specific issues caused many users to experience so much pain that I am afraid we lost them. If we had an alternative at the time, we could have offered people a work around, even if just a temporary one.

So, maybe all the pain is in the past and won't be repeated? That is actually what I thought after Leopard, so I am more pessimistic this time around :) Also, there are two major changes coming in the next year. First, we will be signing the 1Password application. Second, Snow Leopard will likely be released. Both of these bring the potential for issues, and frankly we want to have a back up plan.

I think I will use this reply as a basis for a new blog post :)

[Tom Craft]

First, 1Password is, without a doubt my #1 used applciation, and my #1 reccommended application.

Secondly, I applaud how open you guys are about everything. Updates are fast and furious, change logs are detailed (Apple should take some notes on that), and you guys go out of your way to let us know what is going on.

Personally, I would rather not see 1Password go cross-platform. I am a System Integrator for one of the largest Single Sign On platforms for Healthcare, and its downright comical to see their mouths just drop when I show them 1Password. True, 1Password is web only, whereas our product is applications and web pages, but 1Password is clearly a better product. And ours costs thousands of dollars (heck it takes one Integrator 8 weeks to get a single site up and running). So call me selfish, but I like the fact that the truly excellent software is OS X only. :)

As for the above issue, I'm not sure the exact details of signing as I'm not a developer (yet anyway). But it would seem to me that would be the ideal solution. I also wonder what Apple has said about this, as I agree with one of the above posters that I can't think of why the app would be treated that way if it was a signed app.

Anyway, I'll avoid writing a novel, but know that at least one customer thinks the new Keychain sounds interesting and can't wait to try it out and see what unique things it can do.

[David]

What is the status of this?
I'm getting tired of not being able to use 1Password as it was designed.
Also, I think that until this issue is resolved it is misleading to have the phrase "fully Leopard compatible" on the 1Password page. If I purchased this app based on that phrase only to have the keychain issues I'm having now, I'd be very angry with you.

[Ian]

I upgraded without being aware of this problem. Is it possible to downgrade to an earlier version?

[Dave Teare]

@David: The new Agile Keychain is very close, and in fact was released into Beta yesterday. If you are willing to try the *beta*, you can see this post for setup instructions.

@Ian: To be honest I am not 100% sure how Leopard handles the implicit signing of the 1Password application. We have had mixed reports about the success of downgrading. Given your situation, I would recommend trying the Agile Keychain that I linked to in the comment to David.

Please note that upgrading to the latest Beta will cause this problem to occur. Turning off your Firewall during the upgrade and migration process should theoretically avoid the problem entirely. I will be posting more detailed instructions once I have had time to test them personally.

[Alex Taylor]

I too would like to chime in that even with this new format launched early, communicating with Apple and testing the possibility of a workaround/change in behaviour for Leopard to get a signed version of 1Password into the Code Equivalence Database so that a signed version could be released without users having to click allow on each item.

I too will be extremely hesitant to switch to the new Agile Keychain format.

I currently use my1Password for syncing between home and work and find it currently quite flawless for my uses, I don't use the web frontend much but I would prefer that than needing to install a program on a machine if I just wanted to get a password. For that reason I really like my1Password (and the fact that everything is encrypted and decrypted clientside)

I think you should still pursue the code signing route.

[Dave Teare]

@Alex: Thanks for the feedback!

From what we saw on the discussion boards, Apple is well aware that newly signed apps need user confirmation to allow access to their OS X Keychain items. From the keychain's perspective, it is really a completely new application and so it makes sense that user confirmation is required.

For 99.9% of applications this is fine as they only store a few items. 1Password is quite unique as it has pushed the limits of the technology and stored hundreds or thousands of items into the keychain. If I were in Apple's shoes I would not want to invest the time to make a change like this for a single application. The possibility exists that they might make decide to invest the time to make the change, but it would take months to develop and test the change, and then wait for a new release of OS X. People in this bind need a guaranteed fix, and soon.

We are very close to releasing version 2.9, and in fact I just posted about how it contains the Agile Keychain and how this new format opens up the possibility to sync your keychain without relying on MobileMe.

With that said, please note my comments above (from a few weeks ago). We will be signing the 1Password application eventually. Code signing has many benefits and we plan to start using it in 1Password. We elected to develop the Agile Keychain as a first step so we would have a back up plan just in case code signing had any unforeseen affects on anyone.