Why you should migrate your 1Password data to the Agile keychain format now
There is a lot of confusion, misconception, and mystery about the OS X keychain in general and I want to take the time in this post to clear some of those up as well as talk about 1Password's interaction with OS X formatted keychains. When I was at Macworld Expo in January, I had several people come up to the booth and ask "how is this different than the built in keychain?" and it is a question we get a lot. The problem with answering that question is that there are varying opinions and ideas as to what OS X keychains are and exactly what they do. So I would ask "what do you think the keychain does?" and I got a lot of different answers, several "I don't really know" responses, and some pretty defensive postures to the question and I learned it is not the best question to ask. However, it really is at the root of the issue around this topic. So I want to start with a simple example of what happens when you save your mail account password in the default keychain and expand from there.
You add a new POP/IMAP/MobileMe mail account in Apple Mail and it has an option of saving it in the keychain so that you do not have to type it in every time you want to check your mail. This is a good thing. However, let's explore what really happens here a little deeper and talk about the tools used to do so.
First, there is the keychain itself. The keychain is a file located in your user folder. Specifically, you click on your home folder, double click on Library, and you will see a folder named Keychains. Double click on this folder and you will see all the keychains for this user account. Each user account has a different set of keychains although system wide keychains are allowed and are stored outside the user account folder structure. In many cases, you will see only one keychain and it will be named login.keychain. You can have as many keychains as you want to create but only one keychain at a time can be designated as the default keychain. You cannot tell which one is the default keychain by looking in the folder so I'll talk about this later. What is important to note though is that the keychains are only files. The file contains data formatted in a special way as defined by Apple's keychain specification documentation. The format is actually open source and can be inspected by anyone. You can think of these as database files. Files that contain your data in a defined format. What is most important to note though is that by themselves, these file do absolutely nothing.
#1 Keychains files do not DO anything - They are simple database files - They contain the data
You can think of this the same way as photos, MP3s, and videos you have stored on your Mac. The files themselves do not do anything but hold the data. You need to load iPhoto, iTunes, and Quicktime in order to view or play these files. This leads us to Keychain Access.
Keychain Access is an application provided by Apple that allows you to view and edit the contents of a .keychain file. Keychain Access understands the special format of a keychain file and can read and write to the contents of this file the same way that iPhoto can read picture files and allow you to edit and save them. When you load the Keychain Access utility you will see a screen similar to this:
In the upper left pane is a list of the all the keychains. The one that is listed as bold is the default keychain. Normally this will be the login.keychain. This is important to note for the next part but keep that in mind for now. The pane below lists the type of items that Apple has defined in the keychain format and include cerificates, keys, passwords, and secure notes. The largest pane to the right shows the list of all the items stored within the keychain. I have highlighted the mail passwords I have stored for some Gmail accounts that I used in Apple Mail. I can examine the contents of an item by double clicking on it. Doing so will open a window that looks like this:
This shows all the fields that are available for a password type in a keychain. You'll notice that it contains fields for a title, URL, notes, and password. The password field is actually concealed and requires you to check the box next to Show password in order to view it and here's a fairly unknown secret: it is the only field that is encrypted. That is right, if you were to examine the contents of the keychain file with a text editor you could see the other fields plainly. However, the password field is encrypted and you would not be able to see the password in plain text. This will be a fairly important point that I will talked about when we get to how 1Password uses keychain. If you click on the Access Control tab at near the top of the window you will see something similar to this:
The keychain format allows for Access Control Lists on a per item basis. ACLs determine what what applications are allowed to access an item. This prevents applications from accessing passwords that do not belong to it. In the example above you will see that Apple Mail and Keychain Access are the only applications allowed to access this entry. At the top of this window are options for allowing access to this item. For example, if Safari tried to access this item you would be prompted to allow access since the Confirm option is enabled. (Confirm is the default setting for all items) The important part here is that Keychain Access is the means to view and edit the information in a keychain. Although you can manually add items to a keychain, they can only be used by applications that are allowed to read them. Keychain Access doesn't actually use any of the passwords and although it can display the data it too needs permission to view the encrypted data.
#2 Keychain Access is a means to view and edit data stored in a keychain file but does not make use of the data
So how do we get all these password into a keychain file? If we needed to manually create one of these entries every time we wanted to use a new password it would be cumbersome. Of course we never do that because our applications do it for us. When we created the Gmail account in Apple Mail it stored the password in the default keychain because it has been programmed to do so. The coders who created Apple Mail had to create a special section of code for interfacing with a keychain and to store their data in the keychain. You'll notice that in the iTunes store you always have to type in your password because iTunes did not program in the ability to store that password in your default keychain. It is up to applications to store passwords in the keychain. The keychain file itself does not do it as it is just a database file. Keychain Access does not do it unless you manually create an entry and even if you did then the application would need to have been programmed to read it from the file. Since you can have more than one keychain and applications do not want to figure out which keychain it needs to go saved in they normally query the system for the default keychain and store them in that keychain.
#3 Applications store passwords in keychain files
So to wrap up our mail example:
1. Apple Mail saves the password in the default keychain
2. The keychain file contains this data and the ACL allows Apple Mail to read it without prompting you
3. You can view, edit, or delete this item using Keychain Access
Hopefully this sheds some light on the relationships between the different components involved, but I want to talk about one more real life example. The Safari browser has a feature named auto-fill for password management. How it is works is it automatically remembers passwords for sites you visit, saves them in the default keychain, and then restores them automatically when you return to the site. This sounds wonderful and if it actually worked everywhere then 1Password would be less useful. Of course it doesn't handle multiple logins for the same site, separate logins for different sub-domains, and has many other limitations which drove us to develop 1Password. The only point I want to make is that in relation to web passwords it is the Safari application itself that is storing passwords in the default keychain when auto-fill is enabled. The keychain file itself is still just a database file and Keychain Access has no role in this. :)
#4 Safari stores passwords in the default keychain when you use it auto-fill feature
Now I want to talk about how this relates to 1Password. When 1Password was first created we decided that instead of creating our own database file for storing on the data, we could just create a separate keychain file and store our data in that file. After all, keychains are just database files. The advantage to this was that all the code to do this is already published and Cocoa (the OS X development tool) already included all the framework for reading, writing, and manipulating keychain entries. So 1Password created a 1Password.keychain file that was separate of the default keychain and was only used by 1Password. You could use Keychain Access to view the information but the only applications allowed to access the entries in the 1Password keychain were 1Password and the browsers that made use of the entries.
#5 1Password uses a separate keychain that is independent of the default keychain
However, as shown above the number of fields was limited and only the password field was encrypted. We wanted 1Password to be able to store more information and we wanted it to be encrypted. The way we handled this was by "stuffing" the password field. We gathered all the fields we used and all the additional information and we encapsulated them in JavaScript Object Notation (JSON) and then basically made this entire object the password. You can read more about JSON here. JSON starts an object with a { symbol so if you were to examine a 1Password item with Keychain Access you would only see the { since it only displays one line at a time. Apple allowed for but did not really consider multi-line passwords with Keychain Access. If you press the down arrow you will be able to scroll through all the data that 1Password is stuffing in the password field. This is also why you should never edit the password field in the 1Password.keychain using Keychain Access. This stuffing method allows us to create many more fields for an item as well as insuring that this data will be encrypted.
#6 1Password extends the OS X keychain format by stuffing a lot of extra information in the password fields
Another advantage of using the OS X keychain format was that .Mac/MobileMe syncing was available so we didn't have to worry about developing a sync engine. The OS X keychain served us well but after using it for a very long time we learned a lot more about its limitations and quirks. For example, .Mac/MobileMe keychain syncing has a serious issue that causes the encrypted data to become corrupt. To date Apple has provided no explanation or fix for this. You can read about this in this forum thread. When we created the 1Password Touch application we also learned that adding an application to the Leoapard firewall causes it to re-confirm the ACLS on all the items in the keychain associated with that application. This may be fine for our mail example where there are only a couple of entries but when you have 500 items it is unacceptable. We also started thinking about how we would ever be able to support file attachments to items at some point in the future and came to the realization that the limitations and problems with the OS X format were too great and that we need to migrate to a new format. We developed the Agile keychain format to take the strengths of the OS X keychain format forward while leaving the limitations and problems behind. You can read more about the design here. We've spent a lot of time developing it has it has been very well tested and received at this point and encourage everyone to switch to the new format. We have an article describing how to do this here.
#7 Agile Keychain format provides the features needed to expand 1Password's functionality and provides great syncing options
The last point I want to address in this post is in regards to some users' being hesitant to switch to the new format. Some of this is founded in different ideas about how keychains work and I hope I have addressed that above. Others are concerned about being able to move backwards if they ever decide to stop using 1Password and have the notion that the 1Password.keychain can be used by other applications. I want to clear that up a bit but I first want to state that we have no intention of ever locking anyone's data into 1Password. This is why we have export options including a generic CSV file that can be read an interpreted by tons of applications. You can always get your data out of 1Password. However, what you cannot do is use a 1Password.keychain to go back to using Safari's auto-fill feature with it. Although the 1Password.keychain file is a standard OS X keychain that can be accessed with Keychain Access, the data stuffing in the password field that I mentioned above means that most applications wouldn't know what to do with the JSON data even if they had permission to read it. Not to mention that Safari's auto-fill only stores domain names and not full URLs for its entries. So there is not a path backwards using the 1Password.keychain file. You would need to export the data and import elsewhere and you can do this with the Agile keychain format too. Clinging to the old format doesn't buy you anything but the potential to experience issues like those described above.
#8 There are no advantages to using the OS X keychain format for your data - Only disadvantages compared to the new format
In summary, if you have not already done so, please migrate your 1Password data from the OS X keychain format to the Agile keychain format. You'll be glad you avoided any issues described above and you'll be ready for future features in 1Password.
