Security features like this works, until it doesn't.
When I installed WireGuard on macOS, I was happy to discover that it stores private keys in the system keychain. Accessing the private key requires Touch ID.
Then one day after a system upgrade, the WireGuard app crashed on startup with:
[APP] Unable to open config from keychain: -25300
Then the app deleted all profiles because they are all "orphaned", apparently for "keychain consistency" [0].
From then on, I just use a text editor to edit my WireGuard configuration files, and import them to the app. The bug still exists to this day [1].
So many of the keychain APIs are "deprecated" (read: marked as "we will nuke this API whenever we want") and have been for a decade or more, but have no replacement either.
A lot of the Security Framework came from Classic, hell, the idea of a keychain came from Classic! Just a mess.
I'm using WireGuard myself. Not on MacOS, but on Windows/Linux/Android.
The reason I went with WireGuard to begin with, is due to how secure it suppsedly was. Having long standing unresolved issues like that is a bit worrying. I'm still most likely safe, but it's not exactly reassuring, either
This is likely just that if someone chooses an OS that requires a special HW just to build a release with no easy cross-compilation possible, and to pay a gatekeeper just to publish it, and that can't even run on an emulated computer properly, they can't expect FOSS maintainers will be bothered with much more than "here's a source code, build it/test it/do QA yourself if you like" for their particular pet platform.
For platforms where development is unhindered by arbitrary obstacles like these and others, wireguard works fine. :)
This is not a feature built for security nerds. This is a feature built for corporate compliance box tickers, and it works precisely as wanted in corporate environments, where all lost secrets can be easily replaced after visiting the helpdesk room personally.
Security nerd here. I would have expected it. But only in the sense that I have no faith in this industry to write secure software and protect from data loss.
> You linked a commit and a Reddit post. Is there a proper Github issue as well?
It's funny you mentioned that. The GitHub repository is a mirror of https://git.zx2c4.com/wireguard-apple. You cannot create issues there. You're supposed to send a bug report to the mailing list, which is enough friction that none of us did so far.
Its unfortunate no updates for Wireguard macOS have been pushed out for 3 years, App Store is still showing 1.0.16 as the latest version – as also confirmed on the Wireguard website at the latest version.
No doubt macOS APIs have moved on a lot in 3 years and so perhaps bugs that are being seen are simply a reflection of the lack of updates to keep up with API changes ? Just thinking aloud here, I don't know if the concept of OS API changes are applicable to this particular PR / bug.
> I've passed on reporting so many bugs, because I refuse to create a Microsoft account to do it.
99% of the time it's perfectly fine to just email the maintainer with a bug report and maybe a patch.
> In fact, I don't understand how sending an email is high friction.
Judging from the fact that after so many years, such a widespread (as seen in the Reddit post and here) issue still hasn't resulted in a single bug report in the mailing list, I'd say most people (including myself) do indeed consider using a mailing list to be higher friction.
If I'm not mistaken, unless you're already CC'ed by a participant, you'd have to subscribe to the whole mailing list just to receive notifications for a single thread. For an "issue" on a Git forge, participation is usually as easy as upvoting and then clicking the "Subscribe" button.
Assuming you're already using MS Github. Anyway, even if I was, a lot of Github project require filling out an absurd Simon Says form. Or that scene from Indiana Jones where he takes a wrong step and falls through the floor. That's nothing to do with email or Microsoft specifically, it's just another reason I almost never bother reporting bugs. Talk about friction.
Yeah, I hate these forms (issue templates) too. Every time I have to fill such a form, I always feel that they are discouraging me from writing the bug description in a natural, coherent way. I'd spend more effort to make sure the description is clear and contains all necessary information, if there's no issue template requirements.
Yes, but most people won't. Most people will submit a bug report with important information missing that means we have to go back and forth. Also, there is often specific debugging information that people would not even know would be useful.
In the current age, its useful because AI will submit tickets using the CLI that don't use the form, so I can immediately close and lock them and go on living my life.
Edit (March 2025):
Wireguard, to my surprise, has no bug tracking system like Jira or issue tracking area such as GitHub.
If you want to see this fixed, the only way to instigate change is to write team@wireguard.com and jason@zx2c4.com.
I decided years ago that I want to protect my keys with a hardware device that is independent of the vendor of the machine that I'm currently using. I also want to have backup keys and I want to be able to freely move them around various physical locations. Or have a backup shipped to me across the world if necessary.
YubiKeys were the only solution that fit these requirements, so I went with those. I have been using them for the last ten years or so and I've been very happy with this solution. See https://github.com/drduh/YubiKey-Guide for a description of how to make your RSA keys stored on YubiKeys work with GPG and SSH at the same time.
One fundamental problem with agentic development is that accessing the secure enclave key requires human presence. If you're running headless and aren't logged in, Secretive (not sure about others) will not be able to access the enclave, even when you only chose "notify" and not "touchid".
This is why I'm skeptical that the approach can scale reliably, even though I'm relying on it today. (Yes, PATs would fix this, but the enclave model is what I'm trying to retain for now.) What I've done instead is wrote a small daemon that sets the SSH authentication socket and does agent forwarding. This works when I'm traveling with laptop...but not yet from my mobile phone.
NB: I'm not using Rex instead of herdr and because Rex uses its own protocol, I always need to have one SSH tunnel to reset the remote agent socket. Not optimal, but it works. All of this feels wrong from a UX perspective, and it feels like I'm holding on too hard on secure enclave for what my use cases are. Feedback appreciated!
I'm not using the secure enclave, but I've been using YubiKeys for many years, and now that AI has arrived, I consider the above to be a good thing.
This means, for example, that unless I am physically present with my key inserted, activated, and touched on every operation, an AI agent cannot access any of my production servers. As I do not want AI agents to access my production servers, this is a very good protection technique. I am assuming that no matter how well you try to sandbox and restrict your AI agents running on your local machine, they might eventually be able to access any file in the file system.
1. The Secure Enclave being inaccessible when you're not present is a feature, not a bug. If you need to run headless then its not the correct solution to your problem and you'll have to find another.
2. You don't need an SSH agent proxy, you can use a `match` block in your `ssh_config` with something like `host * exec "test -z $SSH_TTY` to check whether you're operating over an SSH connection and select Secretive's SSH agent if you're using the machine locally.
I got a similar idea recently. I was becoming annoyed with the coding agent, running on my laptop, being unable to use sudo (with my supervision) when ssh-ing to my remote server. I wanted to build this as a PAM module.
I started building this but after a while I googled (well… asked an agent) a bit more and found out that it already exists: https://github.com/z4yx/pam_rssh
I’ve been a happy user of Secretive for years, including for Git commit signature which it make it a separate key with interaction but not for pull / push. I find the trade off ideal for GitHub.com and have other keys with different setup for servers.
SSH keys in plaintext was one of the first thing that bagel [0] our developer endpoint secret detection focused on. Then we embedded bagel as a library in our developer endpoint protection [1].
The author says he doesn’t like the idea of having keys sitting on his file system ready to be snatched by bad actors.
That’s why they should be passphrase protected.
You use ssh agent to store the keys in memory and for convenience you can store the passphrases in keychain. MacOS OpenSSH has the UseKeychain config directive for that.
I use Yubikey/FIDO2 myself, but not because private keys cannot be protected on a file system.
Secure Enclave is still more secure in that it should be impossible to access the private key. Password protected keys are still vulnerable to malware that can read the keyboard, trick you in to typing the password, or that have exploits in macOS to read the memory.
Isn’t the macOS keychain user-readable by default?
Assuming the attacker has user rights (which it needs to read the key) it could also read the passphrase, right?
Update: I tried it out and stand corrected. ssh-add doesn’t store it in the user-readable keychain. So the passphrase is protected. (And also not recoverable by the user).
Key on disk plus a passphrase is still only 1FA. Key in the SE + an unseal PIN is 2FA. There's already plenty written about why one is better better than the other.
Do you type your passphrase every time you log into a remote machine? If not, the private key is in RAM and can be extracted by malware and stolen. If so, that's way more annoying than using touchid or some other physical proof of presence.
I used to install ssh_locker [1] (I don’t know if it still works), and have the passphrase in a keychain separate from the default `login` that I then set in Keychain Access to autolock after a set period of time. Handy for keeping the private key for a while in RAM and then autoremoved, instead of having it there for the whole session. It worked transparently and is a feature that should come with the OS.
ChatGPT says ssh_locker locked the keys with `ssh-add -x` which doesn’t necessarily remove them from RAM. Duh! It does point to suggested minimal changes to its source code to effectively accomplish that though.
I use 1Password's `op` for this, but with an allowance: my dev environment keys.
I found I was getting fingerprint fatigue. Do I need a fingerprint request every time my dev server wants to connect to dev Supabase? No. That's expected, and zero risk. There's no data there.
I worried that a nefarious request would be auto-approved by me. When you see 20 requests a day, your muscle memory just reaches for the sensor.
So I have a separate 1Password vault that a service account [0] can read. That token is in my environment as OP_SERVICE_ACCOUNT_TOKEN, which the `op` tool understands.
It also has access to a vault with 'AI-safe' keys that I want Claude to be able to access without asking me.
Now, these requests work invisibly. And when I see a fingerprint popup, it's unusual, and I don't auto-approve by habit.
I also keep an item called '_One-day keys' in a separate vault. (The underscore just pushing it to the top of the list for easy access.)
It has prod keys, mostly restricted read-only. If required, I move that over to the open vault. Claude knows this item and has instructions to remind me to move it back out when it's finished what it was doing.
Proton Pass with `pass-cli` has a similar thing, for anyone using that. You can give agents an expirable token and get an audit log of what item was read and when.
Secure Enclave, TPM and other such security features are unfortunately so badly documented and need so many hoops to jump through to get them working, with any mistake either causing you to be locked out completely or to not give you the protection you wanted. There really should be a better way to make these technologies usable...
One of the problems I constantly face is that the SSH configuration is not portable when I use these esoteric things. Even with Secretive (which plugs a simple SSH socket), remote agent forwarding sometimes fail. (ie. Due to the old version on the server or configuration issue, or something else setting SSH_AUTH_SOCK...)
Moreover, it is not possible to drop these files into a file-sharing folder, like OneDrive or iCloudDrive or SMB share, and share them with many hosts at the same time. (Especially when different operating systems are involved...)
I found the hacker way simplest and easiest; put a basic Makefile with bunch of rules, have a "central" host/machine to distribute the keys and configuration from that point on...
I used yubikeys for ssh for years. First via the older clunky gpg as ssh key feature. Then via the FIDO api. My biggest issue was the chicken and egg issue when using this setup on a new machine or god forbid in a container etc. windows and WLS wasn’t super thrilled either.
I switched to 1Password ssh because I actually like the fact that I can simply share my keys. It’s still not perfect. Agent forwarding needs an extra configuration and depending how many keys you have and how verbose your ssh config gets it becomes also very finicky.
This solution feels for me like a slight setback because it’s a macOS only solution. If you have a Linux or windows machine next to a Mac you have to deal with the insecure key file on your hard drive again either via yubi keys or software solutions.
But I think I would take this over a yubikey. I really disliked the fact that one usb port was constantly in use.
The way I think of it is YubiKeys should be used to basically provision a computer with an approved device specific key. Whether it's a passkey or SE/TPM secured SSH/GPG/TLS key: the Yubikey holds the credential everyone trusts and therefore can be used from any device to gain access and enroll a new approved key.
Sure and pop in whenever I do a git commit? I also sign my commits. I used GPG (different key on the same hardware key) and now via SSH key. I had both a compact and bigger key. I got the compact key because my very first yubikey nearly broke. I lifted my Mac while it was connected and it cracked in the middle. I bought a compact one next which can stay inside and that one you don’t want to take out. It’s so small you’ll lose it. I know because that happened to my backup key … (that was actually the idea for the bigger one I bought but I lost the damn thing during the setup phase)
But having the yubikey bound to your Mac at all times is also stupid because it doesn’t come with a fingerprint or other biometric sensor. The idea is that you use this thing as a token not that you leave the key in the lock at all times. For that alone I would favor the enclave solution.
These days I'm mainly doing development inside VM's. Previously I was embedding Github key to each VM, but since I setup Secure Enclave (with Secretive), I'm only forwarding it's agent to VM and each git action on origin now requires approval using fingerprint reader. I think this a good way to prevent malware from gaining access to github
PAT are vulnerable, once stolen they can be used from anywhere until their (usually) long term expiry.
Better use short lived tokens with auto refresh. For example OAuth tokens. They can be used for the same use cases like PAT, but they expire after a max of 2 hours.
2 hours is plenty. The recent binance hack was 20 minutes after all. A 2 hour token is enough time to get all that can be gotten or done by a token in almost any system especially well defined ones like GH or GL.
It would be nice if GitHub or GitLab could limit a token to a given IP, but with how proxies and dynamic ips work I would imagine it would be a hassle to get it working correctly for everyone, but it’s better than nothing.
Those access token can still be stolen just the same. The Secure Enclave is better because you can ask it to authorise the action, but you can never get the key out of it, so the latest malicious npm package can’t send off your access tokens or ssh keys.
In case you want to just have an encrypted SSH key with password in keychain, you can make it work automatically when logged in (and when Mac is unlocked) by putting in .ssh/config:
On macOS use “Secretive”. It’s a small app that can generate keys from Secure Enclave and prompts you with touchid whenever those keys are being used
On iOS/ipad/(also macOS) you can also use the SSH terminal emulator Termius which has support for SE keys built in as well. So I have the set of keys synced to my VPS, this way I know that only my physical devices can log in and nothing else.
It's often the case that we make much effort into securing the key but expose it via an open agent socket at a well known path or use an open control master path for multiplexing.
Certain automation and dev tools like Ansible make it a pain to work if agent key requires a confirmation.
Weirdly enough, this is _why_ I ended up using Secretive: I needed a second ssh-agent because selecting which keys are forwarded per-host is very fiddly, and while standard ssh-agent kinda supports it, 1Password doesn't. Anyway, _a_ solution for this is having separate SSH agents, and Secretive is another agent on top of 1Password.
Not sure how many people feel like this, the risk in plain sight is a lot easier to handle than the risk hiding under the layers. You can keep shifting the storage for the key, one way or another you have to open up a channel to access to key or use the key. That channel would then become the vulnerability target.
To prove possession of a private key, one needs only to sign something with that key (this is how SSH key auth works). This is how things like the macOS Secure Enclave operate - the private key is stored in a separate hardware chip that only exposes an interface to sign something with that key, never to actually present the key. To obtain the key would require having physical access to the computer, ripping it open, and performing advanced deconstruction and analysis of that chip.
I dont want to be rude, I am aware of the operational semantics. Getting what you want to be signed with the key is the external channel operation I was referring to. In this particular case, somehow the attack vector being disabled is the actual private key being locked from any export. This gives a sense of security, however the channel abuse can bypass the security.
I was looking in to this just earlier today. The native macos version worries me a little because it seems someone just happened to discover it in MacOS, rather than it being a documented feature. The worry being Apple could change or remove it later locking you out of your keys since they never advertised this as a thing you could do.
Just to clarify this - you can have multiple public keys for the same user on the server end, in ~user/.ssh/authorized_keys. It's obvious when you know it, but I think it can be a source of confusion when discussing backup keys.
I have applied this at our company using Secretive, and extended to sudo over SSH: The remote end uses an SSH Agent PAM module that forwards sudo requests to the connected SSH Agent, which is Secretive running on the employee Macs, so we get remote sudo with TouchID prompts.
No more passwords to enter anywhere, and whatever an (AI) agent or malware is trying to do will require a human interaction at least, and not work when the screen is locked. The public keys authorized for this are distributed to the fleet via puppet, so we get control over who gets to sudo permissions and can easily provision or decommission machines.
I've been using Secretive (a nice software wrapper for these) for a few days for my Github keys (where 1Password holds the rest of my SSH keys). It works nicely, but obviously you can't back up a key that started life in a secure enclave, so you'll need alternative access arrangements for anything you setup that way.
2nd this. Found out about it when this type of thing came up about a week ago. I had a hard time getting the stock stuff going but Secretive made it easy to set up and I love using it.
Security features like this works, until it doesn't.
When I installed WireGuard on macOS, I was happy to discover that it stores private keys in the system keychain. Accessing the private key requires Touch ID.
Then one day after a system upgrade, the WireGuard app crashed on startup with:
Then the app deleted all profiles because they are all "orphaned", apparently for "keychain consistency" [0].From then on, I just use a text editor to edit my WireGuard configuration files, and import them to the app. The bug still exists to this day [1].
[0] https://github.com/WireGuard/wireguard-apple/commit/dfb685f2...
[1] https://www.reddit.com/r/WireGuard/comments/wyt828/osx_wireg...
Developing for macos, I’ve gotten bit by keychain consistency issues far too many times. It’s a mess.
So many of the keychain APIs are "deprecated" (read: marked as "we will nuke this API whenever we want") and have been for a decade or more, but have no replacement either.
A lot of the Security Framework came from Classic, hell, the idea of a keychain came from Classic! Just a mess.
I'm using WireGuard myself. Not on MacOS, but on Windows/Linux/Android. The reason I went with WireGuard to begin with, is due to how secure it suppsedly was. Having long standing unresolved issues like that is a bit worrying. I'm still most likely safe, but it's not exactly reassuring, either
This is likely just that if someone chooses an OS that requires a special HW just to build a release with no easy cross-compilation possible, and to pay a gatekeeper just to publish it, and that can't even run on an emulated computer properly, they can't expect FOSS maintainers will be bothered with much more than "here's a source code, build it/test it/do QA yourself if you like" for their particular pet platform.
For platforms where development is unhindered by arbitrary obstacles like these and others, wireguard works fine. :)
I had this as well - funny enough I asked my local AI to "find the removed config" and it was able to restore everything.
Lucky for you! I first ran into this before AI agents were a thing, so :(
This literally just happened to me yesterday. Again.
There are features built for security nerds (like PGP/GPG, etc)
And features built for normal people
The overlap is very rare
This is not a feature built for security nerds. This is a feature built for corporate compliance box tickers, and it works precisely as wanted in corporate environments, where all lost secrets can be easily replaced after visiting the helpdesk room personally.
True, there's that as well
well but deletion isn't what even those "security nerds" would have expected...
Security nerd here. I would have expected it. But only in the sense that I have no faith in this industry to write secure software and protect from data loss.
I keep hoping better liability laws will force management to support doing it the right way, if not anything else than avoid hefty fines.
> The bug still exists to this day
You linked a commit and a Reddit post. Is there a proper Github issue as well?
> You linked a commit and a Reddit post. Is there a proper Github issue as well?
It's funny you mentioned that. The GitHub repository is a mirror of https://git.zx2c4.com/wireguard-apple. You cannot create issues there. You're supposed to send a bug report to the mailing list, which is enough friction that none of us did so far.
>enough friction that none of us did so far.
Emailed per https://news.ycombinator.com/item?id=50450200
Definitely still a bug right? Linked them to your parent comment.
Edit: oops sorry team, think you're on it (4d ago): https://github.com/whyredfire/wireguard-apple/commit/0bac142...
Guess it can serve as feedback on the friction when issues can't simply be opened though!
> Definitely still a bug right? Linked them to your parent comment.
Thanks! Yes, it is still a bug.
> Edit: oops sorry team, think you're on it (4d ago)
I don't think @zx2c4 has seen this PR, and the author of the PR has not sent a bug report yet, so it's probably fine :P
> Thanks! Yes, it is still a bug.
Its unfortunate no updates for Wireguard macOS have been pushed out for 3 years, App Store is still showing 1.0.16 as the latest version – as also confirmed on the Wireguard website at the latest version.
No doubt macOS APIs have moved on a lot in 3 years and so perhaps bugs that are being seen are simply a reflection of the lack of updates to keep up with API changes ? Just thinking aloud here, I don't know if the concept of OS API changes are applicable to this particular PR / bug.
All right! :)
>friction
I appreciate when both options exist, because I've passed on reporting so many bugs, because I refuse to create a Microsoft account to do it.
In fact, I don't understand how sending an email is high friction.
> I've passed on reporting so many bugs, because I refuse to create a Microsoft account to do it.
99% of the time it's perfectly fine to just email the maintainer with a bug report and maybe a patch.
> In fact, I don't understand how sending an email is high friction.
Judging from the fact that after so many years, such a widespread (as seen in the Reddit post and here) issue still hasn't resulted in a single bug report in the mailing list, I'd say most people (including myself) do indeed consider using a mailing list to be higher friction.
If I'm not mistaken, unless you're already CC'ed by a participant, you'd have to subscribe to the whole mailing list just to receive notifications for a single thread. For an "issue" on a Git forge, participation is usually as easy as upvoting and then clicking the "Subscribe" button.
Assuming you're already using MS Github. Anyway, even if I was, a lot of Github project require filling out an absurd Simon Says form. Or that scene from Indiana Jones where he takes a wrong step and falls through the floor. That's nothing to do with email or Microsoft specifically, it's just another reason I almost never bother reporting bugs. Talk about friction.
Yeah, I hate these forms (issue templates) too. Every time I have to fill such a form, I always feel that they are discouraging me from writing the bug description in a natural, coherent way. I'd spend more effort to make sure the description is clear and contains all necessary information, if there's no issue template requirements.
Yes, but most people won't. Most people will submit a bug report with important information missing that means we have to go back and forth. Also, there is often specific debugging information that people would not even know would be useful.
In the current age, its useful because AI will submit tickets using the CLI that don't use the form, so I can immediately close and lock them and go on living my life.
from his link:
I decided years ago that I want to protect my keys with a hardware device that is independent of the vendor of the machine that I'm currently using. I also want to have backup keys and I want to be able to freely move them around various physical locations. Or have a backup shipped to me across the world if necessary.
YubiKeys were the only solution that fit these requirements, so I went with those. I have been using them for the last ten years or so and I've been very happy with this solution. See https://github.com/drduh/YubiKey-Guide for a description of how to make your RSA keys stored on YubiKeys work with GPG and SSH at the same time.
One fundamental problem with agentic development is that accessing the secure enclave key requires human presence. If you're running headless and aren't logged in, Secretive (not sure about others) will not be able to access the enclave, even when you only chose "notify" and not "touchid".
This is why I'm skeptical that the approach can scale reliably, even though I'm relying on it today. (Yes, PATs would fix this, but the enclave model is what I'm trying to retain for now.) What I've done instead is wrote a small daemon that sets the SSH authentication socket and does agent forwarding. This works when I'm traveling with laptop...but not yet from my mobile phone.
Example:
NB: I'm not using Rex instead of herdr and because Rex uses its own protocol, I always need to have one SSH tunnel to reset the remote agent socket. Not optimal, but it works. All of this feels wrong from a UX perspective, and it feels like I'm holding on too hard on secure enclave for what my use cases are. Feedback appreciated!> requires human presence
I'm not using the secure enclave, but I've been using YubiKeys for many years, and now that AI has arrived, I consider the above to be a good thing.
This means, for example, that unless I am physically present with my key inserted, activated, and touched on every operation, an AI agent cannot access any of my production servers. As I do not want AI agents to access my production servers, this is a very good protection technique. I am assuming that no matter how well you try to sandbox and restrict your AI agents running on your local machine, they might eventually be able to access any file in the file system.
Two thoughts on this:
1. The Secure Enclave being inaccessible when you're not present is a feature, not a bug. If you need to run headless then its not the correct solution to your problem and you'll have to find another.
2. You don't need an SSH agent proxy, you can use a `match` block in your `ssh_config` with something like `host * exec "test -z $SSH_TTY` to check whether you're operating over an SSH connection and select Secretive's SSH agent if you're using the machine locally.
I got a similar idea recently. I was becoming annoyed with the coding agent, running on my laptop, being unable to use sudo (with my supervision) when ssh-ing to my remote server. I wanted to build this as a PAM module. I started building this but after a while I googled (well… asked an agent) a bit more and found out that it already exists: https://github.com/z4yx/pam_rssh
I’m a happy user!
I’ve been a happy user of Secretive for years, including for Git commit signature which it make it a separate key with interaction but not for pull / push. I find the trade off ideal for GitHub.com and have other keys with different setup for servers.
SSH keys in plaintext was one of the first thing that bagel [0] our developer endpoint secret detection focused on. Then we embedded bagel as a library in our developer endpoint protection [1].
[0] https://boostsecurityio.github.io/bagel/ [1] https://boostsecurityio.github.io/dazio/docs/install/
The article starts on an incorrect premise.
The author says he doesn’t like the idea of having keys sitting on his file system ready to be snatched by bad actors.
That’s why they should be passphrase protected.
You use ssh agent to store the keys in memory and for convenience you can store the passphrases in keychain. MacOS OpenSSH has the UseKeychain config directive for that.
I use Yubikey/FIDO2 myself, but not because private keys cannot be protected on a file system.
Secure Enclave is still more secure in that it should be impossible to access the private key. Password protected keys are still vulnerable to malware that can read the keyboard, trick you in to typing the password, or that have exploits in macOS to read the memory.
Isn’t the macOS keychain user-readable by default?
Assuming the attacker has user rights (which it needs to read the key) it could also read the passphrase, right?
Update: I tried it out and stand corrected. ssh-add doesn’t store it in the user-readable keychain. So the passphrase is protected. (And also not recoverable by the user).
Key on disk plus a passphrase is still only 1FA. Key in the SE + an unseal PIN is 2FA. There's already plenty written about why one is better better than the other.
Do you type your passphrase every time you log into a remote machine? If not, the private key is in RAM and can be extracted by malware and stolen. If so, that's way more annoying than using touchid or some other physical proof of presence.
I used to install ssh_locker [1] (I don’t know if it still works), and have the passphrase in a keychain separate from the default `login` that I then set in Keychain Access to autolock after a set period of time. Handy for keeping the private key for a while in RAM and then autoremoved, instead of having it there for the whole session. It worked transparently and is a feature that should come with the OS.
[1]: https://github.com/ddribin/ssh_locker
ChatGPT says ssh_locker locked the keys with `ssh-add -x` which doesn’t necessarily remove them from RAM. Duh! It does point to suggested minimal changes to its source code to effectively accomplish that though.
was wondering the exact same thing. dude never heard of passphrases?
I use 1Password's `op` for this, but with an allowance: my dev environment keys.
I found I was getting fingerprint fatigue. Do I need a fingerprint request every time my dev server wants to connect to dev Supabase? No. That's expected, and zero risk. There's no data there.
I worried that a nefarious request would be auto-approved by me. When you see 20 requests a day, your muscle memory just reaches for the sensor.
So I have a separate 1Password vault that a service account [0] can read. That token is in my environment as OP_SERVICE_ACCOUNT_TOKEN, which the `op` tool understands.
It also has access to a vault with 'AI-safe' keys that I want Claude to be able to access without asking me.
Now, these requests work invisibly. And when I see a fingerprint popup, it's unusual, and I don't auto-approve by habit.
I also keep an item called '_One-day keys' in a separate vault. (The underscore just pushing it to the top of the list for easy access.)
It has prod keys, mostly restricted read-only. If required, I move that over to the open vault. Claude knows this item and has instructions to remind me to move it back out when it's finished what it was doing.
[0]: https://www.1password.dev/service-accounts/get-started
I believe 1Password has a sort of MCP server now as well, have you tried that out? I'm curious how well that works in place of using the CLI like this
Proton Pass with `pass-cli` has a similar thing, for anyone using that. You can give agents an expirable token and get an audit log of what item was read and when.
1Password + `op` cli is a good alternative choice here.
+ fnox
You should checkout the TKey (https://www.tillitis.se/products/tkey/)
It's effectively an open source version of a YubiKey, the hardware itself is all open source and it handles ed255519 SSH keys really well.
inb4 they're not the same, I know, I really like what the TKey team is doing.
Secure Enclave, TPM and other such security features are unfortunately so badly documented and need so many hoops to jump through to get them working, with any mistake either causing you to be locked out completely or to not give you the protection you wanted. There really should be a better way to make these technologies usable...
Why would you trust the NIST P-256 curve?
It is not one of the cryptographically safe curves, likely intentionally so.
https://safecurves.cr.yp.to/
The `sc_auth` trick was posted about a year ago, since macOS 26 Tahoe added the SSH related functions into the keychain.dylib.
> https://news.ycombinator.com/item?id=46025721
One of the problems I constantly face is that the SSH configuration is not portable when I use these esoteric things. Even with Secretive (which plugs a simple SSH socket), remote agent forwarding sometimes fail. (ie. Due to the old version on the server or configuration issue, or something else setting SSH_AUTH_SOCK...)
Moreover, it is not possible to drop these files into a file-sharing folder, like OneDrive or iCloudDrive or SMB share, and share them with many hosts at the same time. (Especially when different operating systems are involved...)
I found the hacker way simplest and easiest; put a basic Makefile with bunch of rules, have a "central" host/machine to distribute the keys and configuration from that point on...
I used yubikeys for ssh for years. First via the older clunky gpg as ssh key feature. Then via the FIDO api. My biggest issue was the chicken and egg issue when using this setup on a new machine or god forbid in a container etc. windows and WLS wasn’t super thrilled either. I switched to 1Password ssh because I actually like the fact that I can simply share my keys. It’s still not perfect. Agent forwarding needs an extra configuration and depending how many keys you have and how verbose your ssh config gets it becomes also very finicky. This solution feels for me like a slight setback because it’s a macOS only solution. If you have a Linux or windows machine next to a Mac you have to deal with the insecure key file on your hard drive again either via yubi keys or software solutions.
But I think I would take this over a yubikey. I really disliked the fact that one usb port was constantly in use.
The way I think of it is YubiKeys should be used to basically provision a computer with an approved device specific key. Whether it's a passkey or SE/TPM secured SSH/GPG/TLS key: the Yubikey holds the credential everyone trusts and therefore can be used from any device to gain access and enroll a new approved key.
Why do you share private keys?
SSH private keys should be passphrase protected.
You don’t have to do that if you use FIDO2 with a Yubikey.
Share as in storing my key in 1Password and use it from different machines. I generated the key from 1Password and don’t intend to export it.
You should reevaluate if you can instead use multiple public keys where you need to connect.
It’s a mind shift worth considering.
You then can accept that your ssh keys are different per hardware token (because the private key cannot be extracted).
No more sharing of private keys in your password manager.
you can’t just pop it out once you’re logged in?
Sure and pop in whenever I do a git commit? I also sign my commits. I used GPG (different key on the same hardware key) and now via SSH key. I had both a compact and bigger key. I got the compact key because my very first yubikey nearly broke. I lifted my Mac while it was connected and it cracked in the middle. I bought a compact one next which can stay inside and that one you don’t want to take out. It’s so small you’ll lose it. I know because that happened to my backup key … (that was actually the idea for the bigger one I bought but I lost the damn thing during the setup phase) But having the yubikey bound to your Mac at all times is also stupid because it doesn’t come with a fingerprint or other biometric sensor. The idea is that you use this thing as a token not that you leave the key in the lock at all times. For that alone I would favor the enclave solution.
The Yubikey 5 Nano and 5C Nano DO come with touch.
The idea is precisely that you keep them permanently connected in the laptop.
These days I'm mainly doing development inside VM's. Previously I was embedding Github key to each VM, but since I setup Secure Enclave (with Secretive), I'm only forwarding it's agent to VM and each git action on origin now requires approval using fingerprint reader. I think this a good way to prevent malware from gaining access to github
Why not use PAT?
I’ve grown to appreciate ssh should only be used with git for signing commits and as a deploy key (eg in a CI system).
I always use https for day to day git operations. My keys reside in gopass and are extracted by gopass-git-credential when needed by git.
I use that within devcontainers
https://github.com/gopasspw/git-credential-gopass
PAT are vulnerable, once stolen they can be used from anywhere until their (usually) long term expiry.
Better use short lived tokens with auto refresh. For example OAuth tokens. They can be used for the same use cases like PAT, but they expire after a max of 2 hours.
Which oauth tokens are you talking about? Because GitHub’s created by oauth or gh apps have a lifetime of either 8h or forever.
Gitlab uses 2h. TIL that GitHub uses 8h.
2 hours is plenty. The recent binance hack was 20 minutes after all. A 2 hour token is enough time to get all that can be gotten or done by a token in almost any system especially well defined ones like GH or GL.
It would be nice if GitHub or GitLab could limit a token to a given IP, but with how proxies and dynamic ips work I would imagine it would be a hassle to get it working correctly for everyone, but it’s better than nothing.
IP prefix should be enough to narrow down to your ISP + rough geographic location. Not perfect but requires no extra setup.
Those access token can still be stolen just the same. The Secure Enclave is better because you can ask it to authorise the action, but you can never get the key out of it, so the latest malicious npm package can’t send off your access tokens or ssh keys.
absolute security does not exist, but each small improvement adds up. I quite like having to authorize each remote git action with Touch ID.
I wish there was a way to cache the touch. As it stands, this is mostly unusable for commit signing in a rebase-heavy workflow.
In case you want to just have an encrypted SSH key with password in keychain, you can make it work automatically when logged in (and when Mac is unlocked) by putting in .ssh/config:
Docs: https://docs.github.com/en/authentication/connecting-to-gith...Normally with gpg agent that leaves the key unlocked it is not a problem.
Has anyone managed to extract a fingerprint reader from an Apple Magic Keyboard? I usually run my Macs in clamshell mode.
I have seen a few of these done at this point (i.e. hardware extracted from a Magic Keyboard)
This is the most recent one I found via Googling, I have not actually used this product:
https://standalone-touch-id.square.site/
On macOS use “Secretive”. It’s a small app that can generate keys from Secure Enclave and prompts you with touchid whenever those keys are being used
On iOS/ipad/(also macOS) you can also use the SSH terminal emulator Termius which has support for SE keys built in as well. So I have the set of keys synced to my VPS, this way I know that only my physical devices can log in and nothing else.
The thing I don't like about the Secure Enclave is that it does not yet support ed25519 keys.
Whats wrong with P256?
It's a shame that the poster doesn't mention what they ended up using instead in their update. Presumably a hardware key.
What's the general consensus around ssh-agent?
It's often the case that we make much effort into securing the key but expose it via an open agent socket at a well known path or use an open control master path for multiplexing.
Certain automation and dev tools like Ansible make it a pain to work if agent key requires a confirmation.
Weirdly enough, this is _why_ I ended up using Secretive: I needed a second ssh-agent because selecting which keys are forwarded per-host is very fiddly, and while standard ssh-agent kinda supports it, 1Password doesn't. Anyway, _a_ solution for this is having separate SSH agents, and Secretive is another agent on top of 1Password.
Not sure how many people feel like this, the risk in plain sight is a lot easier to handle than the risk hiding under the layers. You can keep shifting the storage for the key, one way or another you have to open up a channel to access to key or use the key. That channel would then become the vulnerability target.
To prove possession of a private key, one needs only to sign something with that key (this is how SSH key auth works). This is how things like the macOS Secure Enclave operate - the private key is stored in a separate hardware chip that only exposes an interface to sign something with that key, never to actually present the key. To obtain the key would require having physical access to the computer, ripping it open, and performing advanced deconstruction and analysis of that chip.
I dont want to be rude, I am aware of the operational semantics. Getting what you want to be signed with the key is the external channel operation I was referring to. In this particular case, somehow the attack vector being disabled is the actual private key being locked from any export. This gives a sense of security, however the channel abuse can bypass the security.
Somebody posted a tool here in the last couple of weeks that was doing this (using macos secure storage for local credentials/ keys)
I was looking in to this just earlier today. The native macos version worries me a little because it seems someone just happened to discover it in MacOS, rather than it being a documented feature. The worry being Apple could change or remove it later locking you out of your keys since they never advertised this as a thing you could do.
It is the same as you lost your device. You must have a second backup key, and treat one in Secure Enclave as disposable key.
Just to clarify this - you can have multiple public keys for the same user on the server end, in ~user/.ssh/authorized_keys. It's obvious when you know it, but I think it can be a source of confusion when discussing backup keys.
> That ssh: at the end is the FIDO2 application ID — it’s part of the key, not a comment.
I think this is incorrect and the "ssh:" at the end is indeed a comment.
If someone like the native approach or is familiar with SeKey, I created sekey-sh to simplify the process: https://github.com/cavoirom/sekey-sh
Work on macOS 26 and later.
I have applied this at our company using Secretive, and extended to sudo over SSH: The remote end uses an SSH Agent PAM module that forwards sudo requests to the connected SSH Agent, which is Secretive running on the employee Macs, so we get remote sudo with TouchID prompts.
No more passwords to enter anywhere, and whatever an (AI) agent or malware is trying to do will require a human interaction at least, and not work when the screen is locked. The public keys authorized for this are distributed to the fleet via puppet, so we get control over who gets to sudo permissions and can easily provision or decommission machines.
Pretty stoked about it so far.
Isn't a GitHub app project Secretive doing that but better?
No app is better.
I've been using Secretive (a nice software wrapper for these) for a few days for my Github keys (where 1Password holds the rest of my SSH keys). It works nicely, but obviously you can't back up a key that started life in a secure enclave, so you'll need alternative access arrangements for anything you setup that way.
2nd this. Found out about it when this type of thing came up about a week ago. I had a hard time getting the stock stuff going but Secretive made it easy to set up and I love using it.
Definitely a recco from me as well. -- git hub here: https://github.com/maxgoedjen/secretive
.
1
ss
We should have had this 20 years ago. It is almost pathological how difficult it is to reliably use hardware keys as ssh credentials.
ed25519-sk werks on my machine...
Unless you specifically mean (f)TPMs/secure envlaves.
..