Now over 20 years old, SSH (Secure Shell) is still the most common method of remotely accessing a Linux server, thus making its use a common target for attackers attempting to infiltrate corporate networks. While the protocol itself carries a number of advanced security properties, it does allow for human error, opening the door for unwarranted privileged access to sensitive company resources.
A common area of misuse is committing a private key to a public source code repository, or somewhere else it can be easily obtained. This has been a known threat vector for a long time, but researchers at Wordfence, a security service for Wordpress, noticed a recent uptick in SSH scanning activity, finding that attackers were searching for common private key filenames such as id_rsa more frequently. Even if it’s passphrase protected, getting a hold of a private key would make it fairly simple for someone to access machines with the associated public key.
Keeping track of all valid credentials across a company is something that IT departments continually struggle with. The stakes are high if they fall into the wrong hands, essentially handing over the proverbial ‘keys to the kingdom’. A recent study by Dimensional Research found that 90% of respondents do not have a complete and accurate inventory of all SSH keys, and 63% don’t actively rotate keys, even when an administrator leaves the company. How many times have we read about a disgruntled former employee who wreaked havoc on systems they still had access to?
Managing privileged access to servers has always been a rigorous exercise in key management. There are a number of best practices to follow to avoid the many common pitfalls, such as storing private keys in a vault service or maintaining a strict rotation policy, but they don’t get to the crux of the matter – the credentials are static. We have a (not so) crazy thought here at ScaleFT – what if we got rid of them entirely? It’s our belief that static credentials are a thing of the past, representing the failed perimeter model of corporate security.
Problems With PKI Today
The traditional PKI (Public Key Infrastructure) that backs SSH was built for a different time, where a key exchange meant enough to grant trust. What works for Alice and Bob in theory, doesn’t always work in practice or at scale, however. The core problem lies with the false assumption that ownership of a private key equates to an identity profile. While we may say “Alice’s private key”, there’s no associated authentication process that could verify it was Alice who generated the key pair to begin with, or that Alice is the only person who currently possesses the private key.
Without a proper authentication process, there is no clear way to attribute to a person, nor a feasible way to provide any historical context. As these keys get issued and distributed across fleets of infrastructure and cloud services, the challenge only compounds. Each public key accrues more privileges over time – the longer it’s been alive, the more likely it’s been shared with a resource, making it extremely difficult to track and subsequently revoke. With this model, time is a bug that can’t be fixed.
Authorizing a specific request should also go beyond the mere presence of the public key within the authorized_keys file of the server. Independent of authentication, authorization is about who should and should not be able to access a specific server from a known device, and what permissions they have on the machine. You may have a policy that states only members of the engineering team can only access a production server through a bastion host. Adhering to that policy in practice, however, is impossible without a proper authorization process.
In addition to your own corporate policies, compliance controls such as those from PCI-DSS and ISO 27001 are strict about access controls and audit logs. What companies are forced to do is implement strict guidelines about how to generate, protect, and use keys, be extremely diligent about not letting employees use shared keys, maintain the authorized_keys file across every server through configuration management software, maintain a potentially large revocation list… so on and so forth. There are tools and products to help with this, but again – everything is built around managing static credentials.
Trust in a Zero Trust World
We’ve come to a consensus across the industry that the network perimeter is breaking down in the modern cloud era. Even the traditional perimeter arms dealers are starting to come around, talking about Zero Trust in their marketing materials. It’s time to take that one step further and agree on a new model of trust attestation to match, where decisions are removed from the network layer and handled correctly at the application layer. Flipping the model around like this means starting with zero privileges by default, and only granting access to a resource once a request has been fully authenticated AND authorized. Verify, then trust – the mantra of Zero Trust.
Google’s BeyondCorp is the marquee example of Zero Trust done right, an internal initiative meant to better secure how employees access corporate resources. The end result is a carefully crafted distributed system that makes real-time trust decisions based on what is known about a user and their device, allowing the thousands of Googlers across the globe to work securely from any location without the use of a VPN. BeyondCorp is just one example, and we’ve seen more and more companies start to adopt a similar approach.
It’s important to understand a couple of things about Zero Trust in order to make it a reality the way Google did. First, authentication and authorization are two clearly distinct processes, a fact that has commonly been conflated. In the most simple of terms, authentication is about who you are and authorization is about what you can do. Note that authorization depends on authentication, but makes a decision based on additional factors. Second, a meaningful trust attestation is about adhering to a policy that states a person (or service) from a known device can access a specific resource at a given time. It’s not necessarily about the credential itself, it’s about the context surrounding the request. Even so, we still need a way to correctly represent the identity and permissions associated with the attestation that was made.
Ephemeral Client Certificates
Ephemeral computing resources have become the norm for cloud-native environments – instances, containers, and functions spin up on-demand, execute, and then tear themselves down when complete or no longer needed. Despite the automation that powers the environment, there is still a need for systems administrators to be granted privileged access – in cases of emergency, or regular maintenance. It stands to reason quite naturally that static credentials would be a poor mechanism to back the access controls for these dynamic resources.
The thinking behind ephemeral credentials is that a request triggers the auth processes, where a trust attestation is made against the policies of the resource. If valid, a single use credential is issued which is used to initiate a secure session with the resource. This credential is tightly scoped to the specific request, rendered useless in any other scenario.
The fundamentals of this workflow can be accomplished through a client certificate-backed PKI architecture. Client certificates became a supported feature of OpenSSH in 2011 with the introduction of version 5.4. Unlike standard SSH keys, which carry no metadata, certificates are cryptographically verifiable metadata objects that can inject things like usernames, roles, and more. Time now becomes a feature because the scope of the credential truly represents a dynamic attestation of trust.
The fine-grained access controls inherent in a client certificate-backed PKI architecture solves for the problems with SSH keys because the credential can be tied to an actual user. The threat vector of lost, stolen, or misused credentials is also eliminated, saving managers from the numerous headaches associated with SSH key management.
ScaleFT Server Access
To put these ephemeral credentials into action, you need to surround the client-certificate PKI with the supporting access controls. Companies with the means can build their own system similar to Google’s BeyondCorp, but this is no simple task, as you need to operate your own Certificate Authority along with a distributed system capable of performing dynamic authentication and authorization against policies in real-time.
For those who don’t wish to invest that much effort into building such a complex system, but want the same security outcomes, ScaleFT Server Access is a simple to use SaaS solution that works out of the box for both SSH and RDP. We designed our product around client certificates from the ground up, operating a programmable Certificate Authority that issues client certificates that expire in about 3 minutes, and are tightly scoped to the user, device, and resource being accessed. That short period of time is configurable, meant to account for things like clock drift because an expired certificate won’t be accepted by the server – which is really the whole point.
To use ScaleFT, your servers run a lightweight agent that configures OpenSSH to trust the ScaleFT CA for client certificate auth. This agent is not involved in the auth processes, the server’s SSH daemon is able to independently verify the user’s client certificate. The agent also manages the local user accounts, and audits all login events. Your users then run an app on their client machines that communicates its state to the platform to make a dynamic authorization decision against the policy of the resource.
Our product doesn’t require any modifications to SSH, or changes to the workflows you’re used to – everything is built into the protocol itself using the local proxycommand feature. Because the authorized_keys file is irrelevant, you can eliminate the .ssh directory entirely. There’s also no need for you to manually validate hosts, or the need to maintain a rotation policy.
All of the characteristics of ScaleFT mitigate the threat vectors that come with traditional PKI, delivering a more secure and compliant environment that won’t get in the way of employee productivity. We abstract away the complexities of operating the PKI and supporting access controls, so you can focus on what matters most to your business.