From humble beginnings as a local LAN party in Sweden, DreamHack has grown to become the world’s largest Digital Festival, bringing 20,000+ people together for a weekend of gaming, music, art, and more. The event is such a technological spectacle that it has even held a spot atop the Guinness Book of World Records for the fastest Internet connection, and most generated traffic.
From an infrastructure perspective, the event is especially unique and impressive because the large scale environment is setup and torn down in a matter of days by a group of highly passionate volunteer SREs, who refer to themselves as dhtech. Christian Svensson is one such volunteer, who came to ScaleFT to solve a very specific challenge within the DreamHack architecture - how to effectively manage privileged access across the network.
Setting Up the Space
The DreamHack environment is distributed between an off-site colo in Sweden and the on-site event space. The event space is what gets spun up and torn down with whatever servers and network infrastructure they can get their hands on. All in all, they operate about 20 servers off-site and 40 servers on-site. Getting up and running involves more than just installing software, the team has to deal with the surrounding power, Internet, and physical constraints. One thing is clear - everything is maxed out.
The Challenge with Privileged Access
The sheer amount of administrative work needed to get done in a short period of time means the dhtech team needs to be able to quickly gain access to the servers in a secure manner. Prior to using ScaleFT, the team relied on a FUSE filesystem integration with LDAP to read ssh keys. A problem with this setup is that it would often break with even the most minor of network glitches, resulting in connection errors. For the dhtech team, consoling servers to re-gain access is something that gets tiring real quick. In a fast-paced environment such as DreamHack, trouble accessing infrastructure can be damaging to the task of keeping the party going for the attendees.
The task of wrapping an extra layer around ssh keys to manage privileged access was something Christian wanted to avoid. He would much rather leverage existing technologies that deliver the intended experience over having to write and maintain something on his own. The primary goal was to remove the pain of having to create new keys and then manually upload them to the servers. A tedious task on its own that only compounds at scale. Time is the most valuable resource to the dhtech team, but as SRE’s, they also know the importance of security and reliability. After looking around at various options, Christian found that ScaleFT best met his requirements without getting in the way of the end user experience.
Using ScaleFT to Securely Access Hosts
The dhtech team setup the ScaleFT platform in the colo, alongside a number of bastions used to connect to the servers and network devices on-site. The team used Puppet to configure the servers, making it easy to install the ScaleFT agent, register with the bastions, and enroll into the ScaleFT project. The way ScaleFT handles host enumeration was useful as well, as the team could quickly see which hosts were properly installed.
The dhtech team each had accounts with ScaleFT, and the client app installed on their machines to enroll with the platform. As volunteers, each team member brough their own devices with their own personal configuration. Having the ScaleFT client work natively across MacOS, Windows, and Linux meant no quirks or extra tweaks. At the command line, ScaleFT uses ProxyCommand to abstract the underlying ssh workflow. Making a hop through a bastion is as simple as passing a ‘–via’ command.
“ScaleFT’s bastion integration was a huge time saver - being able to just type ‘ssh my-host’ was excellent.” - Christian Svensson
Christian’s background as a former SRE at Google means that he understands how a Zero Trust platform that uses Certificate Authorities instead of credential rotation is a more effective way to control access. When Christian found ScaleFT, it had a familiar feel as the BeyondCorp project within Google, and he was pleased to be able to follow the same principles with DreamHack. Given his expertise and experience, his feedback is welcome and much appreciated. We’re always looking to improve our product and end-user experience, so what better way than to put us into battle (no pun intended) with a high pressure event like DreamHack?!
Christian asked us about supporting multiple bastions and registering multiple servers, which are all great ideas for our product roadmap. He also pointed out a few gaps in our documentation, which we’re always looking to improve and add to. At the end of the day, the dhtech team was pleased with ScaleFT, and looks forward to using again at future events. Christian relayed that the phrase “super dope” was uttered in Swedish - verkar ju så satans coolt :)
Thanks to passionate volunteers such as Christian, the DreamHack event went off without a hitch, and we at ScaleFT are honored to have helped play a part. DreamHack returns to Austin at the end of April, and the Summer event is on the books for mid-June. If this event sounds up your alley, tickets are available at: http://tickets.dreamhack.com.
DreamHack is the World’s largest digital festival. DreamHack’s core and origin is the LAN party. The events are a platform for esport, knowledge and creative competitions, music acts, lectures by game developers, Internet and game culture, cosplay, the fair DreamExpo and much more. DreamHack Summer 2017 is open for all ages but can contain content like tournaments, exhibitors and partners showing games and other material not suitable for all ages.