System description

Since the start of the TSD projects, Sigma2 has been partner and supported the design and development of the TSD HPC and storage services. With the HPC Colossus system and part of the storage system being owned by Sigma2, TSD is part of the national e-infrastructure for research.

The aim of TSD is to offer storage and processing of data in a secure environment. University IT infrastructure in general comes with a number of security measures already included, but the UiO network is open and all computers have official ip addresses and quite liberal access lists. To achieve a higher degree of security and to make working with control and risk management easier, we chose to disconnect from UiOs regular network and offer just a limited number of ways one can interact with the system.

All projects/user groups are hence issued with their own dedicated virtual network interconnecting any number of dedicated project servers (Windows and/or Linux).  

Windows servers

 Windows 2012/19 server with SAS, Matlab, stat, R, and more. 

Linux servers

RedHat 8.0 with Libre (open) Office and R. 

Storage

All projects are issues with a basic amount of storage space which can be expanded if needed.  Projects with large storage need can get support and resources through Sigma2, the national e-infrastructure provider.

High performance computing

The main bulk of the High Performance Computing in TSD is provided as part of the National e-infrastructure operated by Sigma2. Projects apply to Sigma2 to get access to the Sigma2 operated partition of the HPC cluster Colossus, via https://www.sigma2.no/apply-e-infrastructure-resources

Backup

Backup is handled by UiOs regular backup system with the addition of encryption. The encryption key is only available on the dedicated terminal server with a copy stored in safes on two separate locations. 

Import/export

Data transfers to and from the services is handled by a special purpose file staging service and the project administrator controls access rights for all project members. By default all project members are able to transfer data in, but only the project administrator can do a data transfer out. 

Connecting

Connecting to the system is first done by accessing a login server via an encrypted SSH tunnel. From the login server users will connect to project VMs via PCoIP (Windows)/Thinlinc (Linux). The login procedure requires a one-time password that you get from a smartphone/yubikey.

Solution outline

The system is built on the idea that having a robust firewall around a system where one provides a full separation of the projects, is the best policy. A two-step authentication is needed to gain access to the system. Inside the system, every project has its own VLAN and its own virtual file system. This means that projects cannot find any information about any other project on the system.  

For storage purposes we use a logically separated part of Astrastore (UiOs and Norstores storage resource). Encrypted backups are done in UiOs regular backup system and has a separate encryption key for each project. Use of PostgreSQL databases is also offered in TSD 2.0 and there is also a secure high performance computing resource, Colossus, available to the projects. 

Infrastructure 

The solution is run on dedicated computers in a separate location in USITs machine room where only USITs operational personnel have access. To achieve complete separation of project environments running on the same hardware, we use RHEV KVM as a hypervisor. This means that a physical computer can be divided into several separate virtual computers which for all intents and purposes are working independently.

Maintenance and operational tasks 

System maintenance and operational tasks are performed by personnel from the USIT operations group. All access demands two-step authentication. Operations are separately managed for this solution to ensure that the security cannot be compromised by a successful break in on any other operational server at UiO. 

Stand-alone environment 

Project servers are presented to the user as a Windows or Linux terminal server with local storage. Cut/paste, mapping of disks and more, is turned off. All data transfers in and out of the system has to take place via the file staging service. 

Security