News on HPC systems @ UiO, application deadline for CPU time through Sigma2, interesting conferences and external courses.
- The IT department IT i forskning, formidling og utdanning FFU(NO), or Division for Research, Dissemination and Education (RDE)(EN), is responsible for delivering IT support for research at University of Oslo.
- The division's departments operate infrastructure for research, and support researchers in the use of computational resources, data storage, application portals, parallelization and optimizing of code, and advanced user support.
- Announcement of this newsletter is done on the hpc-users mailing list. To join hpc-users list, send a mail to sympa@usit.uio.no with subject "subscribe hpc-users Mr Fox" (if your name is Mr Fox). The newsletter will be issued at least twice a year.
Hi all, and welcome back from summer vacations! Before jumping head first into the fall semester and getting flooded with more AI (generated) news, let's take some time to look back at where we were before summer in our completely hand-written newsletter, free from any generative pre-trained transformed text. Enjoy!
Data@UiO 2023 conference - video recordings available
Big thanks to all who participated in the Data@UiO 2023 conference! If you missed parts of the conference or need a refresher after summer, video recordings are now available. In addition to some very nice keynote presentations from Maren Ranhoff Hov and Hege Berg Henriksen, we launched and demonstrated Educloud On Demand, we showed several use cases and services related to AI that we're working on and we showed some highlights of our roadmaps and plans forward for our research platforms and services.
Educloud on Demand
As was demonstrated at the Data@UiO 2023 conference, our new Educloud On Demand service is in production and available to anyone with Fox access!
OnDemand is a web portal that allows you to easily run software like Jupyter Lab and Notebook, RStudio, VSCode and many other interactive applications right from your browser, while taking advantage of the compute power on Fox. Your application will run through the slurm queue system just like a regular HPC job, but there is no need to set up ssh tunnels, X forwarding or even write your own job script - all you need to do is select your software, choose resources like cpu/ram/accelerators and click start.
To try it, just go to https://ondemand.educloud.no/ and log in with your Educloud account.
We're really excited about this new and user friendly way to provide HPC services, and we hope you will find good use for it. Please do get in touch if you have problems, suggestions, ideas, or anything else - we are just starting out, and depend on your feedback so we can improve the service where it matters.
New NIRD in production
The NIRD2020 project is now finished and new NIRD is officially in production since February 22. The system is based on IBM's Spectrum Scale, Spectrum Discover and Spectrum Protect, with total capacity of 32 petabytes (PB).
The new infrastructure offers, among other things, adaptable application services, support for file and object storage and several protocols, and application programming interfaces (APIs).With an I/O throughput of 229 GB/s, the new NIRD will support demanding workflows, such as artificial intelligence/machine learning and data-intensive analysis.
BioNT - BIO Network for Training
BioNT - BIO Network for Training - is an international consortium of academic entities and small and medium-sized enterprises (SMEs). BioNT is dedicated to providing training programs and fostering a community for digital skills relevant to the biotechnology industry and biomedical sector and University of Oslo is part of this consortium through the IT Department.
The first course they organize will be held online from Sept 4 to Sept 8.
A practical introduction to bioinformatics and RNA-seq using Galaxy
September 4, 2023 - September 8, 2023
Registration deadline: August 6, 2023
Location: Online
Details and registration: https://www.cecam.org/workshop-details/1260
Norwegian Ai Cloud
The Norwegian Ai Cloud (NAIC) project is aiming to provide infrastructure and support for Norwegian machine learning (ML) and AI. The project was initiated before summer and researchers have concluded the initial user and stakeholder survey. From the inputs we have gathered we will now improve the implementation plan to better suit the end users.
Sigma2 cost for UiO-users
During the renegotiation of the Sigma2-BOTT-RCN agreement, UiO managed to lower our cost for 2023 to approximately 24MNOK instead of the planned 37,5MNOK. The short story is that UIO, as all other users, will pay the actual operational costs, but we will pay up-front as a guarantee, and not per use. The BOTT universities will pay a total of 53MNOK, and none of the BOTT universities will pay more before the total usage of BOTT on the NRIS systems goes above a total of 53MNOK in operational costs.
It is of utter importance that all researchers and especially those at UiO are aware of the actual operational cost connected to their work, and that you try to apply for external funding to cover your part of the operational cost. How much, and if, you will have to pay for your operational costs directly from your project depends on your Faculty and how they choose to handle the invoice.
Centrally at UIO the administration will shave a significant part of the invoice off, this will function as a “centrally covered discount” before the reminder of the invoice is sent to the Faculties and Museums based on their usage.
TSD and the IT Department in EU projects
TSD and the IT Department partners up in two new EU-projects on health data. First, we are associated partners to Directorate of eHealth in a project on how to adapt and form the EHDS (European Health Data Space) vison into reality. Second, during July we got to know that our EOSC application where we are associated with Elixir / Elixir.no for showcasing how to build and operate TREs (Trusted Research Environments) in the EU was accepted.
Fox Supercomputer - get access now
The Fox cluster is the 'general use' HPC system within Educloud, open to researchers and students at UiO and their external collaborators. There are 24 regular compute nodes with 3,000 total cores and five GPU accelerated nodes with NVIDIA RTX 3090 and NVIDIA A100 cards available. Access to Fox requires having an Educloud user, see registration instructions. About 250 projects have already joined Educloud!
For instructions and guidelines on how to use Fox, see Foxdocs - the Fox User Manual.
Software request form
If you need additional software or want us to upgrade an existing software package, we are happy to do this for you (or help you to install it yourself if you prefer that). In order for us to get all the relevant information and take care of the installation as quick as possible, we have created a software request form. After filling in the form a ticket will be created in RT and we will get back to you with the installation progress.
To request software, go to the software request form.
New Sigma2 e-Infrastructure allocation period 2023.2, application deadline 20 August 2023
The Sigma2 e-Infrastructure period 2023.2 (01.10.2023 - 31.03.2024) is getting nearer, and the deadline for applications for HPC CPU hours and storage (for both regular and sensitive data), is 20 August. This also includes access to LUMI-C and LUMI-G.
Please note that although applications for allocations can span multiple allocation periods, they require verification from the applicants prior to each application deadline to be processed by the Resource Allocation Committee for a subsequent period. Hence any existing multi-period application must be verified before the deadline to be evaluated and receive an allocation before the new period starts. This does not apply to LARGE projects.
Kind reminder: If you have many CPU hours remaining in the current period, you should of course try to utilize them asap, but since many users will be doing the same there is likely going to be a resource squeeze and potentially long queue times. The quotas are allocated according to several criteria, of which publications registered to Cristin is an important one (in addition to historical usage). The quotas are based on even use throughout the allocation period. If you think you will be unable to spend all your allocated CPU hours, it is highly appreciated to notify sigma@uninett.no so that the CPU hours may be released for someone else. You may get extra hours if you need more later. For those of you that have run out of hours already, or are about to run out of hours, take a look at the Sigma2 extra allocation page to see how to ask for more. No guarantees of course.
Run
projects
to list project accounts you are able to use.
Run
cost -p nn0815k
to check your allocation (replace 0815 with your project's account name).
Run
cost -p nn0815k --detail
to check your allocation and print consumption for all users of that allocation.
HPC Course week/training
Norwegian Research Infrastructure Services NRIS has an extensive education and training program to assist existing and future users of our services. UiO has joined NRIS training providing training to all Norwegian HPC users, instead of just focusing on UiO users, this makes it possible to provide a more streamlined and consistent training by consolidating the training events. The courses are aimed to give the participants an understanding of our services as well as using the resources effectively.
There are courses coming up in October and we will send details when the logistics are in place.
We have archived the previous course : https://documentation.sigma2.no/training/events.html
Training video archive: https://documentation.sigma2.no/training/videos.html
CodeRefinery
CodeRefinery arranges training and e-Infrastructure for Research Software Development. The next training course is arranged online September 19-21 and 26-28, 2023.
More details and registration : https://coderefinery.github.io/2023-09-19-workshop/
If there is interest we will arrange a local exercise group for UiO, where we can take the online course together and help out each other. If you are interested in this please register first and then send a mail to hpc-drift@usit.uio.no
Other hardware needs
If you are in need of particular types of hardware (fancy GPUs, ARM, Kunluns, Dragons, Graphcore, etc.) not provided through our local infrastructure, please contact us (hpc-drift@usit.uio.no), and we'll try to help you as best we can.
Also, if you have a computational challenge where your laptop is too small but a full-blown HPC solution is a bit of an overkill, it might be worth checking out NREC. This service can provide you with your own dedicated server, with a range of operating systems to choose from.
With the ongoing turmoil about computing architectures we are also looking into RISC-V. The European Processor Initiative is aiming for ARM and RISC-V and UiO needs to stay on top of things.
Publication tracker
The Division for Research, Dissemination and Education (RDE) is interested in keeping track of publications where computation on RDE services are involved. We greatly appreciate an email to:
hpc-publications@usit.uio.no
about any publications (including in the general media). If you would like to cite use of our services, please follow this information.
Fox checking that NIRD archive is ready for production
