News on HPC systems @ UiO, application deadline for CPU time through Sigma2, interesting conferences and external courses.
- USIT Underavdeling for IT i forskning ITF(NO), or Division for research computing (RC,EN), is responsible for delivering IT support for research at University of Oslo.
- The division's departments operate infrastructure for research, and support researchers in the use of computational resources, data storage, application portals, parallelization and optimizing of code, and advanced user support.
- Announcement of this newsletter is done on the hpc-users mailing list. To join hpc-users list, send a mail to sympa@usit.uio.no with subject "subscribe hpc-users Mr Fox" (if your name is Mr Fox). The newsletter will be issued at least twice a year.
News and announcements
Hi all, and happy new year! Our predictions from a year ago was that 2021 should be better than 2020, and as promised we managed to get Fox into production. Unfortunately, mostly due to pandemic restrictions and communication constraints, it seems we still have some work to do to get Fox fully utilized. So if you want to try running jobs on a cluster with no queue wait times and ample resources available (e.g. 8 x Nvidia A100 GPUs), now is your chance! If unsure how to start using Fox, please read ahead. We'll also give some words on the next national storage system, Europe's latest supercomputer, and a Norwegian AI cloud. And of course, the deadline for applying for the next Sigma2 allocation period is coming up very soon. Read more about these and other topics below.
Available computing resources - for free?
I guess you all know the saying that there's no free lunch. While that is surely true, you as an UiO researcher are at least eligible for free CPU hours on our infrastructure - LightHPC, ML nodes and Fox. While the two first are free for all at UiO at a first come first served model, Fox is part of the Educloud Research payment model. So if you need extras like a connected desktop, extra storage or advanced user support, these are available for a very nice price while, if you are a UiO researcher, access to low threshold HPC resources and scientific software is completely free. (And if you use a lot of CPU hours on Fox we'll even help you get access to the national HPC clusters.)
Fox Supercomputer - get access now
The Fox cluster is the 'general use' HPC system within Educloud, open to researchers and students at UiO and soon also their external collaborators. There are 24 regular compute nodes with 3,000 total cores and five GPU accelerated nodes with NVIDIA RTX 3090 and NVIDIA A100 cards available. Access to Fox requires having an Educloud user, see registration instructions.
For instructions and guidelines on how to use Fox, see Foxdocs - the Fox User Manual.
What is Educloud Research?
Fox is part of the Educloud Research platform, but what is actually Educloud Research? Play video below to learn more.
Software request form
If you need additional software or want us to upgrade an existing software package, we are happy to do this for you (or help you to install it yourself if you prefer that). In order for us to get all the relevant information and take care of the installation as quick as possible, we have created a software request form. After filling in the form a ticket will be created in RT and we will get back to you with the installation progress.
To request software, go to the software request form.
ML nodes
The Machine Learning (ML) nodes are a few nodes dedicated to Machine Learning using GPU accelerators. If you are using one of the larger ML frameworks, such as TensorFlow or PyTorch, then the ML nodes can help you accelerate your training.
Due to the popularity of the ML nodes we are in the process of assisting some of the heavy users to migrate to Fox. We have kept most popular software up to date, send us an mail if we miss anything.
For more info and status, see the ML nodes page.
Please send any questions or comments to itf-ai-support@usit.uio.no and applications for access can be submitted here.
NAIC
We are happy and excited to inform you that our USIT initiated grant application for the Norwegian Artificial Intelligence Cloud (NAIC) was selected for the negotiation phase by the NFR. This project will pave the way for the Norwegian researchers from academia and government research institutes as well as from industries to get access to a pool of ML/AI infrastructure. The project kick-off is expected to be approximately this summer, after negotiations with RCN. Partners are UiO (lead), NTNU, UiT, UiB, UiA, SINTEF, NORCE, Simula and Sigma2.
FAIR@UiO
Both UiO and external funders have started to require increased transparency in research with increased reuse and sharing of data. To make that possible in practice requires rich metadata so that the data itself becomes FAIR (Findable, Accessible, Interoperable, Reusable). The investment in FAIR data - Open Science and data management combined with work on data management plans and training in data management continues at UiO with the FAIR@UiO project.
Lightweight HPC - free for all UiO users
The service called lightweight HPC is a service with close to no entry level, free for all and interactive. This service is for users who have needs that are too small to involve any kind of project management but just need to run some computations on a powerful server. You can think of it as an extension of your desktop, calculator or laptop. As this little farm of compute nodes has been set up with different architectures and capabilities (such as ARM, Intel and AMD in various generations), it also serves a testing ground for new/experimental technology.
All systems in the lightweight-HPC farm share a common $HOME mounted
on the UiO IBM-ESS storage, meaning that you'll find your files on all the
systems under this service. As the systems are interactive you might find that
some nodes are more loaded than others, so as with all free seating situations
you might have to look for a less loaded one.
Users who overuse the system will face themselves in the happy situation of being
candidates for migration to larger HPC systems, like the Fox in the Educloud, the
national Sigma2 systems or even the world class systems like the Pre-exascale
system Lumi where Norway/Sigma2 participates.
The system freebio.hpc holds a selection of bio-informatics databases so if you
need something along these lines look for it there (under /work).
More info about the lightweight HPC systems is published on the service page.
The only requirement to get access to this service is that you have a UiO username, so if you want to check it out, please apply for access through nettskjema.
New name for the Metacenter: NRIS
As some of you might have noticed, the Metacenter has disappeared since our previous newsletter. The collaboration between Sigma2 and the four universities NTNU and the Universities of Bergen, Oslo and Troms? has now changed name to Norwegian research infrastructure services, or just NRIS. The background for the name change is related to a broader identity work for Sigma2 and the collaborating universities, based on the desire to appear more clearly as a joint service provider to our users. For you who use the Sigma2 systems, the main change will be the new email address for technical support - support@nris.no.
Contract signed for NIRD 2020
We're happy to report that Sigma2 has signed the contract for NIRD 2020 with Move to deliver an IBM storage solution to be installed in Lefdal Mine Datacenter by summer 2022. Starting at a capacity of 32 PB and expandable to 70PB the new solution will easily replace the current NIRD capacity of 12PB. With an I/O throughput of 229 GB/s, the new NIRD will support demanding workflows such as machine learning and data intensive analysis. Similar to the new IBM ESS storage system at UiO (used, e.g., in TSD, Educloud and Fox), the solution is based on IBMs Spectrum Scale, Spectrum Discover and Spectrum Protect and will be operated by NRIS.
LUMI
LUMI is Europe's latest supercomputer and is based around AMD accelerators (GPUs). Researchers in Norway can get access to this system either through Sigma2 or by applying to EuroHPC JU.
LUMI-G, which is the accelerator partition of LUMI, will consist of 2560 nodes each with 4 x AMD MI250X GPUs. Each of these GPUs will be connected directly to LUMIs interconnect for inter-node GPU-to-GPU communication. LUMI will cater for a range of application and based on the architecture Machine Learning applications will benefit quite heavily. While the CPU partition (LUMI-C) is already in production, LUMI-G is planned to be in production before the end of the e-Infrastructure allocation period 2022.1.
New e-Infrastructure allocation period 2022.1, application deadline 1 February 2022
The e-Infrastructure period 2022.1 (01.04.2022 - 30.09.2022) is getting nearer, and the deadline for applications for HPC CPU hours and storage (for both regular and sensitive data), is 1 February. This also includes access to LUMI-C and LUMI-G. Please note that although applications for allocations can span multiple allocation periods, they require verification from the applicants prior to each application deadline to be processed by the Resource Allocation Committee for a subsequent period.
Kind reminder: If you have many CPU hours remaining in the current period, you should of course try to utilize them asap, but since many users will be doing the same there is likely going to be a resource squeeze and potentially long queue times. The quotas are allocated according to several criteria, of which publications registered to Cristin is an important one (in addition to historical usage). The quotas are based on even use throughout the allocation period. If you think you will be unable to spend all your allocated CPU hours, it is highly appreciated to notify sigma@uninett.no so that the CPU hours may be released for someone else. You may get extra hours if you need more later. For those of you that have run out of hours already, or are about to run out of hours, take a look at the Sigma2 extra allocation page to see how to ask for more. No guarantees of course.
Run
projects
to list project accounts you are able to use.
Run
cost -p nn0815k
to check your allocation (replace 0815 with your project's account name).
Run
cost -p nn0815k --detail
to check your allocation and print consumption for all users of that allocation.
HPC Course week/training
We shall arrange a introduction to HPC course during this May. This course would be useful for new comers and for those who want to refresh there knowledge.
Dates: 4-5 May, 2022
Registration and further details will be available by February 12th and will be published on the NRIS training events page.
A course for advanced users is being planned for May as well, but the details are not finalized yet.
For more details on training activities please visit the NRIS training events page.
Availability of other computing resources
If you want to explore ARM based compute systems and also explore next generation Vector/SIMD/SVE units and its impact on your code please come forward as we have a nice set of ARM and Allinea tools to run on our ARM testbed. UiO is working closely with Simula which runs a project looking at novel hardware for exascale. If your interests are along those lines we are happy to introduce you.
Other hardware needs
If you are in need of particular types of hardware (fancy GPUs, kunluns, dragons, Graphcore, etc.) not provided through our local infrastructure, please contact us (hpc-drift@usit.uio.no), and we'll try to help you as best we can, similarly to our ARM collaboration with Simula mentioned above.
Also, if you have a computational challenge where your laptop is too small but a full-blown HPC solution is a bit of an overkill, it might be worth checking out NREC. This service can provide you with your own dedicated server, with a range of operating systems to choose from.
With the ongoing turmoil about computing architectures we are also looking into RISC-V, the European Processor Initiative is aiming for ARM and RISC-V and UiO needs to stay put.
Publication tracker
USIT Department for Research Computing (RC) is interested in keeping track of publications where computation on RC services are involved. We greatly appreciate an email to:
hpc-publications@usit.uio.no
about any publications (including in the general media). If you would like to cite use of our services, please follow this information.
Fox working on documentation
