TL;DR: A loose collective around @greenstheorem:matrix.org, @subjekt91:matrix.org, @jonasled:jonasled.de and more are hosting Pixelflut at GPN22!
How to take part:
- Come to our desk (drawing to be added - just look for the projection on the wall a bunch of fibers coming from the ceiling) and connect to our access switches on the desk (10GBase-T/5GBase-T/2.5GBase-T/1GBase-T or SFP28 25GBase / 10GBase)
- ASK US FIRST and THEN connect in Colo to our core switches
- VLAN is always untagged, you will get an IP via DHCP
- Start flooding 192.168.16.2:1234
Useful links:
https://github.com/TobleMiner/sturmflut
Long version:
A loose collective around @greenstheorem:matrix.org, @subjekt91:matrix.org, @jonasled:jonasled.de and more will host Pixelflut at GPN22!
We are not affiliated with the GPN or the GPN NOC!
For any issues, please contact DECT 6969 or directly @greenstheorem:matrix.org (DECT 4444 / SIP Backup 4445)
Pixelflut-Express
As the NOC cannot provide a sufficient number of high speed links and have made the explicit request to completely separate any Pixelflut traffic or network, we are running our own ultra-high-speed network called "Pixelflut-Express"
We explicitly ENCOURAGE using the Pixelflut-Express network for purposes other than Pixelflut! Want to run a high performance storage cluster? Need high speed ports for experiments? Anything else? Peer with us!
You cannot access the internet or any other parts of the GPN Network from the Pixelflut-Express network! It is completely isolated! If you want to access anything outside the Pixelflut-Express network, you MUST have multiple NICs. No, we cannot provide access to the GPN network within a VLAN or similar. We have explicitly been asked to NOT do that.
Pixel-Peering / Pixel-Colo
In order to take part in the Pixelflut, you must connect to the Pixelflut-Express network.
You can access the Pixelflut-Express network in the YoloColo and Hackcenter.
Please note that Pixelflut-Express and GPN networks are completely isolated. It is very likely that you want to connect to both networks at the same time and thus need 2 NICs. For information on how to connect to the GPN Network in YoloColo, please consult GPN22:NOC
In the YoloColo, we are running 2x Mellanox SN2700 switches in MLAG (subject to change - we can add more switches if required). That means you can connect your servers dual-homed! Dual Homing must happen via LACP in fast mode! If you have significant bandwidth requirements for inter-node traffic, you MUST NOT mix dual-homed and single-homed! The reason is that we don't want to put unnecessary pressure on the MLAG IPL
Each of those switches has 32x100G ports (downwards compatible to 40G / 25G / 10G), some of which can be broken out.
We will bring a limited number of cables and transceivers (100G-CWDM4 transceiver, 10GBase-LR transceivers, QSFP28 cables, QSFP28 breakout cables, QSFP to SFP adapters), which you can borrow. However we cannot guarantee that we will have enough for everyone. If you have cables or transceivers, please bring them with you!
We will also bring a limited number of NICs (around 10x Mellanox ConnectX-4 CX456A 2x100G QSFP28, 4x Supermicro AOC-STG-i4S 4x10G SFP+ Intel XL710-BM1 based) which you can borrow in case you need them.
There will be a 400GBit/s (4x 100G) crosslink from YoloColo to the Hackcenter. We are running 1x Mellanox SN2410 (48x25G + 8x100G) and possibly 1x Ruckus ICX7650-48ZP (24x 10/5/2.5/1G + 24x1G) in the hackcenter. Currently we are not planning to provide QSFP28 ports in the hackcenter. If you need them, please let us know and we will see whether we can deploy another switch there
VLANs (e.g. for internal cluster traffic) are available on request. All ports will run in hybrid mode (VLAN 1 / untagged -> Pixelflut, all other VLANs -> tagged). Inter-VLAN routing is available on request
To allow for easier planning, please submit Peering requests in advance (see above for contact information). We will add them to this list. Please do not edit those tables yourself! Only Pixelflut-Orga should edit those tables.
| Total Amount | Type / Speed | Dual-Homed? | Name | Purpose |
|---|---|---|---|---|
| Internal | Internal | Internal | Internal | Internal |
| 8 | QSFP28 / 100G | Yes | Pixelflut-Orga | MLAG-IPL |
| 4 | QSFP28 / 100G | Yes | Pixelflut-Orga | Crosslink YoloColo <-> Hackcenter |
| 10 | QSFP28 / 100G | Yes | @greenstheorem:matrix.org | EPYC 7742 + Ryzen servers |
| 7 | QSFP28 / 40G | No | @jonasled:jonasled.de | Jonasled PVE Ceph |
| 4 | QSFP28 / (100+40G) | No | @subjekt91:matrix.org | Subject91 |
| Public | Public | Public | Public | Public |
| 1/2 | QSFP28/100G | Maybe | @masterx244:nplusc.de | Fujitsu @masterx244:nplusc.de |
| 1/2 | QSFP28/100G | Maybe | @markus:matrix.fablab-altmuehlfranken.de | @markus:matrix.fablab-altmuehlfranken.de |
| 4 | SFP+ (=> 2x QSFP28) | N/A | @tilcreator:computational.beauty | @tilcreator:computational.beauty |
| Placeholder | Placeholder | Placeholder | Placeholder | Placeholder |
Currently left: ??? QSFP28 Ports
| Total Amount | Type / Speed | Dual-Homed? | Name | Purpose |
|---|---|---|---|---|
| Internal | Internal | Internal | Internal | Internal |
| 4 | QSFP28 / 100G | N/A | Pixelflut-Orga | Crosslink YoloColo <-> Hackcenter |
| 2 | QSFP28 / 100G | N/A | Pixelflut-Orga | Access Switch (ICX7650-48ZP) |
| Public | Public | Public | Public | Public |
| 2 | QSFP28 / 100G | N/A | @aveexyliium:matrix.org | Downlink Quanta / SN2410 |
| Placeholder | Placeholder | Placeholder | Placeholder | Placeholder |
Currently left: 48x SFP28 ports
| VLAN ID | Name | Purpose |
|---|---|---|
| 1 / untagged | Pixelflut-Orga | Main-Pixelflut VLAN |
| Beispiel | Beispiel | Beispiel |
| Beispiel | Beispiel | Beispiel |
| NIC Type + Amount | Name |
|---|---|
| 1x Mellanox ConnectX-4 CX456A (LP/FP?) | @masterx244:nplusc.de |
| 1x Mellanox ConnectX-4 CX456A (LP/FP?) | @markus:matrix.fablab-altmuehlfranken.de |
| 2x SFP+ NICs | @markus:matrix.fablab-altmuehlfranken.de / @t3chler:matrix.t3chler.de |
| Beispiel | Beispiel |
| Beispiel | Beispiel |
IP addressing / VLANs
For receiving a IP address, please turn on DHCP. We are giving out IPs from the 192.168.0.0/19 space with a lease time of 7 days. Internet access is currently not planned.
We might be able to provide out of band access (only link between Pixelflut-Express network and GPN network) for management purposes via Wireguard Tunnels. This is TBD.