GPN22:Pixelflut

aus dem Wiki des Entropia e.V., CCC Karlsruhe

A loose collective around @greenstheorem:matrix.org, @subjekt91:matrix.org, @jonasled:jonasled.de and more will host Pixelflut at GPN22!

We are not affiliated with the GPN or the GPN NOC!

For any issues, please contact DECT 6969 or directly @greenstheorem:matrix.org (DECT 4444 / SIP Backup 4445)

Pixelflut-Express

As the NOC cannot provide a sufficient number of high speed links and have made the explicit request to completely separate any Pixelflut traffic or network, we are running our own ultra-high-speed network called "Pixelflut-Express"

We explicitly ENCOURAGE using the Pixelflut-Express network for purposes other than Pixelflut! Want to run a high performance storage cluster? Need high speed ports for experiments? Anything else? Peer with us!


You cannot access the internet or any other parts of the GPN Network from the Pixelflut-Express network! It is completely isolated! If you want to access anything outside the Pixelflut-Express network, you MUST have multiple NICs. No, we cannot provide access to the GPN network within a VLAN or similar. We have explicitly been asked to NOT do that.

Pixel-Peering / Pixel-Colo

In order to take part in the Pixelflut, you must connect to the Pixelflut-Express network.

You can access the Pixelflut-Express network in the YoloColo and Hackcenter.

Please note that Pixelflut-Express and GPN networks are completely isolated. It is very likely that you want to connect to both networks at the same time and thus need 2 NICs. For information on how to connect to the GPN Network in YoloColo, please consult GPN22:NOC

In the YoloColo, we are running 2x Mellanox SN2700 switches in MLAG (subject to change - we can add more switches if required). That means you can connect your servers dual-homed! Dual Homing must happen via LACP in fast mode! If you have significant bandwidth requirements for inter-node traffic, you MUST NOT mix dual-homed and single-homed! The reason is that we don't want to put unnecessary pressure on the MLAG IPL

Each of those switches has 32x100G ports (downwards compatible to 40G / 25G / 10G), some of which can be broken out.

We will bring a limited number of cables and transceivers (100G-CWDM4 transceiver, 10GBase-LR transceivers, QSFP28 cables, QSFP28 breakout cables, QSFP to SFP adapters), however we cannot guarantee that we will have enough for everyone. If you have cables or transceivers, please bring them with you!

We will also bring a limited number of NICs (around 10x Mellanox ConnectX-4 CX456A 2x100G QSFP28, 4x Supermicro AOC-STG-i4S 4x10G SFP+ Intel XL710-BM1 based) which you can borrow in case you need them.

There will be a 400GBit/s (4x 100G) crosslink from YoloColo to the Hackcenter. We are running 1x Mellanox SN2410 (48x25G + 8x100G) and possibly 1x Ruckus ICX7650-48ZP (24x 10/5/2.5/1G + 24x1G) in the hackcenter. Currently we are not planning to provide QSFP28 ports in the hackcenter. If you need them, please let us know and we will see whether we can deploy another switch there

VLANs (e.g. for internal cluster traffic) are available on request. All ports will run in hybrid mode (VLAN 1 / untagged -> Pixelflut, all other VLANs -> tagged). Inter-VLAN routing is available on request

To allow for easier planning, please submit Peering requests in advance (see above for contact information). We will add them to this list

YoloColo Ports
Total Amount Type / Speed Dual-Homed? Name Purpose
Internal Internal Internal Internal Internal
8 QSFP28 / 100G N/A Pixelflut-Orga MLAG-IPL
4 QSFP28 / 100G Yes Pixelflut-Orga Crosslink YoloColo <-> Hackcenter
10 QSFP28 / 100G Yes @greenstheorem:matrix.org EPYC 7742 + Ryzen servers
7 QSFP28 / 40G No @jonasled:jonasled.de Jonasled PVE Ceph
Public Public Public Public Public
Placeholder Placeholder Placeholder Placeholder Placeholder

Currently left: 35 QSFP28 Ports

Hackcenter Ports
Total Amount Type / Speed Dual-Homed? Name Purpose
Internal Internal Internal Internal Internal
4 QSFP28 / 100G Yes Pixelflut-Orga Crosslink YoloColo <-> Hackcenter
2 QSFP28 / 100G Yes Pixelflut-Orga Access Switch (ICX7650-48ZP)
Public Public Public Public Public
Placeholder Placeholder Placeholder Placeholder Placeholder

Currently left: 48x SFP28 ports

VLANs
VLAN ID Name Purpose
1 / untagged Pixelflut-Orga Main-Pixelflut VLAN
Beispiel Beispiel Beispiel
Beispiel Beispiel Beispiel
Borrowed NICs
Überschrift Überschrift Überschrift Überschrift
Beispiel Beispiel Beispiel foo
Beispiel Beispiel Beispiel foo
Beispiel Beispiel Beispiel foo

IP addressing / VLANs

<To be determined, probably in 192.168.0.0/16>

We might be able to provide out of band access (only link between Pixelflut-Express network and GPN network) for management purposes via Wireguard Tunnels. This is TBD.