GPN22:Pixelflut: Unterschied zwischen den Versionen

Aktuelle Version vom 31. Mai 2024, 20:35 Uhr

TL;DR: A loose collective around @greenstheorem:matrix.org, @subjekt91:matrix.org, @jonasled:jonasled.de and more are hosting Pixelflut at GPN22!

How to take part:

- Come to our desk (drawing to be added - just look for the projection on the wall a bunch of fibers coming from the ceiling) and connect to our access switches on the desk (10GBase-T/5GBase-T/2.5GBase-T/1GBase-T or SFP28 25GBase / 10GBase)

- ASK US FIRST and THEN connect in Colo to our core switches

- VLAN is always untagged, you will get an IP via DHCP

- Start flooding 192.168.16.2:1234

Useful links:

https://github.com/TobleMiner/sturmflut

Long version:

A loose collective around @greenstheorem:matrix.org, @subjekt91:matrix.org, @jonasled:jonasled.de and more will host Pixelflut at GPN22!

We are not affiliated with the GPN or the GPN NOC!

For any issues, please contact DECT 6969 or directly @greenstheorem:matrix.org (DECT 4444 / SIP Backup 4445)

Pixelflut-Express

As the NOC cannot provide a sufficient number of high speed links and have made the explicit request to completely separate any Pixelflut traffic or network, we are running our own ultra-high-speed network called "Pixelflut-Express"

We explicitly ENCOURAGE using the Pixelflut-Express network for purposes other than Pixelflut! Want to run a high performance storage cluster? Need high speed ports for experiments? Anything else? Peer with us!

You cannot access the internet or any other parts of the GPN Network from the Pixelflut-Express network! It is completely isolated! If you want to access anything outside the Pixelflut-Express network, you MUST have multiple NICs. No, we cannot provide access to the GPN network within a VLAN or similar. We have explicitly been asked to NOT do that.

Pixel-Peering / Pixel-Colo

In order to take part in the Pixelflut, you must connect to the Pixelflut-Express network.

You can access the Pixelflut-Express network in the YoloColo and Hackcenter.

Please note that Pixelflut-Express and GPN networks are completely isolated. It is very likely that you want to connect to both networks at the same time and thus need 2 NICs. For information on how to connect to the GPN Network in YoloColo, please consult GPN22:NOC

In the YoloColo, we are running 2x Mellanox SN2700 switches in MLAG (subject to change - we can add more switches if required). That means you can connect your servers dual-homed! Dual Homing must happen via LACP in fast mode! If you have significant bandwidth requirements for inter-node traffic, you MUST NOT mix dual-homed and single-homed! The reason is that we don't want to put unnecessary pressure on the MLAG IPL

Each of those switches has 32x100G ports (downwards compatible to 40G / 25G / 10G), some of which can be broken out.

We will bring a limited number of cables and transceivers (100G-CWDM4 transceiver, 10GBase-LR transceivers, QSFP28 cables, QSFP28 breakout cables, QSFP to SFP adapters), which you can borrow. However we cannot guarantee that we will have enough for everyone. If you have cables or transceivers, please bring them with you!

We will also bring a limited number of NICs (around 10x Mellanox ConnectX-4 CX456A 2x100G QSFP28, 4x Supermicro AOC-STG-i4S 4x10G SFP+ Intel XL710-BM1 based) which you can borrow in case you need them.

There will be a 400GBit/s (4x 100G) crosslink from YoloColo to the Hackcenter. We are running 1x Mellanox SN2410 (48x25G + 8x100G) and possibly 1x Ruckus ICX7650-48ZP (24x 10/5/2.5/1G + 24x1G) in the hackcenter. Currently we are not planning to provide QSFP28 ports in the hackcenter. If you need them, please let us know and we will see whether we can deploy another switch there

VLANs (e.g. for internal cluster traffic) are available on request. All ports will run in hybrid mode (VLAN 1 / untagged -> Pixelflut, all other VLANs -> tagged). Inter-VLAN routing is available on request

To allow for easier planning, please submit Peering requests in advance (see above for contact information). We will add them to this list. Please do not edit those tables yourself! Only Pixelflut-Orga should edit those tables.

YoloColo Ports
Total Amount	Type / Speed	Dual-Homed?	Name	Purpose
Internal	Internal	Internal	Internal	Internal
8	QSFP28 / 100G	Yes	Pixelflut-Orga	MLAG-IPL
4	QSFP28 / 100G	Yes	Pixelflut-Orga	Crosslink YoloColo <-> Hackcenter
10	QSFP28 / 100G	Yes	@greenstheorem:matrix.org	EPYC 7742 + Ryzen servers
7	QSFP28 / 40G	No	@jonasled:jonasled.de	Jonasled PVE Ceph
4	QSFP28 / (100+40G)	No	@subjekt91:matrix.org	Subject91
Public	Public	Public	Public	Public
1/2	QSFP28/100G	Maybe	@masterx244:nplusc.de	Fujitsu @masterx244:nplusc.de
1/2	QSFP28/100G	Maybe	@markus:matrix.fablab-altmuehlfranken.de	@markus:matrix.fablab-altmuehlfranken.de
4	SFP+ (=> 2x QSFP28)	N/A	@tilcreator:computational.beauty	@tilcreator:computational.beauty
Placeholder	Placeholder	Placeholder	Placeholder	Placeholder

Currently left: ??? QSFP28 Ports

Hackcenter Ports
Total Amount	Type / Speed	Dual-Homed?	Name	Purpose
Internal	Internal	Internal	Internal	Internal
4	QSFP28 / 100G	N/A	Pixelflut-Orga	Crosslink YoloColo <-> Hackcenter
2	QSFP28 / 100G	N/A	Pixelflut-Orga	Access Switch (ICX7650-48ZP)
Public	Public	Public	Public	Public
2	QSFP28 / 100G	N/A	@aveexyliium:matrix.org	Downlink Quanta / SN2410
Placeholder	Placeholder	Placeholder	Placeholder	Placeholder

Currently left: 48x SFP28 ports

VLANs
VLAN ID	Name	Purpose
1 / untagged	Pixelflut-Orga	Main-Pixelflut VLAN
Beispiel	Beispiel	Beispiel
Beispiel	Beispiel	Beispiel

Borrowed NICs
NIC Type + Amount	Name
1x Mellanox ConnectX-4 CX456A (LP/FP?)	@masterx244:nplusc.de
1x Mellanox ConnectX-4 CX456A (LP/FP?)	@markus:matrix.fablab-altmuehlfranken.de
2x SFP+ NICs	@markus:matrix.fablab-altmuehlfranken.de / @t3chler:matrix.t3chler.de
Beispiel	Beispiel
Beispiel	Beispiel

IP addressing / VLANs

For receiving a IP address, please turn on DHCP. We are giving out IPs from the 192.168.0.0/19 space with a lease time of 7 days. Internet access is currently not planned.

We might be able to provide out of band access (only link between Pixelflut-Express network and GPN network) for management purposes via Wireguard Tunnels. This is TBD.

@@ Zeile 1: / Zeile 1: @@
+TL;DR: A loose collective around [https://matrix.to/#/@greenstheorem:matrix.org @greenstheorem:matrix.org], [https://matrix.to/#/@subjekt91:matrix.org @subjekt91:matrix.org], [https://matrix.to/#/@jonasled:jonasled.de @jonasled:jonasled.de] and more are hosting Pixelflut at GPN22!
+How to take part:
+- Come to our desk (drawing to be added - just look for the projection on the wall a bunch of fibers coming from the ceiling) and connect to our access switches on the desk (10GBase-T/5GBase-T/2.5GBase-T/1GBase-T or SFP28 25GBase / 10GBase)
+- ASK US FIRST and THEN connect in Colo to our core switches
+- VLAN is always untagged, you will get an IP via DHCP
+- Start flooding 192.168.16.2:1234
+Useful links:
+https://github.com/TobleMiner/sturmflut
+Long version:
 A loose collective around [https://matrix.to/#/@greenstheorem:matrix.org @greenstheorem:matrix.org], [https://matrix.to/#/@subjekt91:matrix.org @subjekt91:matrix.org], [https://matrix.to/#/@jonasled:jonasled.de @jonasled:jonasled.de] and more will host Pixelflut at GPN22!
@@ Zeile 19: / Zeile 40: @@
 Please note that Pixelflut-Express and GPN networks are completely isolated. It is very likely that you want to connect to both networks at the same time and thus need 2 NICs.
+For information on how to connect to the GPN Network in YoloColo, please consult [[GPN22:NOC]]
 In the YoloColo, we are running 2x Mellanox SN2700 switches in MLAG (subject to change - we can add more switches if required). That means you can connect your servers dual-homed!
 '''Dual Homing must happen via LACP in fast mode!'''
+'''If you have significant bandwidth requirements for inter-node traffic, you MUST NOT mix dual-homed and single-homed! The reason is that we don't want to put unnecessary pressure on the MLAG IPL'''
 Each of those switches has 32x100G ports (downwards compatible to 40G / 25G / 10G), some of which can be broken out.
-We will bring a limited number of cables and transceivers (100G-CWDM4 transceiver, 10GBase-LR transceivers, QSFP28 cables, QSFP28 breakout cables, QSFP to SFP adapters), however we cannot guarantee that we will have enough for everyone. If you have cables or transceivers, please bring them with you!
+We will bring a limited number of cables and transceivers (100G-CWDM4 transceiver, 10GBase-LR transceivers, QSFP28 cables, QSFP28 breakout cables, QSFP to SFP adapters), which you can borrow. However we cannot guarantee that we will have enough for everyone. If you have cables or transceivers, please bring them with you!
 We will also bring a limited number of NICs (around 10x Mellanox ConnectX-4 CX456A 2x100G QSFP28, 4x Supermicro AOC-STG-i4S 4x10G SFP+ Intel XL710-BM1 based) which you can borrow in case you need them.
@@ Zeile 34: / Zeile 57: @@
 VLANs (e.g. for internal cluster traffic) are available on request. All ports will run in hybrid mode (VLAN 1 / untagged -> Pixelflut, all other VLANs -> tagged). Inter-VLAN routing is available on request
-To allow for easier planning, please submit Peering requests in advance (see above for contact information). We will add them to this list
+To allow for easier planning, please submit Peering requests in advance (see above for contact information). We will add them to this list.
+<big>'''Please do not edit those tables yourself! Only Pixelflut-Orga should edit those tables.'''</big>
 {| class="wikitable sortable"
@@ Zeile 43: / Zeile 67: @@
 | Internal || Internal || Internal || Internal || Internal
 |-
-| 8 || QSFP28 / 100G || N/A || Pixelflut-Orga || MLAG-IPL
+| 8 || QSFP28 / 100G || Yes || Pixelflut-Orga || MLAG-IPL
 |-
 | 4 || QSFP28 / 100G || Yes || Pixelflut-Orga || Crosslink YoloColo <-> Hackcenter
@@ Zeile 50: / Zeile 74: @@
 |-
 | 7 || QSFP28 / 40G || No || @jonasled:jonasled.de  || Jonasled PVE Ceph
+|-
+| 4 || QSFP28 / (100+40G) || No || @subjekt91:matrix.org  || Subject91
 |-
 | Public || Public || Public || Public || Public
+|-
+| 1/2 || QSFP28/100G || Maybe || @masterx244:nplusc.de || Fujitsu @masterx244:nplusc.de
+|-
+| 1/2 || QSFP28/100G || Maybe || @markus:matrix.fablab-altmuehlfranken.de
+ || @markus:matrix.fablab-altmuehlfranken.de
+|-
+| 4 || SFP+ (=> 2x QSFP28) || N/A || @tilcreator:computational.beauty
+ || @tilcreator:computational.beauty
 |-
 | Placeholder || Placeholder || Placeholder || Placeholder || Placeholder
 |}
-Currently left: 35 QSFP28 Ports
+Currently left: ??? QSFP28 Ports
 {| class="wikitable sortable"
@@ Zeile 64: / Zeile 98: @@
 | Internal || Internal || Internal || Internal || Internal
 |-
-| 4 || QSFP28 / 100G || Yes || Pixelflut-Orga || Crosslink YoloColo <-> Hackcenter
+| 4 || QSFP28 / 100G || N/A || Pixelflut-Orga || Crosslink YoloColo <-> Hackcenter
 |-
-| 2 || QSFP28 / 100G || Yes || Pixelflut-Orga || Access Switch (ICX7650-48ZP)
+| 2 || QSFP28 / 100G || N/A || Pixelflut-Orga || Access Switch (ICX7650-48ZP)
 |-
 | Public || Public || Public || Public || Public
+|-
+| 2 || QSFP28 / 100G || N/A || @aveexyliium:matrix.org || Downlink Quanta / SN2410
 |-
 | Placeholder || Placeholder || Placeholder || Placeholder || Placeholder
@@ Zeile 89: / Zeile 125: @@
 |+ Borrowed NICs
 |-
-! Überschrift !! Überschrift !! Überschrift !! Überschrift
+! NIC Type + Amount !! Name
+|-
+| 1x Mellanox ConnectX-4 CX456A (LP/FP?) || @masterx244:nplusc.de
+|-
+| 1x Mellanox ConnectX-4 CX456A (LP/FP?) || @markus:matrix.fablab-altmuehlfranken.de
 |-
-| Beispiel || Beispiel || Beispiel || foo
+| 2x SFP+ NICs || @markus:matrix.fablab-altmuehlfranken.de / @t3chler:matrix.t3chler.de
 |-
-| Beispiel || Beispiel || Beispiel || foo
+| Beispiel || Beispiel
 |-
-| Beispiel || Beispiel || Beispiel || foo
+| Beispiel || Beispiel
 |}
 == IP addressing / VLANs ==
-<To be determined, probably in 192.168.0.0/16>
+For receiving a IP address, please turn on DHCP. We are giving out IPs from the 192.168.0.0/19 space with a lease time of 7 days. Internet access is currently not planned.
 We might be able to provide out of band access (only link between Pixelflut-Express network and GPN network) for management purposes via Wireguard Tunnels. This is TBD.