acf_: without realizing it won't participate
I can send 4GB of packets to any host and there isn't much it can do about it :P mercutio: 4gb isn't really that much these days
there are so many gigabit connected hackable hosts i suppose mhoran: I was amazed to discover 100 GbE interconnects when I was looking at Google's peering page. That definitely wasn't a thing when I worked at Cisco. I think 10GbE had just come out. mercutio: the 40 to 100 gbe shift was pretty quick
it's "reasonably affordable" to get 40 to 100 gigabit interconnects now
it's base 10 gigabit stuff that's struggling to come down mhoran: Yeah. mercutio: like if you want a 16 port 10 gigabit switch for a small company you can't really get anything cheap mhoran: Run fiber to every desk. 😂 mercutio: even 100 gigabit ethernet cards for pc are "reasonably affordable"
like $400 USD or something mhoran: Huh. mercutio: they're pci-e x16
mellanox make one with pcie 4
but hardly anything supports pcie 4 yet
apparently some amd boards may
https://www.ebay.com/itm/Arista-DCS-7280SE-72-F-48-Port-10GbE-100GbE-7280E-Switch-1-Year-Warranty/273398255690?hash=item3fa7ce504a:g:ltQAAOSw1dJbbIPL mhoran: Dang. mercutio: https://www.ebay.com/itm/Mellanox-MCX456A-ECAT-CX456A-ConnectX-4-VPI-EDR-IB-100GbE-Dual-Port-QSFP28-PCIe/323751423135?hash=item4b6116ac9f:m:mimSiA5cDCfvjWmI5dPNSeQ mhoran: I was mislead by the model number, totally thought that was a Cisco. mercutio: heh
arista are gaining market share rapidly mhoran: Yeah it's a name I never heard of and now keeps coming up. mercutio: so are huawei though
oh wow
https://www.ebay.com/itm/Arista-DCS-7280CR-48-F-7280R2-48x-100GbE-QSFP-8x-40GbE-QSFP-Switch-Router/401736132191?hash=item5d8956e65f:g:5woAAOSwYqVcb43U
surely it means 48 10gbe mhoran: Wow. mercutio: nope the data sheet says the same
i think google are actually using 100gbe to their servers?
but they're running their own switches i think
well someone probably makes them for them
but they spec what they want
i kind of wish infiniband was more popular
hmm this web site says 25gigabit is going to take off mhoran: Yeah I was pumped about that.
HUh. mercutio: i haven't seen anything that does 25 gigabit yet
there's also rdma over converged ethernet
but i don't think that's really taking off either
hmm this site says 400gbe should become big this y ear
oh no hangon it says shipments will start in december this year
damn this arista switch has 16gb of ram
and 24gb of packet buffer memory
si i suppose the packet buffer memory isn't included in the 16gb
oh it is 60x100gbe
i wonder when people will habitually have more than gigabit internet to the desktop ***: r0ni has joined #arpnetworks
hive-mind has quit IRC (Ping timeout: 244 seconds)
hive-mind has joined #arpnetworks
ziyourenxiang has quit IRC (Ping timeout: 276 seconds) acf_: I have a friend who's going to work at arista in a couple months
apparently they're really pushing the cutting edge right now mercutio: nice ***: ziyourenxiang has joined #arpnetworks
nathani has quit IRC (Quit: WeeChat 1.4) mnathani: @seen nathani BryceBot: mnathani, I last saw nathani 30 min 13 sec ago quitting: Quit: WeeChat 1.4. brycec: Huh... zeit tripped the "'eth0' total download 4.8 GB matches limit [download rate > 3 GB in last 1 hour]" alert at about the same time as yesterday...
No discernible spike in traffic traffic, not that it alerted anyways. But enough to trip the alert.
(Normal hourly rx traffic is 150MB, so 3GB is quite the anomaly.)
Not seeing anything out of the ordinary right now though. Drat.
mercutio: Are y'all moving ARP Metal right now? I thought I had until next week (and I was mid-OpenBSD-upgrade!)
...or apparently less time? Re-read the email, it says it was supposed to have been done last weekend.
uptime 5 minutes, yep... Seems like it.
Not cools, knocking me offline outside your prescribed maintenance window, guys! mercutio: what mtl host?
there's no reason that i know of that your server should have gone down acf_: seeing connectivity problems here
on arp metal mercutio: ipv6? ipv4?
packet loss or unable to reach? brycec: s/minutes/hours BryceBot: <brycec> uptime 5 hours, yep... Seems like it. acf_: http://paste.debian.net/1079242/ brycec: mercutio: stl21 acf_: trace to 8.8.8.8
lots of problems the past few weeks
I might need to move stuff to ec2 temporarily or something mercutio: hmm i can trace 8.8.8.8 fine acf_: I cannot, from 174.136.111.132 mercutio: hmm i'm in taht same /24 where i'm testing from acf_: I'm 95% sure this isn't a problem with my system mercutio: ho hmm
yeh i can trace from arp to there but not from home
checking brycec: My stl21 machine is very unhappy about life - at a mininum its networking is funar. Not sure what yet, still working through logs to see what happened 5 hours ago, why it mysteriously shutdown acf_: note that I *can* reach some internet destinations brycec: (systemd didn't even start!) acf_: hence how I can ssh into this in the first place mercutio: there was a second round of migrations earlier today but there were hardly any machines in that acf_: but it's broken cloudfront and something else v important mercutio: it was around 8 hours ago though
and customers involved were emailed
i don't think your one was ni that list acf_: afaict this seems to have started roughly 2 hours ago mercutio: yeah taht's odd brycec: mercutio: turns out it was, but Garry's emails went to my wrong folder mercutio: oh acf_: gotta run for a couple minutes but I'll be back
(still broken of course) brycec: fuck fuck fuck mercutio: did monitoring not alert you? brycec: mercutio: I'm not sure what's up with that monitoring mercutio: this is odd
i'm only seeing your other subnet with arp entries acf
is your other subnet working fine? up_the_irons: acf_: let me know if you see any difference now acf_: mercutio: yeah, my 174.136.x.x subnet is routed via the other one
so it would make sense for there to be no arp entries for that mercutio: yeah i clicked after taht acf_: oh ok looks like it might be back? mercutio: sweet acf_: yeah afaict it's good now mercutio: it would have been affecting traffic from one router in but not the other
that's why some stuff worked and some didn't
and only that /27 acf_: ahhh interesting up_the_irons: We don't have OSPF redistributing static routes
and we have very, very few customers with static routes
So we didn't see this at first
once we have all customers moved to the new switch, this issue will not appear again
we're almost done
This happened a couple hours ago because I started announcing our IP blocks out a 2nd BGP session, for redundancy, and traffic going into that router then didn't know how to get to your /27
So I've just removed those announcements for now acf_: ahh I see
yeah as soon as mercutio mentioned it only affected my /27 I thought it had something to do with the static route haha