[00:06] *** gizmoguy has quit IRC (Ping timeout: 245 seconds)
[00:07] *** gizmoguy has joined #arpnetworks
[00:25] <mnathani> If my quota limit says: 20000000
[00:25] <mnathani> how many gigs is that?
[00:27] <brycec> @wa 20000000B to GB
[00:27] <BryceBot> convert 20000000 bytes to gigabytes;0.02 GB  (gigabytes);20 MB  (megabytes);160 Mb  (megabits);0.16 Gb  (gigabits);1.6×10^8 bits;19.07 MiB  (mebibytes);~~ ( 0.1 ~~ 1\/10 ) × mini-compact disc capacity ( 185 to 210 MB );~~ 7 × data storage capacity of a 200-character per inch 2400-foot IBM 7330 7-track magnetic tape (~~ 2×10^7 b );~~ 14 × 3.5" floppy disk capacity (high density) ( 1440 kB );information;[information];Time to transfer at T1 spe
[00:27] <brycec> Apparently 20MB?
[00:28] <brycec> mnathani: or just quota -s
[00:30] <brycec> (assuming GNU quota)
[00:30] <brycec> But yeah, quota uses bytes normally
[00:31] <mnathani> I think its in blocks
[00:31] <mnathani> approx 10 gigs
[00:31] <mnathani> @wa 20000000 blocks to gb
[00:31] <BryceBot> convert 20000000 block sizes for a typically formatted magnetic disk to gigabytes;10.24 GB  (gigabytes);0.01024 TB  (terabytes);1.024×10^10 bytes;81.92 Gb  (gigabits);0.08192 Tb  (terabits);8.192×10^10 bits;~~ 0.41 × single-layer Blu-ray capacity (~~ 25 GB );~~ 0.51 × AIT-E turbo native capacity ( 20 GB );~~ 1.2 × dual-layer DVD capacity (~~ 8.5 GB );information;[information]
[00:42] <mnathani> this was the quota command on ARP backup system
[00:43] <mnathani> Quota utilities version 3.17.
[01:03] <mnathani> @wa 10 GB in blocks
[01:03] <BryceBot> Sorry, I couldn't reach the backend API.
[01:03] <mnathani> @iata yyz
[01:03] <BryceBot> YYZ: Toronto Pearson International Airport located in Mississauga, Ontario, Canada
[01:04] <mnathani> @iataabc
[01:04] <mnathani> @iat aabc
[01:04] <mnathani> @iata abc
[01:04] <BryceBot> ABC: Not a valid IATA.
[01:04] <mnathani> @iata hnl
[01:04] <BryceBot> HNL: Honolulu International Airport located in Honolulu, Hawaii, United States
[01:04] <mnathani> @iata bom
[01:04] <BryceBot> BOM: Chatrapati Shivaji International Airport located in Mumbai, India
[01:26] <brycec> blocks are somewhat arbitrary though
[01:26] <brycec> and usually abstracted beneath the FS
[01:26] <mnathani> can anyone ping M.ROOT-SERVERS.NET [2001:dc3::35] (works from ARP, but not from my HE.net tunnel)
[01:27] <mnathani> @smokeping
[01:27] <BryceBot> https://smokeping.cobryce.com/
[01:28] <brycec> works over my HE tunnel
[01:28] <brycec> And no I'm not hitting the root servers (yet) from my smokeping
[01:28] <mnathani> thats wierd, I can ping it from the linux box that has the tunnel setup
[01:28] <brycec> mnathani: on ARP's backup system, "limit" is bytes (you have 20GB) and "blocks" are also bytes"
[01:29] <mnathani> I should only have 10 GB though
[01:29] <mnathani> thats what the portal says
[01:29] <brycec> Well I have 20000000 and the portal says 20GB.
[01:29] <brycec> What's more, du says 8.5GB used, and quota says 8870132 blocks used.
[01:30] <brycec> so aside from a base-2/10 discrepency
[01:30] <brycec> , it's all kosher
[01:30] <brycec> at least for me
[01:31] <mnathani> My du = 7.6G    .
[01:31] <mnathani> portal says 10GB
[01:31] <mnathani> and quota is 20000000
[01:31] <mnathani> probably a typo on up_the_irons part when provisioning
[01:31] <brycec> could be
[01:36] <mnathani> I can ping 2001:4860:4860::8888 just fine
[01:36] <mnathani> from my Windows 8.1 box that is routed via Linux to HE.net
[01:36] <mnathani> but 2001:dc3::35
[01:36] <mnathani> not able to ping
[01:37] * brycec blindly blames Windows
[01:38] <mnathani> is there a different protocol
[01:38] <mnathani> between the windows ping
[01:38] <mnathani> and linux ping
[01:39] <mnathani> not just my windows 8
[01:39] <mnathani> I also have a centos behind the original linux box with the same behavior
[01:39] <brycec> Doesn't one still use ICMP ping?
[01:40] <gizmoguy> mnathani: what does a traceroute or mtr look like?
[01:40] <brycec> vs UDP ping?
[01:40] <gizmoguy> I'm thinking it's either a routing problem or an MTU problem
[01:41] <mnathani> http://pastebin.com/YiirLqmv
[01:41] <mnathani> why would it work from the linux he.net tunnel box
[01:42] <gizmoguy> what does a traceroute from the linux he.net tunnel box look like out of interest?
[01:42] <mnathani> to m-root?
[01:42] <gizmoguy> yeah
[01:42] <mnathani> its in the same pastebin
[01:42] <gizmoguy> just to compare
[01:42] <mnathani> at the bottom
[01:42] <gizmoguy> lol
[01:42] <gizmoguy> sorry
[01:42] <mnathani> np
[01:42] <gizmoguy> I didn't scroll my firefox window
[01:43] <mnathani> I put a bunch of blank lines
[01:43] <gizmoguy> yup
[01:43] <gizmoguy> well that's strange
[01:43] <mnathani> I would expect not to ping any ipv6
[01:43] <mnathani> or all pingable ipv6
[01:43] <mnathani> rather than ping most and not all
[01:44] <gizmoguy> if you want to replicate windows's traceroute on linux
[01:44] <gizmoguy> try "sudo traceroute -I M.ROOT-SERVERS.NET"
[01:44] <mnathani> I have another linux box behind
[01:44] <mnathani> there is no traceroute command on windows
[01:44] <mnathani> oh ok
[01:44] <mnathani> I see
[01:44] <gizmoguy> though I was pretty sure windows tracert used UDP
[01:45] <gizmoguy> I wonder if it's an issue with source IP?
[01:45] <mnathani> how come other destinations like google and facebook come back
[01:45] <gizmoguy> are all your machines on the same /64?
[01:45] <gizmoguy> true.
[01:45] <mnathani> they are
[01:45] <mnathani> how do you source a ping from a different interface /ip on linux
[01:46] <brycec> -I $IP, as I recall
[01:47] <mnathani> confirmed
[01:48] <mnathani> its the source ip
[01:48] <mnathani> http://pastebin.com/CcV0dxy7
[01:48] <gizmoguy> woo 10 points to gizmoguy
[01:48] <mnathani> ping to google from he.net linux box sourced from /64 within routed /48
[01:49] <mnathani> works, however same source to m-root fails
[01:49] <mnathani> could there be a bogon
[01:49] <mnathani> or source address filtering going on?
[01:49] <gizmoguy> i'll try ping you from here
[01:50] <gizmoguy> ping6 2001:470:b148::1
[01:50] <gizmoguy> PING 2001:470:b148::1(2001:470:b148::1) 56 data bytes
[01:50] <gizmoguy> From 2001:470:0:1b4::2 icmp_seq=1 Time exceeded: Hop limit
[01:50] <mnathani> From 2001:470:0:1b4::1 icmp_seq=1 Time exceeded: Hop limit
[01:50] <gizmoguy> yeah, i think you have some issues
[01:50] <mnathani> lol, I get the same thing
[01:50] <mnathani> never thought of pinging from ARP before
[01:51] <gizmoguy> http://pastebin.com/9FqhpmyN
[01:51] <gizmoguy> so HE have a routing loop for your /64
[01:51] <mnathani> I could pick a different /64
[01:52] <gizmoguy> oddly enough, this is the second routing loop in the past 10 mins i've helped solve
[01:53] <mnathani> 2001:470:b148:0:1::1
[01:53] <mnathani> thats in a different /64
[01:53] <mnathani> ?
[01:54] <mnathani> my routed /48" 2001:470:b148::/48
[01:54] <gizmoguy> same issue
[01:55] <gizmoguy> it's probably an issue with the /48
[01:57] <mnathani> yea
[01:57] <mnathani> I have a routed 64
[01:57] <mnathani> as well
[01:57] <mnathani> which I tried using, and that seems to work just fine
[01:57] <mnathani> 2001:470:1d:76e::1
[01:57] <mnathani> 2001:470:1d:76e::25
[01:59] <mnathani> thanks gizmoguy :-)
[02:01] <gizmoguy> np :)
[03:35] <brycec> Fuck this is good 03.30 pie. Nothing like celebratory pie after fixing the "oh shit everything went to hell Thursday at midnight and I've been working ever since to get things back online" fiasco
[03:35] <brycec> I've had some of this pie the other day... but *now* it tastes 10x better
[03:37] <gizmoguy> mmm pie
[03:41] * brycec should've gotten some milk
[06:35] *** novae has quit IRC (Ping timeout: 252 seconds)
[06:38] *** novae has joined #arpnetworks
[06:38] *** novae has quit IRC (Changing host)
[06:38] *** novae has joined #arpnetworks
[06:48] <m0unds> late to the party, but windows traceroute = tracert, also you can get winmtr which is..well, you guessed it, mtr for windows.
[06:50] <staticsafe> winmtr doesn't do v6 :(
[06:51] <m0unds> hasn't been updated in years either
[06:54] <m0unds> my rust server has been invaded by korean, chinese and russian people
[06:54] <staticsafe> :o
[06:54] <m0unds> i set up a log notifier that sends server connect/disconnect and chat stuff to an irc channel
[06:55] <m0unds> every night, between 0000-1000, it's nothing but russian, chinese and korean chat
[06:55] <phlux> We're gowngrading to 1.5Mbps internet out of spite for the company we're currently with (only one other provider)
[06:55] <phlux> rip Netflix streaming
[06:56] <m0unds> could just stream super ugly video at 500kbit/sec
[06:56] <phlux> I'll probably just move from streaming to DVDs/blurays and rip them as soon as they get here
[06:57] <m0unds> ah, dlna streaming server or something?
[06:57] <m0unds> mkv -> tv?
[06:57] <phlux> yeah, via ps3
[06:57] <m0unds> right on
[06:58] <phlux> tbh, I'm fine with 1.5Mbps until there's a larger market out here as long as I can still get on IRC :P
[06:59] <m0unds> are you in a rural area? or is your choice of providers just really shitty vs super shitty?
[06:59] <phlux> I'm in a new development on a military installation
[07:00] <phlux> There are only two companies that offer service out here so far. AT&T offers U-Verse one street away, so I'm hoping that comes our way soon
[07:00] <phlux> For now, we've been using this company called "CMA Access"
[07:00] <m0unds> ah
[07:00] <phlux> They are expensive, and their staff won't believe me when I say problems are on their end
[07:00] <phlux> So I'm done with them
[07:01] <m0unds> gross
[07:01] <phlux> I got into it with a customer service rep and decided "Fuck it"
[07:01] <phlux> Sooooooooo out of spite, we're moving to AT&T DSL until U-Verse is offered again
[07:01] <m0unds> hah
[07:01] <phlux> Thankfully I already have DirecTV for my television needs
[07:03] <phlux> but I'm fairly certain I can live with it..my wife's parents live out in the sticks, and I'm pretty sure all they get is 1.5Mbps and it seems to work fine
[07:03] <phlux> OH, and CMA has a bandwidth limit where AT&T does not
[07:03] <phlux> so that's another plus
[07:04] <phlux> Of course, I won't be able to use too much bw only getting 150kb/s down :P
[07:04] <m0unds> hah
[07:05] <m0unds> at least you're not stuck with satellite stuff
[07:05] <phlux> Very true
[07:06] <phlux> tbh, as long as I can check e-mail, reddit, and IRC without having to restart my modem 500000 times a day, I'll be happier overall.
[07:07] <phlux> Plus I have a son due in April, and AT&T's price is only $24/mo
[07:08] <m0unds> wow, that's cheaper than a pots line here w/no additional svcs
[07:08] <m0unds> also, congrats
[07:09] <m0unds> april birthdays ftw
[07:16] <phlux> thanks man
[07:48] *** hive-mind has quit IRC (Ping timeout: 264 seconds)
[08:21] *** jcv has joined #arpnetworks
[08:42] *** novae has quit IRC (Ping timeout: 252 seconds)
[08:45] *** novae has joined #arpnetworks
[08:53] *** hive-mind has joined #arpnetworks
[10:59] <m0unds_> network just take a dump for anyone else?
[10:59] <m0unds_> seeing ~75-80% pkt loss
[11:01] *** RandalSchwartz has quit IRC (Ping timeout: 272 seconds)
[11:01] *** acf_ has quit IRC (Ping timeout: 264 seconds)
[11:02] *** qbit has quit IRC (Ping timeout: 264 seconds)
[11:02] *** tabthorpe has quit IRC (Ping timeout: 240 seconds)
[11:03] *** d^_^b has quit IRC (Ping timeout: 264 seconds)
[11:03] *** tabthorpe has joined #arpnetworks
[11:03] *** tabthorpe has quit IRC (Changing host)
[11:03] *** tabthorpe has joined #arpnetworks
[11:04] *** KDE_Perry has quit IRC (Ping timeout: 265 seconds)
[11:04] *** KDE_Perry has joined #arpnetworks
[11:05] *** tabthorpe has quit IRC (Read error: Connection reset by peer)
[11:05] *** tooth has quit IRC (Ping timeout: 240 seconds)
[11:07] *** tooth has joined #arpnetworks
[11:07] *** notion has quit IRC (Read error: Connection reset by peer)
[11:07] *** medum has quit IRC (Ping timeout: 252 seconds)
[11:08] *** medum has joined #arpnetworks
[11:08] *** CaZe has quit IRC (Ping timeout: 245 seconds)
[11:08] <mnathani> over v4 or v6?
[11:08] *** d^_^b has joined #arpnetworks
[11:08] *** d^_^b has quit IRC (Changing host)
[11:08] *** d^_^b has joined #arpnetworks
[11:08] <mnathani> and which transit / peer are you coming in on?
[11:09] <m0unds_> any
[11:09] <m0unds_> ntt from one location, mzima from one, nlayer from one
[11:09] *** CaZe has joined #arpnetworks
[11:09] <m0unds_> 50% on nlayer, 80+% on ntt via v4
[11:09] <SpaceDump> Mmm, seems to be a bit crappy at the moment.
[11:10] <SpaceDump> ~95% packet loss at the moment. (v4)
[11:10] <SpaceDump> Oh, 97% now. :D
[11:10] *** pjs has quit IRC (Ping timeout: 265 seconds)
[11:10] <SpaceDump> Oh well oh hell. It will sort it self out soon.
[11:10] <m0unds_> yep
[11:11] <m0unds_> lol, oh man
[11:11] <m0unds_> i need to put my v6 tunnel offline
[11:11] <m0unds_> it's coming up and dropping over and over
[11:11] *** raptelan has quit IRC (Remote host closed the connection)
[11:11] *** thestereobus has joined #arpnetworks
[11:11] *** raptelan has joined #arpnetworks
[11:12] *** twobithacker has quit IRC (Quit: ZNC - http://znc.in)
[11:14] *** mhoran2 has quit IRC (Ping timeout: 244 seconds)
[11:14] *** CaZe has quit IRC (Ping timeout: 240 seconds)
[11:14] *** d^_^b has quit IRC (Ping timeout: 240 seconds)
[11:14] *** eryc has quit IRC (Ping timeout: 244 seconds)
[11:16] *** CaZe has joined #arpnetworks
[11:16] *** pcn has quit IRC (Ping timeout: 252 seconds)
[11:17] *** mnathani has quit IRC (Ping timeout: 264 seconds)
[11:17] *** eryc has joined #arpnetworks
[11:17] *** jm|laptop has joined #arpnetworks
[11:17] <jm|laptop> hello :)
[11:18] <staticsafe> hi
[11:18] <jm|laptop> are there known issues?
[11:18] <thestereobus> I'm seeing dropped packets
[11:18] *** qbit has joined #arpnetworks
[11:18] <jm|laptop> 42 packets transmitted, 2 received, 95% packet loss, time 49886ms
[11:18] <jm|laptop> rtt min/avg/max/mdev = 252.426/253.863/255.300/1.437 ms
[11:18] <staticsafe> yah
[11:18] <jm|laptop> 70.5.250.129.in-addr.arpa. 14400 IN	PTR	ae-2.r04.lsanca03.us.bb.gin.ntt.net.
[11:18] <jm|laptop> stops here
[11:20] *** koan has quit IRC (Ping timeout: 252 seconds)
[11:20] *** koan has joined #arpnetworks
[11:20] *** koan has quit IRC (Changing host)
[11:20] *** koan has joined #arpnetworks
[11:21] *** hive-mind has quit IRC (Ping timeout: 264 seconds)
[11:21] *** thestereobus has quit IRC (Quit: thestereobus)
[11:22] *** Surface_RT has joined #arpnetworks
[11:23] <Surface_RT> hi!
[11:25] *** milki has quit IRC (Ping timeout: 264 seconds)
[11:25] <staticsafe> shit is flapping
[11:26] <m0unds_> ja
[11:26] *** mike-burns has quit IRC (Ping timeout: 252 seconds)
[11:26] *** mhoran2 has joined #arpnetworks
[11:26] *** ChanServ sets mode: +o mhoran2
[11:27] *** anis is now known as anisfarhana
[11:27] *** anisfarhana has quit IRC (Changing host)
[11:27] *** anisfarhana has joined #arpnetworks
[11:27] *** thestereobus has joined #arpnetworks
[11:28] <Surface_RT> figured
[11:28] <Surface_RT> hi static
[11:28] <anisfarhana> o.O , any maintenance is on going?
[11:28] *** andrew32_ has joined #arpnetworks
[11:28] <andrew32_> hey
[11:28] <Surface_RT> not just you
[11:28] <Surface_RT> hi
[11:28] <andrew32_> my vps is down
[11:29] <andrew32_> :)
[11:29] <andrew32_> is something going on? :)
[11:29] <anisfarhana> Ah what a relief. So its not only me then :D
[11:29] <Surface_RT> suddenly, people
[11:29] <andrew32_> :)
[11:29] <anisfarhana> One of tech staff accidentally pulled out the cable maybe
[11:29] * anisfarhana ducks
[11:30] <m0unds_> arpnetworks employs ducks?
[11:30] <andrew32_> hope they can quickly plug it in again ;)
[11:30] <SpaceDump> It's most likely just a ddos.
[11:30] <SpaceDump> Will sort itself out sooner or later.
[11:30] <andrew32_> aaah
[11:30] <m0unds_> s7 has had issues, could be hw or something wigging out again
[11:30] <andrew32_> one of those nasty ntp reflection ones ...
[11:30] <andrew32_> :P
[11:31] <anisfarhana> No more fancy uptime :(
[11:31] <SpaceDump> ?
[11:31] <SpaceDump> The uptime shouldn't be affected. :p
[11:32] <anisfarhana> If they have hardware failure probably uptime will be affected.
[11:32] <SpaceDump> Yeah. that's true.
[11:32] <SpaceDump> But it's unlikely.
[11:32] <Surface_RT> ducks don't require: healthcare, pension, benefits
[11:32] <Surface_RT> very cost efficient and great ROI for shareholders
[11:32] <SpaceDump> Since the pl starts 2 hops before my instance...
[11:32] <Surface_RT> my brain internally read that 'since the polish... perl... giving up starts 2 hops before'
[11:35] <m0unds_> oh look. rain.
[11:39] *** iain has joined #arpnetworks
[11:39] <iain> anyone around
[11:39] <staticsafe> yep
[11:39] <iain> everyone else down?
[11:39] <m0unds_> yep
[11:39] <SpaceDump> Well, lot's of people around, no staff though as it seems.
[11:40] <iain> ugh
[11:40] *** hive-mind has joined #arpnetworks
[11:40] <m0unds_> v6 inbound is good via nlayer (S3, not S7)
[11:40] *** d^_^b has joined #arpnetworks
[11:40] *** d^_^b has quit IRC (Changing host)
[11:40] *** d^_^b has joined #arpnetworks
[11:40] <m0unds_> i'd bet something happened with s7 again, since it's been screwy for a couple weeks
[11:41] <SpaceDump> Good, then my ipv6 anycast dns node should be working at least .:D
[11:41] <SpaceDump> m0unds_: I have no idea what's s7 is. But imho it looks more like a ddos then anything else.
[11:41] <m0unds_> s7 is a switch chassis that has been crashing regularly
[11:41] <SpaceDump>  9. ae-2.r04.lsanca03.us.bb.gin.ntt.net                                         0.0%   709  151.0 151.2 149.8 176.9   2.3
[11:42] <SpaceDump> 10. ge-0-7-0-24.r04.lsanca03.us.ce.gin.ntt.net                                 87.0%   709  220.3 217.6 212.4 306.7  14.0
[11:42] <SpaceDump> 11. 208.79.88.129                                                              98.4%   709  212.9 212.9 211.6 217.4   1.7
[11:42] <m0unds_> ntt feeds into s7
[11:42] <SpaceDump> Works like a charm until hop 10 there.
[11:42] <m0unds_> afaik, anyway since when it dies, any2ix and ntt both die
[11:42] <Surface_RT> so we should take s7 out back and release it like a badly trained pokemon
[11:45] *** rpaulo has joined #arpnetworks
[11:45] <rpaulo> hi
[11:45] <rpaulo> I'm having some connection issues to arpnetworks.
[11:45] <Surface_RT> evrery one is
[11:45] <m0unds_> yeah, there's some sort of network event going on
[11:46] <rpaulo> ok, thanks for letting me know.
[11:48] *** thestereobus has quit IRC (Quit: thestereobus)
[11:53] *** wallshot has joined #arpnetworks
[11:58] <rpaulo> fixed?
[11:58] *** tabthorpe has joined #arpnetworks
[11:58] *** tabthorpe has quit IRC (Changing host)
[11:58] *** tabthorpe has joined #arpnetworks
[11:58] <m0unds_> looks good
[11:58] <wallshot> oooh i can get to my site again
[11:58] *** twobithacker has joined #arpnetworks
[11:58] *** notion_ has joined #arpnetworks
[11:59] <SpaceDump> Yay, my anycast node works again. :D
[12:00] *** pcn has joined #arpnetworks
[12:01] *** milki has joined #arpnetworks
[12:01] *** acf_ has joined #arpnetworks
[12:18] <mercutio> what happedned this morining?
[12:19] <Surface_RT> idk
[12:19] <mercutio> oh so s7 may have crashed
[12:19] <ant> 21:09:36 <arpnetworks> [2e] We're investigating very high traffic (likely DoS) to host kvr24, also consuming resources in other
[12:19] <m0unds_> actually, it says possible ddos
[12:19] <ant>                        areas
[12:20] <Surface_RT> that isnt an irc nick
[12:20] <ant> that's what twitter said
[12:20] <Surface_RT> wait, is that a twitter to irc gateway
[12:20] <ant> i pasted that from bitlbee
[12:20] <mercutio> ahh
[12:21] <m0unds_> yeah, so maybe the ios upgrade did fix s7 if it didn't crash under ddos, haha
[12:21] <mercutio> oh well at least it being looked into
[12:21] <mercutio> i'm sure we'll hear what happened
[12:21] <m0unds_> yup
[12:21] <m0unds_> yeah
[12:21] <mercutio> and prob better than s7 crash :)
[12:21] <m0unds_> yeah
[12:22] *** thestereobus has joined #arpnetworks
[12:24] <up_the_irons> m0unds_: lol re s7
[12:25] <m0unds_> up_the_irons: did it survive the onslaught?
[12:37] *** Surface_RT has quit IRC (Quit: leaving)
[12:39] <up_the_irons> m0unds_: it did
[12:39] <up_the_irons> m0unds_: although it has survived others, even on older IOS.
[12:41] <up_the_irons> wow, all BGP sessions on BIRD remained up.  That's quite cool.
[12:41] <up_the_irons> maybe it wasn't a high pps attack
[12:41] <up_the_irons> just high traffic
[12:43] <mercutio> i imagine any2ix has less ddos traffic volume?
[12:47] <m0unds> there we go
[12:53] <up_the_irons> mercutio: it was pretty big though, coming through peers as well
[13:02] *** thestereobus has quit IRC (Ping timeout: 264 seconds)
[13:02] *** RandalSchwartz has joined #arpnetworks
[13:02] *** RandalSchwartz has quit IRC (Changing host)
[13:02] *** RandalSchwartz has joined #arpnetworks
[13:07] *** pjs has joined #arpnetworks
[13:07] *** pjs is now known as Guest51353
[13:08] <mercutio> ahh weird
[13:10] <mercutio> i suppsoe it not that strange
[13:10] <mercutio> these things are really distributed these days
[13:11] <mercutio> i just kind of assume ddos traffic comes from somefar "far away"
[13:11] <mercutio> s/somefar/somewhere/
[13:11] <BryceBot> <mercutio> i just kind of assume ddos traffic comes from somewhere "far away"
[13:12] *** mnathani has joined #arpnetworks
[13:15] <mercutio> ho hmm, smokeping is pretty bad
[13:15] <up_the_irons> it comes from "everwhere" these days
[13:16] <mercutio> i thought that was just spoofed ip's mostly
[13:16] <mercutio> and that it was usually still few hosts
[13:16] <mercutio> but yeah now with ntp reflection attacks..
[13:16] <mercutio> any idea what kind of ddos it was?
[13:16] <m0unds> i read something that 75% of the ntpds implicated in the attack on cloudflare have either been filtered or reconfigured
[13:17] <mercutio> i bet more of them have been filtered than reconfigured
[13:17] <m0unds> that'd be my guess too
[13:17] <m0unds> but at least the network ops are paying attention
[13:17] <m0unds> haha
[13:17] <mercutio> i think there's not much choice
[13:17] <m0unds> well, there's two choices - address it or don't
[13:17] <mercutio> actually..
[13:17] *** robonerd- has joined #arpnetworks
[13:17] <mercutio> i think most network op's care about painful ddos/packet loss etc
[13:18] *** robonerd- has quit IRC (Client Quit)
[13:18] <mercutio> yeh true, i wonder if ntt, level3, etc are doing anything about it
[13:18] *** robonerd has quit IRC (Ping timeout: 264 seconds)
[13:18] <mercutio> i imagine most small-to-medium providers are
[13:19] <mercutio> but large providers may be like, we can charge more for bandwidth..
[13:19] *** rpaulo has quit IRC (Quit: rpaulo)
[13:19] <mercutio> and it's not necesarily tier1's responsibility to block
[13:19] *** robonerd has joined #arpnetworks
[13:19] *** robonerd has quit IRC (Changing host)
[13:19] *** robonerd has joined #arpnetworks
[13:19] <mercutio> it gets into iffy territory
[13:20] * mercutio hasn't done any blocking personally yet
[13:26] *** robonerd has quit IRC (Read error: Connection reset by peer)
[13:26] *** robonerd has joined #arpnetworks
[13:26] *** robonerd has quit IRC (Changing host)
[13:26] *** robonerd has joined #arpnetworks
[13:27] *** mike-burns has joined #arpnetworks
[13:27] *** ChanServ sets mode: +o mike-burns
[13:27] <brycec> Yup, that was quite a chunk of disruption earlier.  <obligatory smokeping graph: https://smokeping.cobryce.com/images/ARP/ARPWebsite_mini.png>
[13:28] <m0unds> ugly
[13:28] <mercutio> that looks milkd to me?
[13:29] <m0unds> i saw a ton of extra flow sessions open up w/my srx trying to reestablish the tunnel, til i took it down
[13:29] <mercutio> http://postimg.org/image/yd2ezgmhn/
[13:29] <m0unds> pretty funny
[13:29] <m0unds> roughly....8 times as many as normal
[13:31] <mercutio> maybe i was just unlucky
[13:31] <m0unds> my route to one IP is via SJC, and the other is straight to LAX - both nlayer. the LAX one was ~85% max pkt loss, the SJC one was 75.50%
[13:32] <mercutio> it looks like i was 95 to 100% packet loss
[13:32] <mercutio> via ntt
[13:32] <m0unds> that's what i saw via ntt too (work uses centurylink -> ntt)
[13:32] <mercutio> so ntt went out harder
[13:32] <m0unds> yeah, seems that way at least
[13:32] <mercutio> joy
[13:33] <mercutio> does ntt have blackhole community?
[13:33] <m0unds> there was so much loss via ntt that i thought my vps had crashed or something at first because i wasn't getting any icmp replies whatsoever
[13:33] <m0unds> it took almost 2 mins before i started seeing 1 or 2 here and there
[13:33] <mercutio> looks like it has standard 666
[13:33] <mercutio> By default, peers are not configured for the blackhole functionality. Please contact the NTT NOC @ noc@us.ntt.net for this feature.
[13:34] <up_the_irons> mercutio: m0unds : i say filtered too.  We filtered like 150 hosts.  there's NO WAY one can just "wait" for the host admin to fix it
[13:34] <m0unds> yeah, for sure
[13:34] <mercutio> up_the_irons: lots of people are businesses etc
[13:35] <mercutio> that may be able to fix themselves or filter
[13:35] <mercutio> err lots of networks are businesses
[13:35] <mercutio> still probably quicker to filter first
[13:35] <m0unds> yup, as a reactive measure
[13:35] <m0unds> and i guess preventative if there are systems that hadn't participated yet
[13:35] <up_the_irons> yeah but think about VMs here.  lots may be businesses, but also lots are just home / hobby stuff
[13:36] <mercutio> i imagine most participated
[13:36] <m0unds> yep
[13:36] <mercutio> up_the_irons: but you're a provider to others
[13:36] <mercutio> i mean if someone had their own network
[13:36] <up_the_irons> yeah
[13:36] <mercutio> ntp being a problem all over the place
[13:36] <up_the_irons> right
[13:36] <mercutio> if you have no control of course you're going to filter.
[13:36] <mercutio> if you do have control you might or might not filter
[13:37] <mercutio> upgrading ntp everywhere or fixing configs everywhere could be consdiered time-consuming
[13:37] <mercutio> i wonder how big that ddos was
[13:37] <mercutio> i've been hearing from a few various people getting hit by 10 gigabit+ ddos attacks recently
[13:37] <mercutio> for small providers.
[13:38] <mercutio> with less than 10 gigabit transit
[13:38] <mercutio> it used to be considered reasonable safe for ddos attacks if you had 10 gigabit pipes or bigger
[13:39] <mercutio> but it's probably shifting to 40 gigabit for safety or something now, which gets crazy if your average traffic is < 1 gigabit
[13:39] * brycec switches to 100mbps "If you can't take care of this Internet, you don't deserve to have good Internet." P
[13:39] <m0unds> hahah
[13:40] <m0unds> all your smokeping icmp traffic would clog the tuubs
[13:40] <mercutio> 100 people with 100 mergabit ddos is still 10 gigabit :/
[13:40] <m0unds> save some internet for the rest of us
[13:40] <m0unds> jeez
[13:40] <mercutio> i can't type today
[13:41] <mercutio> brycec: do you have any2ix/any2ix host?
[13:41] <mercutio> on your smokeping
[13:41] <mercutio> i'm curious what packet loss was like there
[13:42] *** iain has quit IRC (Remote host closed the connection)
[13:44] <brycec> I have no idea
[13:44] <acf_> http://kremvax.acfsys.net/smokeping.cgi?target=Remote.voipms-dnvrco
[13:45] <m0unds> ouch
[13:45] <mercutio> that's probably the normal ntt thing acf
[13:46] <mercutio> just you're motiring ervery 5 minutes..
[13:46] <brycec> https://smokeping.cobryce.com/?target=Internet.VoipMS.denvervoipms
[13:46] <brycec> ^ Mine for denver.voip.ms
[13:46] <brycec> (And I monitor every minute )
[13:46] <acf_> 20 pings every 2min
[13:46] <mercutio> if you look at his local thing he got heaps of gaps
[13:46] <mercutio> it may be due to monitoring not completing in time?
[13:47] <mercutio> it said 5 minutes acf?
[13:47] <mercutio> well 300 seconds
[13:47] <acf_> hmm ok
[13:47] <mercutio> maybe you using a dns server outside network?
[13:50] <acf_> yeah, probably the DNS thing
[15:03] *** mhoran2 is now known as mhoran
[15:35] *** exm has joined #arpnetworks
[15:46] *** andrew32_ has quit IRC (Quit: Page closed)
[16:47] *** novae has quit IRC (Remote host closed the connection)
[16:48] *** novae has joined #arpnetworks
[18:00] *** exm has quit IRC (Ping timeout: 264 seconds)
[18:28] *** novae has quit IRC (Ping timeout: 240 seconds)
[18:31] *** novae has joined #arpnetworks
[22:11] *** thestereobus has joined #arpnetworks
[22:27] *** exm has joined #arpnetworks
[22:34] *** thestereobus has quit IRC (Quit: thestereobus)
[23:00] *** exm has quit IRC (Ping timeout: 252 seconds)