***: gizmoguy has joined #arpnetworks mnathani: If my quota limit says: 20000000
how many gigs is that? brycec: @wa 20000000B to GB BryceBot: convert 20000000 bytes to gigabytes;0.02 GB (gigabytes);20 MB (megabytes);160 Mb (megabits);0.16 Gb (gigabits);1.6×10^8 bits;19.07 MiB (mebibytes);~~ ( 0.1 ~~ 1/10 ) × mini-compact disc capacity ( 185 to 210 MB );~~ 7 × data storage capacity of a 200-character per inch 2400-foot IBM 7330 7-track magnetic tape (~~ 2×10^7 b );~~ 14 × 3.5" floppy disk capacity (high density) ( 1440 kB );information;[information];Time to transfer at T1 spe brycec: Apparently 20MB?
mnathani: or just quota -s
(assuming GNU quota)
But yeah, quota uses bytes normally mnathani: I think its in blocks
approx 10 gigs
@wa 20000000 blocks to gb BryceBot: convert 20000000 block sizes for a typically formatted magnetic disk to gigabytes;10.24 GB (gigabytes);0.01024 TB (terabytes);1.024×10^10 bytes;81.92 Gb (gigabits);0.08192 Tb (terabits);8.192×10^10 bits;~~ 0.41 × single-layer Blu-ray capacity (~~ 25 GB );~~ 0.51 × AIT-E turbo native capacity ( 20 GB );~~ 1.2 × dual-layer DVD capacity (~~ 8.5 GB );information;[information] mnathani: this was the quota command on ARP backup system
Quota utilities version 3.17.
@wa 10 GB in blocks BryceBot: Sorry, I couldn't reach the backend API. mnathani: @iata yyz BryceBot: YYZ: Toronto Pearson International Airport located in Mississauga, Ontario, Canada mnathani: @iataabc
@iat aabc
@iata abc BryceBot: ABC: Not a valid IATA. mnathani: @iata hnl BryceBot: HNL: Honolulu International Airport located in Honolulu, Hawaii, United States mnathani: @iata bom BryceBot: BOM: Chatrapati Shivaji International Airport located in Mumbai, India brycec: blocks are somewhat arbitrary though
and usually abstracted beneath the FS mnathani: can anyone ping M.ROOT-SERVERS.NET [2001:dc3::35] (works from ARP, but not from my HE.net tunnel)
@smokeping BryceBot: https://smokeping.cobryce.com/ brycec: works over my HE tunnel
And no I'm not hitting the root servers (yet) from my smokeping mnathani: thats wierd, I can ping it from the linux box that has the tunnel setup brycec: mnathani: on ARP's backup system, "limit" is bytes (you have 20GB) and "blocks" are also bytes" mnathani: I should only have 10 GB though
thats what the portal says brycec: Well I have 20000000 and the portal says 20GB.
What's more, du says 8.5GB used, and quota says 8870132 blocks used.
so aside from a base-2/10 discrepency
, it's all kosher
at least for me mnathani: My du = 7.6G .
portal says 10GB
and quota is 20000000
probably a typo on up_the_irons part when provisioning brycec: could be mnathani: I can ping 2001:4860:4860::8888 just fine
from my Windows 8.1 box that is routed via Linux to HE.net
but 2001:dc3::35
not able to ping -: brycec blindly blames Windows mnathani: is there a different protocol
between the windows ping
and linux ping
not just my windows 8
I also have a centos behind the original linux box with the same behavior brycec: Doesn't one still use ICMP ping? gizmoguy: mnathani: what does a traceroute or mtr look like? brycec: vs UDP ping? gizmoguy: I'm thinking it's either a routing problem or an MTU problem mnathani: http://pastebin.com/YiirLqmv
why would it work from the linux he.net tunnel box gizmoguy: what does a traceroute from the linux he.net tunnel box look like out of interest? mnathani: to m-root? gizmoguy: yeah mnathani: its in the same pastebin gizmoguy: just to compare mnathani: at the bottom gizmoguy: lol
sorry mnathani: np gizmoguy: I didn't scroll my firefox window mnathani: I put a bunch of blank lines gizmoguy: yup
well that's strange mnathani: I would expect not to ping any ipv6
or all pingable ipv6
rather than ping most and not all gizmoguy: if you want to replicate windows's traceroute on linux
try "sudo traceroute -I M.ROOT-SERVERS.NET" mnathani: I have another linux box behind
there is no traceroute command on windows
oh ok
I see gizmoguy: though I was pretty sure windows tracert used UDP
I wonder if it's an issue with source IP? mnathani: how come other destinations like google and facebook come back gizmoguy: are all your machines on the same /64?
true. mnathani: they are
how do you source a ping from a different interface /ip on linux brycec: -I $IP, as I recall mnathani: confirmed
its the source ip
http://pastebin.com/CcV0dxy7 gizmoguy: woo 10 points to gizmoguy mnathani: ping to google from he.net linux box sourced from /64 within routed /48
works, however same source to m-root fails
could there be a bogon
or source address filtering going on? gizmoguy: i'll try ping you from here
ping6 2001:470:b148::1
PING 2001:470:b148::1(2001:470:b148::1) 56 data bytes
From 2001:470:0:1b4::2 icmp_seq=1 Time exceeded: Hop limit mnathani: From 2001:470:0:1b4::1 icmp_seq=1 Time exceeded: Hop limit gizmoguy: yeah, i think you have some issues mnathani: lol, I get the same thing
never thought of pinging from ARP before gizmoguy: http://pastebin.com/9FqhpmyN
so HE have a routing loop for your /64 mnathani: I could pick a different /64 gizmoguy: oddly enough, this is the second routing loop in the past 10 mins i've helped solve mnathani: 2001:470:b148:0:1::1
thats in a different /64
?
my routed /48" 2001:470:b148::/48 gizmoguy: same issue
it's probably an issue with the /48 mnathani: yea
I have a routed 64
as well
which I tried using, and that seems to work just fine
2001:470:1d:76e::1
2001:470:1d:76e::25
thanks gizmoguy :-) gizmoguy: np :) brycec: Fuck this is good 03.30 pie. Nothing like celebratory pie after fixing the "oh shit everything went to hell Thursday at midnight and I've been working ever since to get things back online" fiasco
I've had some of this pie the other day... but *now* it tastes 10x better gizmoguy: mmm pie -: brycec should've gotten some milk ***: novae has quit IRC (Ping timeout: 252 seconds)
novae has joined #arpnetworks
novae has quit IRC (Changing host)
novae has joined #arpnetworks m0unds: late to the party, but windows traceroute = tracert, also you can get winmtr which is..well, you guessed it, mtr for windows. staticsafe: winmtr doesn't do v6 :( m0unds: hasn't been updated in years either
my rust server has been invaded by korean, chinese and russian people staticsafe: :o m0unds: i set up a log notifier that sends server connect/disconnect and chat stuff to an irc channel
every night, between 0000-1000, it's nothing but russian, chinese and korean chat phlux: We're gowngrading to 1.5Mbps internet out of spite for the company we're currently with (only one other provider)
rip Netflix streaming m0unds: could just stream super ugly video at 500kbit/sec phlux: I'll probably just move from streaming to DVDs/blurays and rip them as soon as they get here m0unds: ah, dlna streaming server or something?
mkv -> tv? phlux: yeah, via ps3 m0unds: right on phlux: tbh, I'm fine with 1.5Mbps until there's a larger market out here as long as I can still get on IRC :P m0unds: are you in a rural area? or is your choice of providers just really shitty vs super shitty? phlux: I'm in a new development on a military installation
There are only two companies that offer service out here so far. AT&T offers U-Verse one street away, so I'm hoping that comes our way soon
For now, we've been using this company called "CMA Access" m0unds: ah phlux: They are expensive, and their staff won't believe me when I say problems are on their end
So I'm done with them m0unds: gross phlux: I got into it with a customer service rep and decided "Fuck it"
Sooooooooo out of spite, we're moving to AT&T DSL until U-Verse is offered again m0unds: hah phlux: Thankfully I already have DirecTV for my television needs
but I'm fairly certain I can live with it..my wife's parents live out in the sticks, and I'm pretty sure all they get is 1.5Mbps and it seems to work fine
OH, and CMA has a bandwidth limit where AT&T does not
so that's another plus
Of course, I won't be able to use too much bw only getting 150kb/s down :P m0unds: hah
at least you're not stuck with satellite stuff phlux: Very true
tbh, as long as I can check e-mail, reddit, and IRC without having to restart my modem 500000 times a day, I'll be happier overall.
Plus I have a son due in April, and AT&T's price is only $24/mo m0unds: wow, that's cheaper than a pots line here w/no additional svcs
also, congrats
april birthdays ftw phlux: thanks man ***: hive-mind has quit IRC (Ping timeout: 264 seconds)
jcv has joined #arpnetworks
novae has quit IRC (Ping timeout: 252 seconds)
novae has joined #arpnetworks
hive-mind has joined #arpnetworks m0unds_: network just take a dump for anyone else?
seeing ~75-80% pkt loss ***: RandalSchwartz has quit IRC (Ping timeout: 272 seconds)
acf_ has quit IRC (Ping timeout: 264 seconds)
qbit has quit IRC (Ping timeout: 264 seconds)
tabthorpe has quit IRC (Ping timeout: 240 seconds)
d^_^b has quit IRC (Ping timeout: 264 seconds)
tabthorpe has joined #arpnetworks
tabthorpe has quit IRC (Changing host)
tabthorpe has joined #arpnetworks
KDE_Perry has quit IRC (Ping timeout: 265 seconds)
KDE_Perry has joined #arpnetworks
tabthorpe has quit IRC (Read error: Connection reset by peer)
tooth has quit IRC (Ping timeout: 240 seconds)
tooth has joined #arpnetworks
notion has quit IRC (Read error: Connection reset by peer)
medum has quit IRC (Ping timeout: 252 seconds)
medum has joined #arpnetworks
CaZe has quit IRC (Ping timeout: 245 seconds) mnathani: over v4 or v6? ***: d^_^b has joined #arpnetworks
d^_^b has quit IRC (Changing host)
d^_^b has joined #arpnetworks mnathani: and which transit / peer are you coming in on? m0unds_: any
ntt from one location, mzima from one, nlayer from one ***: CaZe has joined #arpnetworks m0unds_: 50% on nlayer, 80+% on ntt via v4 SpaceDump: Mmm, seems to be a bit crappy at the moment.
~95% packet loss at the moment. (v4)
Oh, 97% now. :D ***: pjs has quit IRC (Ping timeout: 265 seconds) SpaceDump: Oh well oh hell. It will sort it self out soon. m0unds_: yep
lol, oh man
i need to put my v6 tunnel offline
it's coming up and dropping over and over ***: raptelan has quit IRC (Remote host closed the connection)
thestereobus has joined #arpnetworks
raptelan has joined #arpnetworks
twobithacker has quit IRC (Quit: ZNC - http://znc.in)
mhoran2 has quit IRC (Ping timeout: 244 seconds)
CaZe has quit IRC (Ping timeout: 240 seconds)
d^_^b has quit IRC (Ping timeout: 240 seconds)
eryc has quit IRC (Ping timeout: 244 seconds)
CaZe has joined #arpnetworks
pcn has quit IRC (Ping timeout: 252 seconds)
mnathani has quit IRC (Ping timeout: 264 seconds)
eryc has joined #arpnetworks
jm|laptop has joined #arpnetworks jm|laptop: hello :) staticsafe: hi jm|laptop: are there known issues? thestereobus: I'm seeing dropped packets ***: qbit has joined #arpnetworks jm|laptop: 42 packets transmitted, 2 received, 95% packet loss, time 49886ms
rtt min/avg/max/mdev = 252.426/253.863/255.300/1.437 ms staticsafe: yah jm|laptop: 70.5.250.129.in-addr.arpa. 14400 IN PTR ae-2.r04.lsanca03.us.bb.gin.ntt.net.
stops here ***: koan has quit IRC (Ping timeout: 252 seconds)
koan has joined #arpnetworks
koan has quit IRC (Changing host)
koan has joined #arpnetworks
hive-mind has quit IRC (Ping timeout: 264 seconds)
thestereobus has quit IRC (Quit: thestereobus)
Surface_RT has joined #arpnetworks Surface_RT: hi! ***: milki has quit IRC (Ping timeout: 264 seconds) staticsafe: shit is flapping m0unds_: ja ***: mike-burns has quit IRC (Ping timeout: 252 seconds)
mhoran2 has joined #arpnetworks
ChanServ sets mode: +o mhoran2
anis is now known as anisfarhana
anisfarhana has quit IRC (Changing host)
anisfarhana has joined #arpnetworks
thestereobus has joined #arpnetworks Surface_RT: figured
hi static anisfarhana: o.O , any maintenance is on going? ***: andrew32_ has joined #arpnetworks andrew32_: hey Surface_RT: not just you
hi andrew32_: my vps is down
:)
is something going on? :) anisfarhana: Ah what a relief. So its not only me then :D Surface_RT: suddenly, people andrew32_: :) anisfarhana: One of tech staff accidentally pulled out the cable maybe -: anisfarhana ducks m0unds_: arpnetworks employs ducks? andrew32_: hope they can quickly plug it in again ;) SpaceDump: It's most likely just a ddos.
Will sort itself out sooner or later. andrew32_: aaah m0unds_: s7 has had issues, could be hw or something wigging out again andrew32_: one of those nasty ntp reflection ones ...
:P anisfarhana: No more fancy uptime :( SpaceDump: ?
The uptime shouldn't be affected. :p anisfarhana: If they have hardware failure probably uptime will be affected. SpaceDump: Yeah. that's true.
But it's unlikely. Surface_RT: ducks don't require: healthcare, pension, benefits
very cost efficient and great ROI for shareholders SpaceDump: Since the pl starts 2 hops before my instance... Surface_RT: my brain internally read that 'since the polish... perl... giving up starts 2 hops before' m0unds_: oh look. rain. ***: iain has joined #arpnetworks iain: anyone around staticsafe: yep iain: everyone else down? m0unds_: yep SpaceDump: Well, lot's of people around, no staff though as it seems. iain: ugh ***: hive-mind has joined #arpnetworks m0unds_: v6 inbound is good via nlayer (S3, not S7) ***: d^_^b has joined #arpnetworks
d^_^b has quit IRC (Changing host)
d^_^b has joined #arpnetworks m0unds_: i'd bet something happened with s7 again, since it's been screwy for a couple weeks SpaceDump: Good, then my ipv6 anycast dns node should be working at least .:D
m0unds_: I have no idea what's s7 is. But imho it looks more like a ddos then anything else. m0unds_: s7 is a switch chassis that has been crashing regularly SpaceDump: 9. ae-2.r04.lsanca03.us.bb.gin.ntt.net 0.0% 709 151.0 151.2 149.8 176.9 2.3
10. ge-0-7-0-24.r04.lsanca03.us.ce.gin.ntt.net 87.0% 709 220.3 217.6 212.4 306.7 14.0
11. 208.79.88.129 98.4% 709 212.9 212.9 211.6 217.4 1.7 m0unds_: ntt feeds into s7 SpaceDump: Works like a charm until hop 10 there. m0unds_: afaik, anyway since when it dies, any2ix and ntt both die Surface_RT: so we should take s7 out back and release it like a badly trained pokemon ***: rpaulo has joined #arpnetworks rpaulo: hi
I'm having some connection issues to arpnetworks. Surface_RT: evrery one is m0unds_: yeah, there's some sort of network event going on rpaulo: ok, thanks for letting me know. ***: thestereobus has quit IRC (Quit: thestereobus)
wallshot has joined #arpnetworks rpaulo: fixed? ***: tabthorpe has joined #arpnetworks
tabthorpe has quit IRC (Changing host)
tabthorpe has joined #arpnetworks m0unds_: looks good wallshot: oooh i can get to my site again ***: twobithacker has joined #arpnetworks
notion_ has joined #arpnetworks SpaceDump: Yay, my anycast node works again. :D ***: pcn has joined #arpnetworks
milki has joined #arpnetworks
acf_ has joined #arpnetworks mercutio: what happedned this morining? Surface_RT: idk mercutio: oh so s7 may have crashed ant: 21:09:36 <arpnetworks> [2e] We're investigating very high traffic (likely DoS) to host kvr24, also consuming resources in other m0unds_: actually, it says possible ddos ant: areas Surface_RT: that isnt an irc nick ant: that's what twitter said Surface_RT: wait, is that a twitter to irc gateway ant: i pasted that from bitlbee mercutio: ahh m0unds_: yeah, so maybe the ios upgrade did fix s7 if it didn't crash under ddos, haha mercutio: oh well at least it being looked into
i'm sure we'll hear what happened m0unds_: yup
yeah mercutio: and prob better than s7 crash :) m0unds_: yeah ***: thestereobus has joined #arpnetworks up_the_irons: m0unds_: lol re s7 m0unds_: up_the_irons: did it survive the onslaught? ***: Surface_RT has quit IRC (Quit: leaving) up_the_irons: m0unds_: it did
m0unds_: although it has survived others, even on older IOS.
wow, all BGP sessions on BIRD remained up. That's quite cool.
maybe it wasn't a high pps attack
just high traffic mercutio: i imagine any2ix has less ddos traffic volume? m0unds: there we go up_the_irons: mercutio: it was pretty big though, coming through peers as well ***: thestereobus has quit IRC (Ping timeout: 264 seconds)
RandalSchwartz has joined #arpnetworks
RandalSchwartz has quit IRC (Changing host)
RandalSchwartz has joined #arpnetworks
pjs has joined #arpnetworks
pjs is now known as Guest51353 mercutio: ahh weird
i suppsoe it not that strange
these things are really distributed these days
i just kind of assume ddos traffic comes from somefar "far away"
s/somefar/somewhere/ BryceBot: <mercutio> i just kind of assume ddos traffic comes from somewhere "far away" ***: mnathani has joined #arpnetworks mercutio: ho hmm, smokeping is pretty bad up_the_irons: it comes from "everwhere" these days mercutio: i thought that was just spoofed ip's mostly
and that it was usually still few hosts
but yeah now with ntp reflection attacks..
any idea what kind of ddos it was? m0unds: i read something that 75% of the ntpds implicated in the attack on cloudflare have either been filtered or reconfigured mercutio: i bet more of them have been filtered than reconfigured m0unds: that'd be my guess too
but at least the network ops are paying attention
haha mercutio: i think there's not much choice m0unds: well, there's two choices - address it or don't mercutio: actually.. ***: robonerd- has joined #arpnetworks mercutio: i think most network op's care about painful ddos/packet loss etc ***: robonerd- has quit IRC (Client Quit) mercutio: yeh true, i wonder if ntt, level3, etc are doing anything about it ***: robonerd has quit IRC (Ping timeout: 264 seconds) mercutio: i imagine most small-to-medium providers are
but large providers may be like, we can charge more for bandwidth.. ***: rpaulo has quit IRC (Quit: rpaulo) mercutio: and it's not necesarily tier1's responsibility to block ***: robonerd has joined #arpnetworks
robonerd has quit IRC (Changing host)
robonerd has joined #arpnetworks mercutio: it gets into iffy territory -: mercutio hasn't done any blocking personally yet ***: robonerd has quit IRC (Read error: Connection reset by peer)
robonerd has joined #arpnetworks
robonerd has quit IRC (Changing host)
robonerd has joined #arpnetworks
mike-burns has joined #arpnetworks
ChanServ sets mode: +o mike-burns brycec: Yup, that was quite a chunk of disruption earlier. <obligatory smokeping graph: https://smokeping.cobryce.com/images/ARP/ARPWebsite_mini.png> m0unds: ugly mercutio: that looks milkd to me? m0unds: i saw a ton of extra flow sessions open up w/my srx trying to reestablish the tunnel, til i took it down mercutio: http://postimg.org/image/yd2ezgmhn/ m0unds: pretty funny
roughly....8 times as many as normal mercutio: maybe i was just unlucky m0unds: my route to one IP is via SJC, and the other is straight to LAX - both nlayer. the LAX one was ~85% max pkt loss, the SJC one was 75.50% mercutio: it looks like i was 95 to 100% packet loss
via ntt m0unds: that's what i saw via ntt too (work uses centurylink -> ntt) mercutio: so ntt went out harder m0unds: yeah, seems that way at least mercutio: joy
does ntt have blackhole community? m0unds: there was so much loss via ntt that i thought my vps had crashed or something at first because i wasn't getting any icmp replies whatsoever
it took almost 2 mins before i started seeing 1 or 2 here and there mercutio: looks like it has standard 666
By default, peers are not configured for the blackhole functionality. Please contact the NTT NOC @ noc@us.ntt.net for this feature. up_the_irons: mercutio: m0unds : i say filtered too. We filtered like 150 hosts. there's NO WAY one can just "wait" for the host admin to fix it m0unds: yeah, for sure mercutio: up_the_irons: lots of people are businesses etc
that may be able to fix themselves or filter
err lots of networks are businesses
still probably quicker to filter first m0unds: yup, as a reactive measure
and i guess preventative if there are systems that hadn't participated yet up_the_irons: yeah but think about VMs here. lots may be businesses, but also lots are just home / hobby stuff mercutio: i imagine most participated m0unds: yep mercutio: up_the_irons: but you're a provider to others
i mean if someone had their own network up_the_irons: yeah mercutio: ntp being a problem all over the place up_the_irons: right mercutio: if you have no control of course you're going to filter.
if you do have control you might or might not filter
upgrading ntp everywhere or fixing configs everywhere could be consdiered time-consuming
i wonder how big that ddos was
i've been hearing from a few various people getting hit by 10 gigabit+ ddos attacks recently
for small providers.
with less than 10 gigabit transit
it used to be considered reasonable safe for ddos attacks if you had 10 gigabit pipes or bigger
but it's probably shifting to 40 gigabit for safety or something now, which gets crazy if your average traffic is < 1 gigabit -: brycec switches to 100mbps "If you can't take care of this Internet, you don't deserve to have good Internet." P m0unds: hahah
all your smokeping icmp traffic would clog the tuubs mercutio: 100 people with 100 mergabit ddos is still 10 gigabit :/ m0unds: save some internet for the rest of us
jeez mercutio: i can't type today
brycec: do you have any2ix/any2ix host?
on your smokeping
i'm curious what packet loss was like there ***: iain has quit IRC (Remote host closed the connection) brycec: I have no idea acf_: http://kremvax.acfsys.net/smokeping.cgi?target=Remote.voipms-dnvrco m0unds: ouch mercutio: that's probably the normal ntt thing acf
just you're motiring ervery 5 minutes.. brycec: https://smokeping.cobryce.com/?target=Internet.VoipMS.denvervoipms
^ Mine for denver.voip.ms
(And I monitor every minute ) acf_: 20 pings every 2min mercutio: if you look at his local thing he got heaps of gaps
it may be due to monitoring not completing in time?
it said 5 minutes acf?
well 300 seconds acf_: hmm ok mercutio: maybe you using a dns server outside network? acf_: yeah, probably the DNS thing ***: mhoran2 is now known as mhoran
exm has joined #arpnetworks
andrew32_ has quit IRC (Quit: Page closed)
novae has quit IRC (Remote host closed the connection)
novae has joined #arpnetworks
exm has quit IRC (Ping timeout: 264 seconds)
novae has quit IRC (Ping timeout: 240 seconds)
novae has joined #arpnetworks
thestereobus has joined #arpnetworks
exm has joined #arpnetworks
thestereobus has quit IRC (Quit: thestereobus)
exm has quit IRC (Ping timeout: 252 seconds)