#arpnetworks 2015-03-20,Fri

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***hive-mind has quit IRC (Ping timeout: 250 seconds) [07:24]
.... (idle for 19mn)
mkbno she said they were running up until a month ago
not that I expect her to really know anything
[07:43]
.... (idle for 15mn)
***awyeah has quit IRC (Ping timeout: 252 seconds) [07:58]
m0unds_packet loss again [07:58]
***milki has quit IRC (Ping timeout: 252 seconds)
jpalmer has quit IRC (Ping timeout: 256 seconds)
KDE_Perry has quit IRC (Ping timeout: 252 seconds)
[07:59]
m0unds_yeah, not just me i guess
haha
[08:00]
***RandalSchwartz has quit IRC (Ping timeout: 265 seconds)
KDE_Perry has joined #arpnetworks
mkb has quit IRC (Ping timeout: 265 seconds)
awyeah has joined #arpnetworks
[08:01]
milki has joined #arpnetworks [08:06]
mkb has joined #arpnetworks [08:12]
hive-mind has joined #arpnetworks [08:19]
.... (idle for 16mn)
mnathani_ has quit IRC (Ping timeout: 264 seconds)
mnathani_ has joined #arpnetworks
[08:35]
................. (idle for 1h20mn)
jpalmer has joined #arpnetworks [09:56]
......................................... (idle for 3h20mn)
RandalSchwartz has joined #arpnetworks
RandalSchwartz has quit IRC (Changing host)
RandalSchwartz has joined #arpnetworks
[13:16]
.... (idle for 15mn)
mercutiothat was lot pf people that left at once 5 hours ago
or so
i wonder if they were all using ipv6
[13:35]
m0undsi was getting awful packet loss on both v6 and v4
i wonder if it's that same annoying thing that was happening last week
[13:44]
mercutiowell my graphs are fine :/
but my route goes over any2ix
[13:47]
m0undsmine is via level3 [13:48]
mercutioi've been dreaming about making a ping thingy
the plot is starting to come together
finding good hosts to monitor is the complex part
nothing's sticking out as terrible on hte monitoring that's on arp's network
i think my test hosts don't really test level3 well though
[13:48]
m0undsthis packet loss stuff seems to start at approximately the same time each day it occurs [13:52]
mercutioat like 6 hours ago? [13:53]
m0undsyea, around there [13:53]
mercutioi updated my smokeping with my newer targets list at least. [13:53]
m0undshappens between 0830-0900 MDT [13:53]
mercutioi'm never around then :) [13:53]
m0undsyeah [13:54]
mercutiodo you have any mtr's or anything? [13:54]
m0undsi can't MTR the arp box because of how severe the packet loss is [13:54]
mercutiodoes it get unusable? [13:54]
m0undsyep
i can't stay connected via ssh at all
[13:54]
mercutiocan you setup a cron on the arp box to mtr out?
does packet loss start before it hits arps network?
[13:54]
m0undsno
aside from the naturally occurring deprioritization stuff
[13:55]
mercutioand it's lossy to arp's web site or other arp hosts at the same time as your vm? [13:55]
m0undshaven't tested it [13:55]
mercutioyeah i hate lboody deprioritisation :) [13:55]
m0undsi guess i will next time this happens [13:55]
mercutiowhat kvr are you on? [13:56]
m0unds03 [13:56]
mercutiooh
yeah :)
well i'm monitoring that host
and i've only seen loss a couple of hours ago
[13:56]
m0undsmy latency to the vm is jittery as all hell
+/- 12ms
[13:57]
mercutioeven now? [13:57]
m0undsyep [13:57]
mercutiodoes it get unusable?i'm seeing 0.1 msec jitter or something
oops
i must have pressed up arrow a bit
yeh no jitter here
well a mtr now could be useful.
i usually use mtr -i=0.1 when i'm testing things :/
which shows up deprioritisation even more
err mtr -i0.1
but shows jitter easier
r1 drops packets with that frequency
[13:58]
m0undsihttp://pastebin.com/F7SJ0GjA [14:02]
mercutiowow
oh you traced to kvr03 :)
[14:03]
m0undsyeah
tracing to te vps now
it's largely the same
[14:04]
mercutiothat was with -i1 ?
it could be a comcast issue
[14:04]
m0undsnegative
it's not
[14:05]
mercutiooh [14:05]
m0undsno packet loss to any other destination i've tested [14:05]
mercutiother's jitter to 96.120.1.73 [14:06]
m0undsbuddy in FL was using the voice server and started noticing packet loss and let me know [14:06]
mercutiobut i mean between arp/comcast. [14:06]
m0undsyes, because this is a residential cable connection [14:06]
BryceBotThat's what she said!! [14:06]
m0undsBryceBot: no [14:06]
BryceBotOh, okay... I'm sorry. 'yes, because this is a residential cable connection' [14:06]
mercutiobut there' snot jitter to the hop prior of 689.86.182.122
but if 96.120.1.73 is a bras, that could be deprioritisation
[14:06]
m0undsit's the headend
so yes
[14:07]
mercutioyeah
mind sharing your ip? :)
[14:07]
m0unds73.26.90.138 [14:07]
mercutioit's worse to your ip
i'm still seeing no loss though
and it's just as bad from another provider
[14:08]
m0undsmy latency to a box w/another provider is 30.1ms +/- .2ms [14:09]
mercutiooh but it's fine from new zealand
well 3 msec jitter
or just under
[14:09]
BryceBotThat's what she said!! [14:09]
m0undsBryceBot: no [14:09]
BryceBotOh, okay... I'm sorry. 'or just under' [14:09]
mercutiocan you try tracing to 202.49.71.24 ?
that's my nz test server
[14:09]
m0undsi have less jitter to that destination than to arp [14:10]
mercutioit's gtt->comcast for forward route [14:10]
m0unds+/- 1.8ms [14:10]
mercutiobut i suspect it's forward path from youu that's the issue
yeah i know right
i imagine it's not going comcast->level3
[14:10]
m0undsbut that doesn't explain why a bunch of other peoples' connections took a dump around 30 seconds after i started seeing packet loss [14:11]
mercutiothey may all be on comcast? [14:11]
m0undscongestion could explain the jitter, but i couldn't care less about jitter
no, their sessions to freenode dumped
[14:11]
mercutiobut yeah level3 may be having issues
oh hmm
[14:12]
m0undson arp V6 IPs (i saw at least one with no rdns that was arp's prefix) [14:12]
mercutioit's not even peak time though
true
the mind boggles :)
[14:12]
m0undsthat's why i'm wondering if someone's machine is doing something at that time
because it happens right around that time each time
[14:12]
BryceBotThat's what she said!! [14:12]
m0undsBryceBot: no [14:13]
BryceBotOh, okay... I'm sorry. 'because it happens right around that time each time' [14:13]
mercutioyeah [14:13]
m0undslike maybe saturating the link to upload or download something [14:13]
mercutioi don't think kvr03 is having issues
and i don't think it's likely you're all on kvr03
and i don't think r1 is having issues
but s7 coudl be having issues
it doesn't really explain ipv6 screwing up thouguh
[14:13]
m0undsnope [14:14]
mercutioactually it does
ntt is on s7
if it's a general issue
[14:15]
m0undsso, having looked at the users w/out cloaks who dropped when i mentioned packet loss, they were all connected via some arp svc [14:15]
mercutioyeah
incoming skips s7
for ipv4 from r1
and my r1 based routes have been fine
but outgoing doesn't skip it
[14:15]
m0undshey, milki jpalmer RandalSchwartz awyeah - are you guys connecting to IRC via an ARP VPS or dedi? if you're on a VPS, mind sharing which host? [14:17]
mercutiobut you wouldn't think someone would ddos at the asme time every day [14:17]
m0undsright, that's why i'm thinking it's something saturating a link
or similar
[14:17]
mercutioand downloading at high speed over transit enough to saturate links isn't normal [14:18]
m0undsoffsite backup, something like that [14:18]
mercutioyeah it could be that they're pushing backups onto arp [14:18]
m0undsif they all happen to be on kvr03, that'd be great
haha
[14:18]
mercutioon a dedicated server
i really don't think kvr03 is at fault
but i'm happy to be wrong
oh hmm
kvr03 still goes throuugh s1
i was wondering if hitting a host directly would bypass s1
[14:18]
m0undsoutbound route from the vm on kvr03 is via s7 [14:20]
mercutioand that may happen on peering but not transit [14:21]
***hoggworm has joined #arpnetworks [14:21]
mercutioinbound route is via s7 too i think you'll find
it just responds with the link address to level3 probably
but it'll be the same host
it's only r1 that's assymetric
because s1 doesn't have full route table
[14:21]
m0undsoh, were you meaning tracing the host kvr03.arpnetworks.com indicates s1? [14:21]
mercutiowell i was meaning kvr03 may have more direct path than your vm
but nah it's the same path ish
[14:22]
m0undsyeah, looks the same
it's pretty symmetrical. much better than before level3 was turned on
[14:22]
mercutioyeah level3 isn't bad [14:23]
BryceBotThat's what she said!! [14:23]
m0undsused to be inbound from nlayer/mzima/whatever and outbound via ntt [14:23]
mercutionlayer would have been fine probably [14:23]
m0undsand it was unusable during "peak" time for me
yeah, it was NTT that sucked
way way way oversubbed at comcast
[14:23]
mercutionlayer's been having weird shit happen recently
nlayer/mzima/tinet/gtt
i can't tell which is which haha
[14:23]
m0undsyeah, hahah [14:24]
mercutioas4436 is having issues
which is that
[14:24]
m0undsmzimtinettttttt [14:24]
mercutioit says nlayer :/ [14:24]
m0undsyeah, some stuff still does [14:24]
mercutiohttp://www.peeringdb.com/view.php?asn=4436
yeah so that's the legacy nlayer
[14:24]
m0undsthe net formerly known as nlayer [14:25]
mercutiowhat happened to mzima haha
i wonder how many routes comcast has
245
including ipv6
there's some non aggregated stuff in there too
i imagine they have more asn's, that's just AS7922
it kind of sucks that comcast are huge and don't peer :/
[14:25]
m0undsthey started splitting up their network a bit more
based on market and stuff, tons of ASNs
[14:34]
mercutioahh
probably a good thing
[14:34]
m0undsyeah [14:35]
mercutiomkb isn't on comcast is he [14:35]
BryceBotThat's what she said!! [14:35]
mercutioas you said it doesn't seem like it's just a comcast issue
but prioritising making sure users connections well seems sensible
freenode on the other hand..
dropping on freenode could be anything
that said, i've been stable for a whil again
[14:35]
does this packet loss happen at toher times than 7:30 to 8:00 MDT? [14:44]
m0undsit has before, yes
it happened a ton last week (i think it was last week)
[14:44]
mercutiobut atm i mean
yeh i remember that
[14:44]
m0undswell, it hasn't happened since last week [14:44]
mercutioahh [14:45]
m0undstoday's the first day. the time when it occurred in the past started at 0830-0900 MDT and then would occur again throughout the day [14:45]
mercutiothat gave me loss though
this one didn't give me loss
also kvr03's reoccuring issues in the past got dealt with
the specific to kvr03 issues
which was fixed about 10 days ago
but there was the issue efecting everyone around then too
and the effecting everyone was never determined the cause of i think
[14:45]
m0undsborrowing smokeping data from acf_ - http://kremvax.acfsys.net/smokeping.cgi?epoch_start=1426863600;hierarchy=;epoch_end=1426888048;target=Remote.arpnet-lsanca;displaymode=n;start=2015-03-20%2008%3A00;end=now;Generate%21=Generate%21
it's on there
[14:49]
mercutioahh i see [14:50]
m0unds0750 PDT 0850 MDT [14:50]
mercutioit's level3 outbound
to ec2
i wonder if it's level3 having issues
[14:50]
m0undsi think this is local though
microsecond ping time
on that graph, avg is .688ms
[14:51]
mercutiooh
nah his smokeping is weird
the top is to arpnetworks.com
but the last 30 hours is to ec2.acfsys.net
[14:51]
m0undsyeah, from the local box
second is from ec2
[14:52]
mercutioit's the second that's having issues thoguh [14:52]
m0undsthe graph i linked, top part, showed 30+% packet loss to arpnetworks.com from within the network
for a period of 10 mins
[14:52]
mercutiooh the link didn't work [14:53]
m0undsoh [14:53]
mercutioi dununo why [14:53]
m0undsweird [14:53]
mercutioit wa so long, and it didn't make sense why [14:53]
m0undsok, well set the time today (03/20/2015) and 0700 as the starting time
to now
http://i.imgur.com/Z5WZ0y4.png as an alternative
[14:53]
mercutiooh that's better
mine was set to 08:00
[14:54]
m0undsyeah [14:54]
mercutioand it was ll grreen :/ [14:54]
m0undsthat weird little blip is there, even on arp>arp stuff [14:54]
mercutioyou're so right
http://la.meh.net.nz:24/cgi-bin/smokeping.cgi?target=meh.arpmeh
that's to arp vm from vm on arp dedicated
hmm
maybe it is kvr03 issue?
i can't rmeember what host acf is on
[14:54]
m0undshttp://i.imgur.com/jFD4Yic.png
wishing i still had my server at home so i could have additional data
but i shut down everything here but an rpi
haha
a rpi*
[14:57]
mercutioacf ios on kvr27
looking at logs
he may have more than one host thouugh
[14:59]
m0undsi saw something about kvr10 - i was on 27 before (had a project that ended) [15:00]
mercutiooh
i saw discussion about 10 too
but he said he had high latencies to 27
but that may have been you :)
[15:01]
m0undsyeah, we were trying to figure out the ntt weirdness
around then
[15:01]
mercutioi should just monitor them all :/ [15:01]
..... (idle for 22mn)
brycecThanks for the reminder m0unds, I keep meaning to setup smokeping on my rpi
(but it still won't be today)
[15:23]
BryceBotThat's what she said!! [15:23]
mercutioi need to set it up on my san jose vultr host
which seeems to be ntt
[15:29]
m0undsif i weren't running an awful java app on mine i would [15:29]
mercutioso useful for testing transit from arp i suppose [15:29]
m0undsi need to get another one [15:29]
mercutiothe network is pretty bad on vultr [15:29]
m0undsis it? [15:29]
mercutiowell it depnds
http://sj.meh.net.nz:24/10m
vs http://la.meh.net.nz:24/10m
[15:30]
m0undsthey're both the same for me, 5MB/sec [15:30]
mercutiomaybe youu're closer [15:30]
m0undsi likely am [15:30]
mercutiovultr has random issues more often too
oh wow
from uk sj is way faster than la
ntt vs level3
[15:30]
m0undsi've only used a couple hosts long term over the last several years
arp and linode for the last 2 or so
[15:31]
mercutioi only use vultr cos they're cheap
just for smokeping and bw tests haha
and somewhre convenient to ssh to
ok this is strange
vultr is going ntt->level3
and getting better performance than arp going level3->level3
[15:32]
i wonder if something is up with level3
arp to vultr was fast.
but sj vultr shouldn't be twice the speed of arp
esp when they're both l3 to uk
(that was doing a uk test)
and vultr has the extra ntt to level3 hop
[15:40]
http://pastebin.com/HNz6uQtK
anyone want to try my curl download test?
it downloads 7 10mb files
[15:49]
***YogeeBear has joined #arpnetworks [15:50]
mercutioi probably should just stick it on site
http://weallsee.net/testspeed
[15:51]
brycecmercutio: http://sprunge.us/GdAh but 10MB is a pretty small test file for a good connection with 100mbps Internet. [16:02]
mercutiobrycec: it's good to be small [16:02]
brycec(that's from my home cable connection) [16:02]
mercutiolarge fikles you usually have in background
smaller ones you have in foreground
and it gives you a good idea if connection is going wekll
[16:03]
brycecBut for a speed test, a 10MB file on a connection that can carry 10MB/s is a poor sample size. [16:03]
mercutioi usually time the whole script
but adding up the numbers it looks pretty good
[16:03]
brycecAh [16:03]
mercutiobut la is slower than sj
why is la so slow
[16:03]
brycec(I'm also closer to SJ) [16:04]
mercutiooh
but look how fast you get to new zealand
the top one is new zealand
[16:04]
brycecah
faster than .au, impressive and surprising
[16:04]
mercutiola.meh.net.nz is a nz ip advertised on an arp vm
i hope it's not going via nz
nz is actually closer to the US than asutralia
and syd is vultr :/
[16:04]
brycecyeah but aren't most trans-pacific cables going to Sydney directly? [16:05]
mercutioi thought it was more imrpessive that it's faste than los angeles
nope
well they are
but it's a longer cable
i probably should use an arp ip
but as a quick test, la.meh.net.nz ping looks fien?
[16:05]
brycec(currently running mtr's to see)
(Looks like I get to NZ via Cogent's SJ -> Plain.net.nz direct)
(And Sydney via NTT LA / Equinix Sydney)
http://sprunge.us/GNCS if you're curious
la pings were slightly slower than san jose, and both went over NTT
[16:06]
mercutiowell it matters more what my route to you is
it's 3.18 kernel on la vs 3.19 on the others
it's gtt->comcast from nz
and it's going direct via san jose
and la is sending out via level3 to comcast.
with way worse jitter
i mean icmp deprioritisation and all that, but it's 0.3 msec jitter vs 2.1 msec jitter.
the jitter on seattle level3 is showing 0.3 msec jitter though
the jitter is on te-0-1-1-ten01.spokane.wa.seattle.comcast.net
and te-0-1-0
they both hit that hop
which alternates.
[16:13]
brycec(I should point out that Spokane, roughly wear I live, is a city 5 hours East of Seattle) [16:18]
mercutioactually they both hit be-37-sur03.spokane.wa.seattle.comcast.net
and the jitter isn't there
[16:18]
BryceBotThat's what she said!! [16:18]
brycec(So, presumably Seattle serves as our upstream, but it is aways away) [16:18]
mercutioso probably it's forward path
ahh your mtrs only have one ping
[16:18]
brycecYeah, for expediency [16:20]
mercutioit's hard to know thouugh
yeah
can you run that test again just to make sure it's not random that la is going slow?
[16:20]
brycecsure [16:23]
mercutioi mean 30 megabit isn't /that/ slow [16:23]
brycecdo you want just la's mtr? [16:24]
mercutiojust la's mtr
and la's download speed
actually just doing la a few times over for the 10mb test would be fine
it gets over 40mb/sec from vultr
so i doubt it's a general throughput issue
which is making me wonder if level3 is being weird
[16:24]
YogeeBearDCC SEND STARTKEYLOGGER 0 0 0 سمَـَّوُوُحخ ̷̴̐خ ̷̴̐خ ̷̴̐خ امارتيخ ̷̴̐خ [16:25]
***YogeeBear has left "DCC SEND STARTKEYLOGGER 0 0 0 سمَـَّوُوُحخ ̷̴̐خ ̷̴̐خ ̷̴̐خ امارتيخ ̷̴̐خ" [16:25]
mercutioif it's slightly problematic generally, it increases the chances that it's being more problematic other times [16:26]
brycecmercutio: http://sprunge.us/LRJG
(and traffic on my own net is relatively quiet)
[16:26]
mercutiowow
it's all ove rthe place.
i reckon there's packet loss
can you try the same thing with garlic.plain.net.nz:24/10m ?
i expect it'll be the same
[16:28]
brycecsure, one sec [16:33]
mnathani_anyone use policy based routing to redirect certain traffic out a different gateway? [16:33]
mercutioi've done it before mnathani_ [16:34]
brycecmercutio: http://sprunge.us/dMJd [16:34]
mercutiousing ip rule and multiple tables [16:34]
brycecI redirect specific subnets and addresses over my work vpn, yeah [16:34]
mercutioyeah same diff with garlic
it's xen, garlic is dom0
why does your router have such high pings?
just seems a bit strange :)
[16:35]
brycecbecause I don't have direct ethernet to it [16:37]
mercutiooh it's bridged? [16:37]
brycec(ethernet-coax-ethernet) [16:37]
mercutioahh [16:37]
brycecdecanet specifically
pings to other hosts on this side of the bridge are .3ms
[16:37]
mercutiomaybe we could try iperf udp test? [16:38]
hoggwormsup with yogeebear trying to dccc the file "startkeylogger" [16:38]
brycechoggworm: #frenode is already aware [16:38]
mercutioit's a pita if you can't easily forward udp [16:38]
brycechoggworm: https://nullroute.eu.org/~grawity/startkeylogger.html [16:38]
hoggwormbrycec: cheers [16:38]
brycecnp [16:39]
mercutiothat startkeylogger thing is old apparently
and it's been in other channels earlier
[16:39]
hoggwormoh
haven't seen it yet on ircnet/efnet/dalnet
[16:39]
mercutioapparnelty it's an old mirc thing [16:40]
brycecI first got a random CTCP in Arabic from them :/ [16:40]
mnathani_brycec: so destination address based routing? [16:40]
brycecmnathani_: just simple "route add..." stuff [16:40]
mercutioi used it for source based routing [16:40]
mnathani_I am trying to route my chromecast over a vpn [16:41]
brycec(and I use pf to limit sources) [16:41]
mercutioyeah that's easy mnathani_ [16:41]
mnathani_without forcing all wireless traffic over the vpn
oh and I want to be able to stop routing over the vpn with a command
[16:41]
mercutiohttp://blog.scottlowe.org/2013/05/29/a-quick-introduction-to-linux-policy-routing/
does that help you?
[16:41]
bryceccommand: "Woman, unplug this cable" :p [16:41]
mnathani_ie: no config changes or dhcp lease changes on the chromecast [16:42]
mercutioof course you'll want to set a static ip or permanent dhcp ip address on the chromecast [16:42]
mnathani_dhcp reservation is what I have now
I am attempting to do this using vlans / cisco layer 3 switch and Router if required
[16:45]
mercutioif you have a linux box
just set the chromecast to gateway to the linux box
and look at that link i linked for policy routing
basically youu can have two route tables
ip rule add from 192.168.30.200 lookup custom
so you can have a table called custom
[16:46]
mnathani_would a linux vm work, or will it work out better to have a physical box with multiple interfaces? [16:47]
mercutiovm is fine
ip route add default via 192.168.30.1 dev eth1 table custom
then you do something like that
[16:47]
mnathani_are these going to converto to iptable rules? [16:48]
mercutiohmm i did a udp test to uk nad it was fine [16:48]
acf_network namespaces in Linux are also nifty [16:49]
mercutioit's even got less jitter than vultr
so why is http going slow
[16:49]
..... (idle for 22mn)
this level3 thing just gets weirder
with multiple connections to the uk one of them goes way faster than the other
so there's some load balancing going on
so my udp test may have been fine because it was on the fine connection
so bryces huge varying degrees of speed for the 10mb file may been due to alternating load balanced connections
[17:11]
mnathani_mercutio: how can I test the curl 10 meg files [17:19]
mercutiomnathani_: http://weallsee.net/testspeed
it's a shell script
with the urls
and set -x so you know which host it's going to :)
i'm seeing jitter to comcast now too
from nz
it was fine before, so comcast's load balancing is lame too :)
it seems to be a general issue these days
i think the core issue is that people are using aggregated 10 gigabit ethernet connections with people with high speed connections close by
and that seems to be more of an issue regionally, as long distance connections are 40g+
it may even be aggregated gigabit in smaller locations.
[17:19]
mnathani_http://pastebin.com/byQUYHGx
thats from my residential 60mbit connection in Toronto
[17:31]
mercutiowow you're so slow to japan
and nz
i wonder why
on teksavvy?
[17:32]
m0undshttp://pastebin.com/fDh8A1iW [17:34]
mercutiook i find this more than a little disconcerting.
la is slower than it should be it seems
is a reoccuring theme
[17:35]
m0undshaha [17:35]
mercutioi do wonder why mnathani was so slow to japan and nz though [17:36]
m0undsyea, i'm maxing out at 2.3MB/sec from that la test
could be horribly slow writes
yea, hitting 5.4MB/sec on my desktop
i forgot i was testing on my rpi :)
[17:38]
mercutioit doesn't write
it redirects to > /dev/null
[17:41]
m0undswell, an arm6 cpu is dog-ass slow
regardless
[17:41]
mercutioyeah [17:41]
brycecmercutio: the Ethernet on the rpi is a USB connection to the host
so bus- and cpu-bound ;)
[17:41]
m0undsand that ^ [17:41]
mercutioyeh they suck [17:41]
m0undsit's fine for running crashplan
haha
[17:42]
brycecI wouldn't say they suck, they just have their place [17:42]
m0undsyep
i have one that controls my fermenter and i have one running crashplan
[17:42]
brycec(Find me another $35 computer that is comparably spec'd and has a non-usb and non-seial Ethernet controller :P) [17:43]
mercutiohttp://pastebin.com/4jcYkRjF
this is my vdsl connection
[17:43]
brycecI have one for OOB/serial access to my core network, and one for Kodi. A good gamut of uses. [17:43]
m0undsrpi terminal server sounds good [17:44]
mnathani_I can send you an mtr [17:44]
brycecm0unds: it's powered by the USB port on my wifi AP :P [17:44]
m0undsi have some old-ass cisco 2516-rj
even better, hah
oh, i have one that does aircraft transponder monitoring via sdr dongle
forgot about that one
[17:44]
brycecfun [17:45]
m0undsthe dump1090 binary is a little crashy, but i just automate restarting it [17:45]
mnathani_mercutio: http://pastebin.com/nQR6qvNu [17:47]
mercutioyeah it's any2ix both ways
does that mean you're any2ix to la.meh.net.nz too?
maybe they're overloading their any2ix port
[17:48]
mnathani_I redid the speed test http://pastebin.com/i8g3fz9x [17:49]
mercutioi'm probably going to have to upgrade japan's bandwidth :) [17:50]
mnathani_you mean japan your host
as opposed to the country right
[17:50]
mercutiooh it's fine
yes
it's vultr tokyo server
108.713 GB of 200 GB (54%) (Bandwidth preferences)
it looks like it's fine because most of it's incoming
[17:50]
mnathani_some hosts like liquidweb do free incoming [17:52]
mercutiovultr doesn't :/
but they onyl charge you whichever is highest
and outbound is way lower
on 42.468gb
it's a real pita increasing bandwidth on them
your vps shuts down for ages etc
cos you have to change plans
[17:52]
mnathani_do other things change as well
like cpu cores and memory
[17:54]
mercutioyou get some more memory and more disk [17:55]
mnathani_when you enter the next tier [17:55]
mercutioyeah
but i don't use much disk
/dev/vda1 15G 3.1G 11G 23% /
[17:55]
mnathani_do you have a slave setup for smokeping [18:01]
mercutionope [18:01]
mnathani_or are they all individual [18:01]
mercutioi just stick it on multiple hsots [18:01]
mnathani_I guess its easy enough to compare / correleate events with the web interface [18:02]
mercutioyeah
i find it easier
but i want multiple views etc
so i can choose how it shows :)
so i'm trying arp's test download
and i'm finding it seems worse for performance
it seems to speed up slower
[18:03]
mnathani_do you have 100 meg test files? [18:04]
mercutionope
i can make one
i have 200mb
testfile.zip
it's urandom not a zip file
[18:04]
mnathani_on each host? [18:04]
mercutioyeah that's everywhere
there's also 200k everywhere
i'm smokepinging with 200k
200k actually identifies issues often
but it has a huge latency bias
with 10mb the latency doesn't matter nearly as much
well at < 50 megabit
at > 200 megabit it does
they should all be on gigabit too i think
i also have it on cloudflare
http://weallsee.net/10m
but i don't think they like you doing that
cloudflare is served from that first ip
with light use it's prob ok
[18:04]
does someone else want to try hosting a 10mb file? [18:20]
brycechttps://speedtest.cobryce.com/speedtest/random2500x2500.jpg ot
*it's 12MB but close enough
there's also 3000 and 3500
oh and 4000x4000 is 40MB
*30MB
[18:26]
mercutiohmm it's slightly quicker
3598 vs 3407
so yeah probably just the larger file size
having a copy the same size woudl be better for testing comparitively i reckon
[18:27]
brycec(and I have smaller files - 1000 1500 2000 2500)
(oh and 350 500 750)
[18:28]
mercutiohttp://la.meh.net.nz/mini/speedetst/random2500x2500.jpg
i have that too :)
but not everywhere
[18:29]
brycecgood typo :p [18:29]
mercutiooh shit
s/detst/dtest/
[18:29]
BryceBot<mercutio> http://la.meh.net.nz/mini/speedtest/random2500x2500.jpg [18:30]
mercutiooh yours is https
mine is faster for me now
oh cool you can take the s off
[18:30]
brycecport 80 is there too
I didn't realize I linked to https
[18:31]
mercutioyeah i'm finding mine slightly quicker
but not by much
i should check from uk
weird i'm getting way faster speeds from the uk to mine than yours
way faster than i was getting on 10mb before
like 1.6 to 2mb/sec to yours, and 4.4 to 5.4 to mien
[18:31]
BryceBotThat's what she said!! [18:33]
brycecbrycec is also uploading to his VPS right now [18:34]
mercutioshouldn't matter?
oh it's a vps?
i thought it was dedicated from the mtr
but of cousre..
arp makes all vps's look dedicated :)
[18:34]
bryceclol [18:35]
mercutioare you on 100 megabit? [18:35]
brycecYes, I beleive so
*believe
[18:36]
BryceBotThat's what she said!! [18:36]
mercutiostill, uk and nz are about same distance away [18:37]
brycec(there we go, flash speedtest fixed speedtest.cobryce.com) [18:37]
mercutioand it waas tiny difference from nz and huge from uk
i bet mine's out of date
yip
bah it wants you to login now
http://c.speedtest.net/mini/mini.zip
[18:37]
brycecwhat? [18:38]
mercutioto download new mini.zip
my speedtest is out of date
[18:38]
brycecIt didn't ask me, but I guess I'm already logged in [18:38]
mercutiobut chrome remembered my login
well it downloads that fast
http://la.meh.net.nz:24/mini/
[18:38]
brycecfrom home, I get just better than 6mbps from yours [18:40]
mercutioweird now that was slower on mine than on yours for me
6 megabit?
i'm even getting slower upload
[18:40]
brycecI find the mini test to be fairly unreliable, at least compared to what I get via other methods [18:41]
mercutioyeah it's lame [18:41]
brycecbut it's a useful tool [18:41]
mercutioi prefer my 10mb test.
but what i really want is something that can test a whole lot of normal web sites etc.
but there's huge complexities in there
the problem with speedtest is it just tests close local servers normally
or you tell it to go to some server, adn you don't know which to go to
it doesn't really help getting 100 megabit locally if there are no web sites hosted locally
but how far do you go?
i often find ovh web sites slow
but i don't go to any of them regularly.
it's more i find a web site not loading and i trace to it
and it's ovh
and then it's back again sometime and still going slow
of course it's not necesasrily ovh's network
but like mnathani getting slow speeds to japan and new zealand may not really bother him at all
[18:41]
mnathaniI don't really need to reach nz or jp most of the time [18:46]
mercutiomnathani: exactly
i don't even need to reach japan really
but it's similar ping to california
[18:46]
mnathaniDid you see my second speedtest [18:46]
mercutioand it's another way to the US
so if there was some big outage between NZ and US I coudl bounce via there :)
oh they went u~p
i wonder why it was slow the first time.
so now sydney and la the slowest
i wonder what connection speed they have on any2ix
10 gigabit
[18:46]
maybe i should try upgrading kernel on la [19:02]
ok slightly newer kernel [19:13]
.................................. (idle for 2h46mn)
mnathani_mercutio: 10 mb it too small for the speed test
here are my results with the 200 meg test file:
[21:59]
mercutiomnathani_: why do you say taht? [21:59]
mnathani_http://pastebin.com/28uxqhES [22:00]
mercutiohmm la is still slow :( [22:00]
mnathani_connection takes a few seconds to reach max speed - by then the file download is already complete [22:00]
mercutioyeah but a lot of that's ramping up, the connection stability etc.
often for interactive stuff it's what matters
it's interesting how close nz and au are for speed
it does look like there's some issue there, but i can get fast to tehre
slower than last time
something's a little weird
it pauses around 7mb/sec for ages
but yeah for 200mb file in la from nz i get average speed of around 8 mb/seec, and current speed of aroudn 11.5mb/sec
that is faster than any of your test resutls.
[22:01]
mnathani_I have another VPS I can test on
in Toronto
[22:06]
mercutiooh yip [22:06]
mnathani_the tests from me were on residential connection [22:06]
mercutioi get 17mb/sec to sydney [22:06]
mnathani_are you talking megabits or megabytse? [22:07]
mercutiomegabytes [22:07]
mnathani_lowercase means bits
:-)
[22:07]
mercutionah Mb means megabits
MB means megabytes
mb could be either :)
it shows as M in curl fwiw
the question remains why the slower speeds
[22:07]
brycec17MB/s? Damn that's a nice fat pipe. [22:12]
mercutiouk is consistently giving 5200k/sec within 100k/sec
brycec: it's gigabit
that's testing from the top server on that list
i mean it's gigabit ethernet, not gpon or anything
[22:12]
brycecheh
brycec is currently stressing his network and NAS pulling files @ 100+MB/s, wheee USB3 flash drive
[22:13]
mercutioi can only do around 4.2megabytes/sec from home
heh i can do faster than that at home over network :/
[22:13]
brycecMy home network is only GbE :p [22:14]
mercutioyeah i ordered qdr infiniband cards [22:14]
brycec(averaging 108MB/s currently - disk IO bound) [22:14]
mercutiothey're only $30 US each [22:14]
brycecNice. [22:14]
mercutioit just means i have to get someone to resend them to me
i still don't know how much that'll cost
[22:14]
brycecheh [22:15]
mercutiobut ebay's global shipping program sucks
and the first seller wouldn't ship by usps
and the second seller doesn't ship outside the US
[22:15]
brycec(Also IB wouldn't work well for me - can't very well run another cable across the apartment) [22:16]
mercutioheh
i'm still trying to figure out how the hell i'm going to cable my room
i want to get from one side of the room to the other
runinng cables along the ceiling is guly
ugly
running on the carpet with a rug over could damage them
atm i'm using ethernet and plugging and unplugging
but i'm thinking that i'll have a few on this side
[22:16]
brycecI have a single cat6 strung over top the doorways from front to back of the apartment, it blends in more or less, but every time I see it is a reminder [22:17]
mercutioheh
well i was noticing that fibre is thinner than copper
but it's bright yellow
well the stuff i have is
[22:17]
brycec(or orange, typically) [22:18]
mercutiothis is smf
http://pastebin.com/6JyuxBEX
that's windows test to home on zfs
bloody samba is a cpu hog
[22:18]
brycecOnly a 100MB test file? Did that ever leave RAM? [22:20]
mercutioyeah
it's windows
it's disk cache sucks
if it was ram it'd be 2000+
i have 32gb of ram
it'd have to be huge to be bigger than ram
and it does seem consistent
[22:20]
bryceclol [22:21]
mercutiowell mostly
i'm using ssd's with write cache
err slc write cache
and linux will buffer some writes
but i care about network performance rather than disk performance for these tests :)
mnathani_: can you test downloading bryce's image?
there should be 5000x5000
i dunno what his cap is like heh
the strange thing about mnathani getting slower speed is that it's a any2ix route
and i'm getting fine any2ix performance
and japan and nz should be using any2ix too
and teksavvy only listened any2ix california rather than north/south
[22:21]
mnathani_http://la.meh.net.nz/mini/speedtest/random2500x2500.jpg < that file?
or you mean from his box
[22:25]
mercutioyeh but s/2500/5000/g
but yeah from his box
do both of them
see if it's similar
[22:26]
brycec(it's an ARP VPS) [22:26]
mnathani_is the url handy [22:26]
brycecspeedtest.cobryce.com/speedtest/random2500x2500.jpg [22:26]
mercutiooh there's 3500 but not 5000
oh you know what
[22:27]
brycecup to 4000 [22:27]
mnathani_are those actual images? [22:28]
mercutioif there's a transparent proxy it'll hit with bryce's link but not mine
yeah
they're just random pixels though
[22:28]
brycecmnathani_: they'll render, sure... but it's static/random [22:28]
mercutioit's better than speedtest.net's upload test of zeros
but it's still dedupable
well you can also hack it to cache too
i was playing with it ;)
but it won't give you a result if it goes faster than 1 gigabit
i was just rewriting urls, it's the same mini.zip thingy for the real speed test servers too
[22:28]
mnathani_I probably shouldnt test from 2 hosts simultaneously hitting brycec's vps [22:33]
mercutiohaha
yeah he'll have 100 megabit
nothing has been extremely bad though, so it's diff from the packet loss issue
[22:35]
brycec(and a portion of that is constantly in use :p) [22:36]
mercutioheh la.meh.net.nz has done like 1tb of data in the last 100 days
i thoguht that was quite a lot
in+out combined
oh i've lost it now because i updated the kernel
[22:36]
mnathani_is that your dedicated box at ARP ? [22:37]
mercutioit was more like 900gb i think
it's a vm on dedicated box
[22:37]
mnathani_so gigabit pipe [22:37]
mercutioyeh
but yeah there's a reason why i don't normally do 200mb tests ;)
[22:37]
mnathani_atleast not automated ones
with smokeping
[22:39]
mercutiowell with smokeping it's 200k [22:39]
brycecbrycec notices a few telltale spikes on today's ARP bandwidth graph ;) [22:40]
mercutioand it's pretty flat except sometimes gets better/worse
but stays good/bad
yeah i wonder if up_the_irons will see it on his graphs
[22:40]
mnathani_my memory usage on windows desktop is at 88% and I cant seem to figure out whats using up 21 Gigs of Ram [22:40]
brycecit's relattively minor on mine, so almost certainly unnoticed. [22:40]
mercutioi think some people are doing huge amounts of bandwidth so maybe it'll fade into the background
mnathani_: haha
[22:40]
mnathani_and I dont want to reboot [22:41]
mercutioi'm not even doing anything on windoms and it's using 5.7gb
two chorme windows, task manager, crystal disk mark, steam, a command prompt, skype, and foobar2000
[22:41]
brycechm I do have an arp metal box... I should set something up on it. [22:42]
mercutiochrome is runing a lot of processes for two windows [22:42]
mnathani_almost 90 chrome processes for me [22:43]
mercutioskype is using 174.1mx of ram
mb
how do you find out the number of chrome processes?
[22:43]
mnathani_I counted
in the details
[22:43]
mercutiogod
i don't want to do that :)
[22:43]
mnathani_probably a better way
using grep | wc -l
or something
if you have cygwin
[22:43]
mercutiochrome://memory-redirect/
i don't have cygwin
i have synergy
i just use linux :)
chrome is using 1.3gb of private virtual memry
[22:44]
mnathani_c:>tasklist | grep chrome | wc -l
85
[22:44]
mercutioand 700mb of private memory
sj ;,ddk
oh sweet
[22:44]
mnathani_when I installed git I chose the option to install unix tools in my path [22:45]
mercutio% cat tl | grep -i chrome | wc -l
16
i cut and paste it :)
i had tasklist
actually i piped it to a file, then used notepad on it and cut and paste it
cat tl | awk '{ print $5, $_}' | sort -n
will show you what's using the most ram
well "cat tl" can be tasklist for you
[22:46]
mnathani_'awk' is not recognized as an internal or external command [22:48]
mercutiogawk?
mawk?
nawk?
[22:48]
mnathani_c:>tasklist | gawk '{ print $5, $_}' | sort -n
-nThe system cannot find the file specified.
[22:48]
mercutiohaha shit
just cut and paste it to linux?
or bsd
same diff :)
hell even a mac will work
windows is so behind
[22:48]
mnathani_there is probably a powershell equivalent [22:49]
mercutioi wonder what sort does
you maybe to output to a file and use the sort command
yeah sort doesn't take standard input
and you can tell it how much memory to use
[22:49]
mnathani_task manager has a listing with memory
but its not accurate
[22:50]
mercutiotasklist said 239,852 kb memory
and task manager said 171 mb i think it was
for skype
[22:51]
mnathani_lol
turns out it was VLC
that had massive videos on pause
[22:51]
mercutiodon't you wish for "top"?
was it prebuffering the whole file or something?
[22:51]
mnathani_it had a portion in memory
video was 48 gigs
[22:52]
mercutio23gb though? [22:52]
mnathani_I was doing some video rendering earlier as well
process might just have been hanging in the background or something
I am running the curl speed tests on a Toronto VPS
[22:54]
mercutiowill be interesting to see how it compares [22:55]
mnathani_not sure what the max speed of this vps is [22:56]
mercutioyou shall find out :) [22:57]
mnathani_wow [22:57]
mercutiocomparative is intersting anyway [22:57]
mnathani_12.7 MB/s from LA [22:57]
mercutioi got more than that before
from nz
you'll hvae like 100 msec ping or something
[22:57]
mnathani_you had gigabit though [22:57]
mercutioso do you that menas
means
[22:57]
mnathani_100 megabits / sec = 12.5 megabytes per second [22:58]
mercutioyaeh but 100 megabit physical ethernet never does 12.7mb/sec :)
it can sometimes do 11mb/sec
[22:59]
BryceBotThat's what she said!! [22:59]
mnathani_http://pastebin.com/Bt9cXc6E [23:00]
mercutiohmm
that's actually lokoing really good
[23:00]
mnathani_UK is a bit low [23:01]
mercutiotthe uk server doesn't haev good provider redundancy
it's just using level3
and some people don't have good connections to level3
[23:01]
mnathani_but thats expected going over the pond [23:01]
mercutionot really
it's way closer than nz
japan is going surprisingly well
what speed do you get from that vps?
[23:01]
mnathani_its a customer vps
I never really tested
[23:03]
mercutiooh that route is over any2ix too
it's any2ix/he.net
[23:03]
mnathani_would you be surprised if route to jp from that vps woas out cogent [23:04]
mercutioit's what japan uses to get to it that matters
wow
the forward route is ntt->cogent too
it's going via seattle
what location is slc
oh is that salt lake city?
it's any2ix from nz too
but via northern side
hitting palo alto
[23:04]
mnathani_http://pastebin.com/XavbkEi5 [23:07]
mercutioyeah the reverse route is basically the same
can you try to the nz server again?
i just upped the transmit window size
[23:08]
mnathani_thats the one with the IP [23:09]
mercutioyeah [23:09]
mnathani_instead of hostname? [23:09]
mercutioit's emerald.meh.net.nz
if you want a name for it
bad habit :)
i know what it is :)
[23:09]
mnathani_http://pastebin.com/HSw0G1bL [23:11]
mercutioslightly better
actualyl about 50% better
a bit less
but damn when did he.net get good? :)
[23:12]
mnathani_just tested downlaoding to myself from the toronto VPS : http://pastebin.com/9cYizZzR [23:14]
mercutionice
maybe you should bounce your connections via it haha
cable can often have inconsistent speeds
comcast, cox, verizon are all like that
they all have sub par routing
bulk/cheap
but from the little i read about teksavvy it sonuded like they wanted to be better than other people
so maybe they will peer more and more and upgrade their network etc.
but why some any2ix fast and some slow is bizzare
it doesn't seem like an arp issue
but it's hard to know for suure
[23:14]
mnathani_I consistently max my connection while using usenet [23:19]
mercutioto a close server? [23:19]
mnathani_they use a pool and geo dns [23:19]
mercutiojoh yip
oh yip
well any2ix is ages away from torronto :/
[23:19]
mnathani_haven't run a trace to them [23:19]
mercutiobut i can't for the life of me figure why it would be slower [23:20]
mnathani_http://pastebin.com/ddGVk8mY
thats one of the usenet servers
not sure if they do anycasting too
[23:21]
does appear to be an anycast address [23:27]
mercutioyou suure? [23:30]
mnathani_pretty sure [23:30]
mercutiothat's going to east coast for me it seems [23:30]
mnathani_I got low ping from ARP
and Toronto
and IL
[23:31]
mercutioweird i get high ping from nz
sj to la to dc
connects to hwng in sj
[23:31]
mnathani_how do I get mtr report to be 2 times as wide [23:32]
mercutiolots of things are high ping here though
like ebay
[23:32]
mnathani_so hostnames are visible [23:32]
mercutioand amazon
amazon is around 230 msec
i dunno why it's so high
ebay lost hop it shows is phoenix at 150msec
but it seems to load slow
actually it's better than it used to be
i thought sites like that were meant to be heavily optimised thogh :)
[23:32]
mnathani_try doing a dns lookup [23:35]
mercutiook [23:35]
mnathani_see if you get a different ip [23:35]
mercutioon what name? [23:35]
mnathani_news.newshosting.com [23:35]
mercutionope
same location
news.newshosting.com is an alias for news.iad.newshosting.com.
i assume that is location
[23:35]
mnathani_and 2 ips? [23:36]
mercutioyeah
.28 and .29
same ip you had
[23:36]
mnathani_is there a way to use looking glass
to check as path
and determine anycast
or perhaps they just peer a lot
[23:36]
mercutioBGP.as_path: 9559 17746 4610 4826 12989 29798
it says dc from arp too
and 60+ msec
that's not low ping :)
well it's low ping for east coast
but it's still east coast
[23:37]
mnathani_hmm .. so its not anycast
?
[23:39]
mercutioyeah
but it's nearish to you
[23:39]
mnathani_I think they have different dns names for europe [23:40]
mercutioyeah could do [23:40]
.... (idle for 17mn)
brycecSo I recently started getting random hits for /data/v3/config.dat from some misconfigured Maxthon browser thinger. I got tired of firewalling the random IP's, and I have plenty of bandwidth to spare, so I pointed them at a 48GB (sparse) file instead. The last request gave up around 300MB MWUAhahaha [23:57]
mercutiohaha what
crazy
wouldn't it be better to feed it out really slowly?
[23:58]
brycec46.166.186.230 vps3.cobryce.com:443 - [20/Mar/2015:23:32:31 -0700] "GET /data/v3/config.dat HTTP/1.1" 200 365837887 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.4.3.4000 Chrome/30.0.1599.101 Safari/537.36"
mercutio: but that's harder
It's supposed to be some "safe browsing" file from what I found Googling it.
So I'm possibly overwhelming the hapless computers. :D
[23:58]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)