hi I have the super-basic "small" VPS plan is my speed limited? I'm trying to download a dataset from my VPS to here - and it seems really throttled to like 1M jm iperf seems to bear it out :/ jm|laptop: where is "here" VPS should be able to hit around 100mbit here is my home fibre dual-xDSL east coast? west coast? europe? asia? oh. UK k icmp_req=1 ttl=44 time=166 ms v4 or v6? both / either scp ipv6 and http ipv4 to prove it wasn't ssh overhead and iperf ipv4 it just stalled again >.< try the arpnetworks speed test which is on gigabit: http://arpnetworks.com/100mb.bin that's coming at like 251K/s although I still have my other scp going on no obvious loss, slows down cross-Atlantic so I have a colo in Manchester pulling 1.7M from that .bin, ramhost London pulls ~500K ramhost down to 120K now I wonder if this is a peering thing moar routes could be or transit thing seems to go myISP > L3 LON > L3 NYC > nlayer > us.pktxchng > mzima who is nlayer? nlayer is fine check other direction some of the any2ix peers can be kind of slow with transit back atrato i've noticed be slow before for example mzima > big pause > nlayer > L3 LA > L3 SJ > LA NYC x3 > L3 LON > myISP it really bounces around L3 :S that really should be fine unless your isp has congestion at level 3 18 level3.restless.thn.aaisp.net.uk (195.50.95.67) 145.879 ms 146.302 ms 146.025 ms my isp peers L3 9. ae-82-82.ebr2.NewYork1.Level3.net 0.0% 6 138.8 139.0 138.8 139.5 0.3 ae-72-72.ebr2.NewYork1.Level3.net like that etc? yeah. With an ominous * * * in between mtr is good at skipping over that :) 14 ae-91-91.ebr1.NewYork1.Level3.net (4.69.134.77) 146.221 ms ae-61-61.ebr1.NewYork1.Level3.net (4.69.134.65) 147.240 ms ae-91-91.ebr1.NewYork1.Level3.net (4.69.134.77) 146.204 ms 15 * * * 16 ae-59-224.csw2.London1.Level3.net (4.69.153.142) 147.207 ms 147.286 ms ae-56-221.csw2.London1.Level3.net (4.69.153.130) it does that here on another route not to arp from a uk vps I guess its sub-Atlantic node :) with like two dead hops between london and newyork you have the same mpls exit path thingy where new york shows high ping like london 150ms ? yeh it's the same on london and new york though rather than showing the latency from london to new york you're not using openbsd or something are you? ..... where? openbsd's tcp/ip stack can be kind of bad for perfomrance transatlanttic my router runs OpenBSD takes longer to ramp up only seems to increase window size with scaling router doens't matter it's only end point then no, Debian yeah debian defaults should be fine even so ARP /doesn't/ throttle? nah it doesn't throttle and level3 usually has good speeds unless someone doesn't buy enough level 3 transit or such just coincidence that my monitor is showing almost /exactly/ 1M :) you're at like 150 msec altency? I'm not at liberty to disclose my altency well guessing that 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 166.583/167.351/167.918/0.735 ms your window size would be about 160k which isn't generally a size anything will cap at when my scp finishes I might try another iperf ohh it's not old scp is it? no ok and I tried http also! (see above) ssh used to have window size issues ahh ok well assuming you have more than 128mb ram [02:44] scp ipv6 and http ipv4 to prove it wasn't ssh overhead linux should pick big enough tcp window size ipv6 was the same? yup scp is ipv6 ok that's really bizzare i've often found ipv6 and ipv4 give completely diff performance sometimes one is like twice as fast as the other doesn't make a lot of sense yeah unless the whole ipv6 topology is different which doesn't make a lot of sense oh it is often i think i dunno i even reproduced that with the same provider being used for ipv4 and ipv6 it may just be load balancing tkaing diff path were they cheap? suppose? i dunno maybe the employed a different ipv6 gateway well arp dose does s/different/less\ congested/ i can't remember which was faster sometimes people don't shape ipv6 properly etc too and shape ipv4 hm 722141184 98% 407.45kB/s 0:00:29 ok i need to try again :) hmm that's weirder still weird both are going slow atm I had two scp making that 1M dude i'm getting terrible speeds compared to normal from this host too it's a friends connection but it usually does faster and one has finished so I expected other to take 'spare' bandwidth but it's 500k wtf 587956224 80% 385.63kB/s 0:06:10 like something is making 500K per connection :S i got 647k/sec with ipv6 and 509k/sec with ipv4 for a 10 mb file with my normal provider getting 2267k/sec average 130 msec ping versus 138 msec on the slow [ 3] 0.0-10.2 sec 7.12 MBytes 5.83 Mbits/sec ^^ my other colo and 139 msec with ipv6 on the slow hmm % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 10.0M 100 10.0M 0 0 509k 0 0:00:20 0:00:20 --:--:-- 504k and that's with any2ix peer (and my values include wireless overhead to the laptop) it's prob this stupid upstream :/ oh well wireless doesn't really slow things down unless you're far away part1 of my dataset is here so I can grok it oh there's packet loss to last hop that's really weird thanks for clarifications :) well it can limit you, but doesn't slow down unless you're past bandwidth and it's giving high pings locally basically wait what? how can I check my bandwidth limit? i meant on wireless like if your wirelss can do 20 megabit oh and you're pushing 25 megabit it'll slow it down to 20 megabit I have 5GHz N but if you're pushing 5 megabit then it'll be same speed on wireless and ethernet it doesn't tend to give loss it can, but usually bandwidth is heavily reduced before that pings spike etc [ 3] 0.0-10.0 sec 126 MBytes 106 Mbits/sec not bad for wireless :) yeah so shouldn't be an issue cos you have dual dsl oh how does your dual dsl work? I pull about 50M from my dual xdsl do you have to do multiple connections to max it out? or is it bonding? bondage cool mlpppp :) nth packet downstream oh no that's better so it doesn't split each packet half between each does that give out of order issues? aiui it balances based on latency from LCP data yes; but TCP/IP is designed to deal with that :) err tcp/ip sucks with out of orer :) perfomrance wise it's more designed to cope than deals well with :) worksforme dot com heh if I had a more expensive router at this end I could handle it better but obsd router doesn't do too bad a job i'm surprised it looks at lcp data i wanted to play with bonding some day i was trying to think up novel ways to do it better my ISP is pretty switched on with this stuff I get per second graphing and stuff any idea how they're doing it? i was playing with ppp compression a while back it works fine until you get packet drops and it seems to make more sense on upstream than downstream for adsl they made their own router oh wow ok that's good to know i didn't think anything was doing those kinds of smarts normally http://wiki.aa.org.uk/index.php/FireBrick http://www.firebrick.co.uk/products_2700.php this http://www.firebrick.co.uk/ or indeed this :'/ oh i read something on this site before about some special ppp server i think which part of the PPP sever? it's not that clever :/ To bond downlink on DSL lines you need the ISP to send traffic down multiple lines. The FB6202 LNS can do this on multiple L2TP links with per link speed controls. Bonding downlink does not need anything at the receiving end as the packets simply arrive on one of multiple simple broadband routers and go on to the LAN. It is obviously a good idea to use an FB2700 at the customer end to also bond uplink though. it's not acutally specific to the termination device but the sending ppp Because bonding works on a per-packet basis, even a single session can make use of multiple lines. However, it is worth bearing in mind that whilst IP does not guarantee packet order, most TCP stacks will have trouble if you bond more than about 4 lines or if the lines are very different latency. that's multihoming nah that's bonding dsl the former re: the latter, I only have two :) it's both under bonding yeh but it does out of order a bit ok and the latency is pretty much similar because it comes to my house on the same 6-string drop :) it's actually really easy for it to be different if lacp etc is being used yup again: wfm.com it'll matter less for big packets though I have poor-mans bonding at this end an Alix mpath routing anyway, i'd try a udp test whence? iperf -u whence? which'll tell you if there' any packet loss and it'll tell you how much is out of order if there's loss it's prob reason it's going slow [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.25 MBytes 1.05 Mbits/sec [ 3] Sent 893 datagrams [ 3] WARNING: did not receive ack of last datagram after 10 tries. if there's no loss and no out of order your isp may be doing something weird with tcp/ip traffic you need iperf -s -u on server iperf -c -b 2m on client where client uploads to server well 1mbit is fine too [ 3] 0.0-10.0 sec 1.25 MBytes 1.05 Mbits/sec 0.406 ms 2/ 893 (0.22%) you can use -i 1 as well on server to list per second hmm hmm indeed :) maybe try -l 512 on both to send smaller packets and i think it's -t 30 to do 30 second test [ 3] 0.0-10.0 sec 1.25 MBytes 1.05 Mbits/sec 0.225 ms 0/ 2561 (0%) [ 3] 0.0-10.0 sec 1 datagrams received out-of-order that seems fine hmmm I know: except for the speed :) that's cos you have -1m -b 1m try -b2m hm udp sends at fixed speed but generally speaking if udp is fine at low speeds without loss then the link is going ok [ 3] 0.0-10.0 sec 1.25 MBytes 1.05 Mbits/sec 0.216 ms 0/ 2561 (0%) [ 3] 0.0-10.0 sec 1 datagrams received out-of-order something really only wants 1Mb/s :) -b 2m on the client I did that oh are you sending from your dsl to arp? you have to do arp to dsl $ iperf -u -l 512 -b2m -c arp yeh do it other way around cos client sends to server how? I have NAT? JUST KIDDING port forward? my ISP is better than that :) you have /64 ? I have /26 and /48 actually you can do ipv6 with iperf but you need to set -V and if you have differnt default udp return addresss than incoming address you need to bind on the server to the incoming address -B i think it is it seems much more complicated when desciribing it [ 3] 0.0-10.0 sec 1.25 MBytes 1.05 Mbits/sec 0.224 ms 1/ 2562 (0.039%) [ 3] 0.0-10.0 sec 1 datagrams received out-of-order other direction something somewhere really seems to be restricting me to 1M :) anyway: off to research my dataset thanks for your time mercutio :) i'll just it mikeputn1m: lol nice iron maiden references earlier... up_the_irons: do you normally allow /23s into your network now? mercutio: if it's from a peer soon, i'll have everything, with the new router brb new router \o/ up_the_irons: trit's not a peer is it? everything would be good, it's accepting a way worse secondary route