#arpnetworks 2014-10-18,Sat

up_the_irons

plett: :)

mercutio

but i still think that if being on ams-ix should also be on linx.
which means probably doing niether

up_the_irons

haha

mercutio

it's easy to see the sense in peering locally.

plett

mercutio: Shouldn't be any harder. Both have a set of L2 resellers who will resell connections
https://www.linx.net/join/linxanywhere.html

mercutio

but peering remotely gets complicated quick, with traffic taking funky routes.

plett

Yeah, remote peering probably doesn't make sense for most networks

up_the_irons

it's all going over the internet anyway... ;)
not like there's a secret tunnel from ams to lax that will somehow make things *that* much faster

mercutio

plett: basically if you do it, you want to start doing it in like 4 or 5 places.

[04:54]

up_the_irons

sure, traceroutes will be shorter, but that probably don't really fool anyone that actually cares

[04:54]

mercutio

he.net got really screwed up when new york had flooding.
so having "better" providers can help...
thing is that was once, ages ago.
and it wasn't down, just extremelly lossy/congested.
who are GT?
GTT?
that's level3 isnt' it?
oh, nlayer.

[04:54]

plett

mercutio: I work for a .uk based network and we don't have any network outside of the country. We peer at the two london IXPs, LINX and LONAP but no others. We've looked into L2 relaying to get to AMS-IX and DE-CIX but it doesn't make sense from either a commercial or network point of view

[04:57]

mercutio

plett: isn't dsl terrible there anyway?

[04:58]

plett

We try to make it not horrible :)

[04:58]

mercutio

and there's like one cable provdier that you probably have to pay to send stuff to
i have a friend who used to live in UK
peak time congestion, adsl1..

[04:58]

plett

Yeah. All the regional cable networks got bought out and now operate under the Virgin Media brand (which is owned by Libery Global, who I believe do some US cable too?)

[04:59]

mercutio

it seemed surprising to me. you'd think he lived rural or something. but he lived in cambridge.
and he refused to deal with virgin, because he couldn't get them to stop harassing him about the previous owner of house.
and wouldn't deal with him because he wasn't account holder.

[04:59]

plett

Hah. You say that. Cambridge is a surprisingly small place, and remarkably out of the way of anywhere people would want to run cables

[05:00]

mercutio

oh.
i thought it was a fancy tech town
i don't know much about the uk tbh

[05:00]

plett

It has a university which does fancy tech stuff, and therefore has many small companies nearby who do tech stuff, but not much in terms of network infrastructure

[05:01]

mercutio

it seemed to be like what new zealand was like when we had ATM backhaul.
surprisingly new zealand seems to be getting some gigabit fibre now.
but there's very little local content.

[05:03]

plett

And yes, all the copper phone lines in the UK are run by British Telecom (BT) which is the company formed in the 1980s when the phone network was privatised.

[05:03]

mercutio

he moved to the US, and now has congestion issues again.

[05:04]

plett

BT are actually being fairly good at rolling out new technology. 80Mb/s VDSL2 is now available to something like 2/3rds of the population, with the majority of the rest getting 24Mb/s ADSL2+

[05:04]

mercutio

but much faster burst speed.
oh nice.
yeah that's not bad.

[05:04]

plett

There are still some places which can only get 8Mb ADSL1, but that ADSL1 network has effectively 100% coverage

[05:05]

mercutio

i think vdsl is good enough for most people atm tbh
1 megabit upload on adsl kind of sucks for skype video and uploading photos and stuff.

[05:05]

plett

They also have a GPON fibre-to-the-premises product which can go up to 300Mb/s. That is technically available anywhere you can get VDSL2, but at an extortionate cost. There are a small number of places which have it instead of VDSL, and it costs normal broadband prices there

[05:06]

mercutio

ahh cool.

[05:07]

plett

There is also a trial of G.Fast coming soon, which will in theory give 100Mb/s over copper, basically by moving the DSLAM to the top of the pole in the street or under a manhole cover

[05:07]

mercutio

i reckon fast copper is the way to go atm tbh
it's just so much cheaper/easier.

[05:08]

plett

Only because it's already there

[05:08]

mercutio

i mean i know fibre is technically better. but it's expensive to instal.
true.
i suppose it's a question of when not if.
and whether now is a good time to dig up roads etc.
new zealand's moving to fibre.

[05:08]

plett

Not really any more expensive than copper. Yes, you need a fancy cleaving/fusing tool to put the connectors on the end, but installation engineers already carry ££££ of test gear in their vans

[05:09]

mercutio

with speeds of 30/10 or 100/50
but some people are doing fancy plans now already
of like 1000/20 and 200/20
the upload speed sucking.

[05:09]

plett

Yeah. Even on the fibre-to-the-prem, it's a maximum of 330/30
VDSL is 80/20 or 40/10 here
So yeah, all sucky upload speeds

[05:11]

mercutio

the upload speed sucking seems kind of random
i think it's so they can charge more for businesses.
it's 100/50 on 4g.

[05:11]

plett

There are good technical reasons why it's done like it is
But it does suck for people who do want good upload speeds
On copper you have a maximum frequency which the line can cope with, and you split it up so some frequencies are for upload and some for download. Most people want more download, so that gets way more frequencies allocated to it

[05:12]

mercutio

yeah but on fibre plans gpon can't handle as much upload
but it can handle 2:1 or something ratios fine.

[05:15]

plett

And it's tricky to do different things for different lines because they will crosstalk with each other. So if the exchange is pumping out lots of power on a frequency for one line, it can't also be listening on that frequency on the next line because all it would hear is itself

[05:15]

mercutio

yeah i know why it's the way it is on dsl
adsl annex m improves over adsl2+
and vdsl is basically fine.
but i mean why have 1000/20 fibre :)

[05:16]

plett

GPON is a little bit different. I don't know how they do it in other countries, but here, it's a single fibre at 2.5Gb/s back to the exchange from the GPON node
Each GPON node handles (I think) 48 end user fibre lines, and they are purely passively split and everyone receives everything
And they then use TDM to timeslice the upload and download channels so everyone gets a share

[05:16]

mercutio

it can balance stuff
so each user gets a fair share

[05:18]

plett

Assuming a full GPON node with all 48 lines used, each user would get 52Mb/s to play with (2500/48)

[05:18]

mercutio

yeah i don't think they usually run them anywhere near full

[05:19]

plett

They don't here, because the takeup levels are very low due to the price

[05:19]

mercutio

and i think it's like cable in that it's shared frequencys or whatever, but you can oversubscribe

[05:19]

plett

It's only a single frequency in GPON (well, two, one for up and one for down) and then TDM timeslicing so everyone can use it

[05:20]

mercutio

it's about twice the cost of naked adsl to get 1000/20 fibre here in areas that support it
i think
for wholesale cost.
maybe a bit less. it's not outlandish though
i think the timeslicing can be dynamic
i'm not sure how it works though

[05:20]

plett

Yes, it can by dynamic
If it notices that one node always uses its timeslots for sending data when another node doesn't, it can allocate that one more timeslices on the fly
And download, every node receives every packet and looks at the headers to see what's for it, so the head end in the exchange can decide who gets more or less bandwidth

[05:21]

mercutio

cool.

[05:25]

plett

The real problem is backhaul costs for wholesale lines. As it's all run by BT, an ISP buying wholesale access has to either have a presence at every exchange or buy backhaul from BT

[05:25]

mercutio

and backhaul costs heaps i imagine

[05:25]

plett

A small ISP like us can't afford to have a presence at the many hundreds of exchanges, so has to buy backhaul

[05:26]

mercutio

can you do a buying partnership?

[05:26]

plett

It ends up costing about £50 per Mb/s on 95%ile billing after all the billing tricks and incentive rebates etc
(which is crazy, when I can buy internet transit for £1 per Mb/s when buying in quanties of 1Gb/s)

[05:27]

mercutio

or collective.
wow

[05:27]

plett

Yeah

[05:28]

mercutio

so you don't want users doing more than like 60gb/month?
is there usage based pricing for users?

[05:28]

plett

That depends on the ISP, different companies manage it in different ways
We do usage based charging, yes

[05:28]

mercutio

ahh

[05:29]

plett

Others do an "unlimited" service and rely on network congestion to limit their users throughput

[05:29]

mercutio

how much does it cost to do your own backhaul?
i imagine you can't really allocate each unlimited user a megabit of bandwidth at that kind of costing.
i probably do over 1 megabit 95th pernctile at home
but it's hard to know
actually i might not

[05:29]

plett

Not too much for the backhaul itself from exchanges, they are well cabled places after all

[05:30]

mercutio

i dunno if i stream movies more than 5% of month or not :)

plett

The real costs are in having a presence at the exchange, a bank of DSL linecards and people on call to go and swap kit out

mercutio

oh i thought this was for fibre

plett

The last time I looked, the average per-user allocation in the UK was something like 100Kb/s

mercutio

your own dslams seems silly idea
but having cheaper backhaul sounds smart

plett

Yes. If we were starting to PoP exchanges now, we'd ignore the copper completely and just buy fibre

mercutio

we used to have 48kbit per user average allocation enforced here
or something like that
it went up a bit
but it meant there was packet loss in peak hours freuqently.
and that seemed to be what my friend had :)

plett

Then the only kit we need in the exchange is an ethernet switch to take 1Gb/s LX fibres from BT, and aggregate lots of vlans back to us

[05:32]

mercutio

yeah sounds sensible.
but you have to do that per exchange
not even per city?

[05:32]

plett

Depends

[05:33]

mercutio

and an exchange is like 5000 people?

[05:33]

plett

For fibre, they've been quite clever, and lots of the small exchanges don't have their own fibre nodes at all, they all backhaul to a regional "mega-exchange"

[05:33]

mercutio

oh that's good

[05:34]

plett

So it's not every exchange that you need to get to
There's something like 5600 exchanges in the UK. Population density is nowhere near even though, so fibre only covers about 500 exchanges but covers 2/3rd of the population

[05:34]

mercutio

ahh ok
still, 500 exchanges is a lot to run fibre to

[05:35]

plett

Of those 500 exchanges, you'd probably need a presence at maybe 300 to take fibre everywhere
Yes. There's only 20 people in my whole company. I don't want to have to manage kit in that many places all over the UK :)

[05:35]

mercutio

yeah.
so it makes sense to band together
for smaller providers.

[05:36]

plett

Yes and no. On the retail side, we're all competitors of each other

mercutio

yeah.

plett

It would be like Ford and Honda sharing a factory to save costs

mercutio

but when it's giving money to bt..
not really.
it's like ford and honda lobbying together to reduce petrol taxes.

plett

True

mercutio

because they don't want to invest in making electric cars.

plett

Hah

mercutio

lots of the smaller isp's here have reasonably friendly relations.

plett

I've got to run, I've got to paint a wall